I have ADHD and migraines, and noise- especially "background" or "white" noise - is extremely distracting to me when I'm having a migraine or when my adhd meds aren't working.
Fans in particular can ruin a night of sleep for me, and are very distracting. Conversations 20 feet away about a slightly interesting thing are distracting as well.
I have to turn music down or off while driving when I get close to my destination or when traffic is heavy.