It's only distracting if you let it be.

You say it yourself: it's at the back of your mind. So try and get it out of there.

This is my wee lil trick:

When I consume or contribute to a community's fire-hose, I make sure I do it in a block of time. After that, a quick off-line activity (grab my guitar for 1 minute, play with my rubik's cube, go for a coffee, etc) will reset my brain and allow me to re-focus on something else.

For instance, right now, I've got my sandwich next to me, and I'll eat it as soon as I'm done responding here. After that, it's straight back into working on the presentation for next week's brand meeting :)

tl;dr - Social Networks can be a distraction.