I pause whatever I'm listening to while I talk to co-workers.  Conversing normally lets them know I am giving them my full attention. Leaving them in sends the message that I am just here for what I need.  No small talk please.