I pause whatever I'm listening to while I talk to co-workers.  Conversing normally lets them know I am giving them my full attention. Leaving them in sends the message that I am just here for what I need.  No small talk please. 

 I was thinking more of a social setting, although this may be grey area-ish, maybe 🤔