I leave them in and talk normally. If they ask, "I'm not listening to any music,I'm focused on this conversation"
I think it might be considered not-rude/normal now lol. Not sure.
But why are they left in? To not lose them? Let's say you know you're going to be with a friend for at least a couple of hours. They stay in? At an all night party? A dinner party? Under what circumstances do they come out? Lol