Leave the pauses in, if anything only edit out awkward mouth noises and replace with room tone.

Other mixing considerations:
- De-ess both guests.  Nothing will make me turn off a podcast faster than harsh sybilance
- Match overall loudness of both host and guest using EQ/compression.
- Consider editing out or automating background noise with gates.