All office phones should have a button that allows you to turn off the voicemail light without actually logging in and deleting the voicemails.

This post brought to you by the Post-It that lives over my voicemail button. 

 Use a postit.