That's the face! 
I have a hard time recognizing faces. My sister is actually "face blind", and recognizes people by mannerisms and speech.