Npubs we've interacted with are probably more "trustworthy" then those we haven't. Replies, quotes, reposts, likes, zaps, etc.
That’s good. I’m a fan of “this npub is not a spoof” verifications being assigned “by hand” from peer to peer. But managing such a list is a silly task. So “interaction numbers” like your idea could be integrated by client to help prompt users to keep verifications “up to date”. Some mix of automated and user curated seems best… but still… curation is a pain.