Maybe have them sign something with Nsec to prove they own the keys?