Is it not showing "Delete this identity?"  I don't often test it in public-key-only mode, so if it is not I'll make a fix.