You can do that. Gossip manages to get relays and find events for the intersection of all your lists, not just your following list.