Great take! Filtering signal from noise starts at the Relay level indeed. 

Layers on top that I see: 
1. .Communities built on Relays (NIP-29 groups)  →   Using these as a specific lens / filter instead of your blunt Following List
2. Following Lists  →  For Web of Not Spam only, not for Trust
3. Specialized Verifiers  →  Code verifiers in Zapstore for example