Oddbean new post about login | logout Maybe instead of fighting relays that try to limit abuse with limited tools that they have, we should discuss the issue? What's your ideal relay? Has no limits on number of concurrent connections, subs, requests per second and bytes/events served?
I have 100s of what should have been separate subs packed into 10 subs right now. Max number of filters, max number of kinds in each filter, max number of follows (usually 2k), max number of etags, etc. I am hitting several limits at the same time now. I also need AUTH for separate accounts in the same connection, which doesn't seem feasible without a massive change on how Nostr filters work.
So basically: - higher limit on number of active subs - then other filter/etag/pubkey limits could be handled by splitting into several subs - higher limit on number of connections ?
The hard part is convincing every relay to do a 10x on their sub limits. Many are still forcing Max 10-12 subs. Still doesn't solve the auth problem. So, it might not be worth doing it and going straight into Tor.
Auth is solved by higher limit on concurrent connections?
Yes, up to 10 separate connections coming from the same IP
It's not about convincing, but about improving the rate limiting system. The actual load is roughly equivalent whether you send 1 sub with many filters or 10 subs with less filters in 1 conn or 10 conns etc - you need that data, you will request it one way or another. Rate limiting at sub/conn level is just a dumb way to evaluate and limit the load that client is creating. Better rate limiting would take into account the actual number of indexes touched or table scans performed or filter conditions matched for incoming events and events retrieved and served on per-IP basis, then number of filters/subs/conns wouldn't matter. Could you please publish an example list of subs that you'd like to have persistently active, so that we could have some rough idea of the potential load it could create? Also for pre-sync state and after-sync state - the sync load probably differs greatly from the "watch-for-updates" state? It's been painful to build the current (pretty dumb) event sync for npub.pro, so I agree things need to be improved in this area.
Also, batching filters might result in unintended expansion of result set - instead of 2 filters {npub1,kind1} and {npub2,kind2} sending one {[npub1,npub2],[kind1,kind2]} may return much more than was needed. So it seems like relays should allow more of precise (smaller?) filters. Yep, batching like I do creates much more complicated queries for their indexes to deal with. It doesnt make any sense.