Oddbean new post about | logout
 I said it before and I'll say it again and again: we need ideas to fix the huge traffic issue with noste. 

My computer starts downloading events with 20 MB/s whenever I open a new nostr tab. That's unacceptable. If we want the world to use nostr (not just the US), clients need to work in a low bandwidth environment.

Think smaller. 
 Only PC or phone as well? 
 Mobile data gets completely rekt with nostr. Only a tiny fraction of the world has unlimited data. 
▲ ▼
 Even then, 'Unlimited' is pretty much always limited in some way. At least in the US. It's a deceptive term here. 
▲ ▼
 slowed down unlimited 😅 
▲ ▼
 I got rekt last week, ran out of mobile data while travelling, cost me 40 bucks to extend my data on the fly. Then got rekt again 3 days later.

Its unsustainable  
▲ ▼
 So scaling nostr and saving bandwidth can both be part of the same effort.  Primal doing good work there, but needs to be more appetite in the wider eco system to make it reality. 
▲ ▼
 We will remember this day, as the day the Note size wars began.
nostr:nevent1qqsy7cx4nnwppugdkewnql7x58mnt89yxxuczce9xrxqhx64js6zghcpr4mhxue69uhkummnw3ezucnfw33k76twv4ezuum0vd5kzmp0qgs9pk20ctv9srrg9vr354p03v0rrgsqkpggh2u45va77zz4mu5p6ccrqsqqqqqpes2vjx 
▲ ▼
 Personally, I have to set image disabled and limit my relay usage by using aggregator relay (1 or 2) as "read relay" and other relays for write only in Amethyst. 
▲ ▼
 You're right, thank you 
▲ ▼
 My GPU sounds like a jet turbine when I open Primal. How can this be solved? 
▲ ▼
 I think it would be a start to let people better choose what they see. I see so much content from people I don't want to see. I would only see content from people I explicitly follow (no global bullshit, nothing from the people they follow, etc). I literally only want to see posts from those I 'follow.'

This isn't a broad solution, but it would certainly lower the amount of data my connection needs to download. Currently, well over half of what I see on Nostr is shit I don't even want to see.

I'm sure there are other preference settings (filters) that could greatly reduce data usage. This seems like low hanging fruit, but I'm not knowledgeable enough on Nostr to know for sure. 
▲ ▼
 I’m with you brother.

For me, the biggest lure to Nostr is that you are in control of what you see. 
▲ ▼
 I conducted an experiment to demonstrate the point in my other comment. On average, I would NOT see 18 out of every 25 posts that are loaded on my feed (using the most aggressive filter offered by the client - which clearly isn't much of a filter). I don't understand all the underlying mechanics, but allowing me to only see the 7/25 posts I want to see has to save data. I can't see how it wouldn't. 
▲ ▼
 which client? 
▲ ▼
 Have you tried my thing?
https://github.com/Yonle/bostr

Try clear your relay list in your client and try to only connect to a bouncer. You could connect mine, which is public and there are peoples using it: wss://bostr.lecturify.net

List of relays that are being proxied / bounced is available at https://bostr.lecturify.net 
▲ ▼
 this is probably compatible with lol but incompatible with mom because mom looks at the IP when rate limiting. 
▲ ▼
 You could still try to manually connect and set it as "write relay" in client if broadcasting in bostr does not work properly. 
▲ ▼
 Nostr needs a better experience on mobile overall 
▲ ▼
 @jb55 seems to be on the right path to fix this 
 What's the approach? 
▲ ▼
 local relay + negentropy sync. only pull notes that you don’t have. 
 Where can I read about negentropy sync? 
 Rust nostr supports that too 
▲ ▼
 strfry too afaik! 
▲ ▼
 This is an insanely good writeup of it by its author

https://logperiodic.com/rbsr.html 
▲ ▼
 simplest solution is proxies. bostr, blastr, .. not sure if blastr is reading from every relay though. if it is not then it is not reducing read traffic (which is the most traffic).

another one is aggregating relays that collect notes from a lot of relays (strfry router feature). this is also for reading.

primal's thing is cache service, for reading as far as I understand. it posts directly to relays of your choice. 

all comes whit the cost of more centralization. 
▲ ▼
 Any clients or relays out there that have reposts disabled?

That may help.  My feed is consistently a broken record of reposts.

May help bandwidth and UX. 
▲ ▼
 Most clients need to request a ton of data to filter it out because the relays are meant to be dumb by design if I am remembering right  
▲ ▼
 like when are we dropping the stupid JSON representation ? 

Binary format with 2 version bytes at the beginning  
▲ ▼
 Caching + bloom filter? 
▲ ▼
 - Clients make wasteful requests. 
- Kind 1 is root post and reply, no way to request only one of them.
- Lazy loading threads is impossible because e-tags are used to reference too many things.

Easy fixes for those but low interest in doing it. 
▲ ▼
 "A nostr tab" which client are you using? Coracle attempts to be more bandwidth conscious, so you might compare and see how it stacks up. 
▲ ▼
 I too am interested in this.  I basically avoid nostr when I'm not on WiFi.  I don't have unlimited data and it blows my data plan out of the water after about 5 min when I accidentally use it on a mobile network.

I think there are some settings that can help with this though like, having a different app/profile with just one relay, turning off media loading.  Turning off all extra stuff like nip05, and possibly pfps. 
▲ ▼
 having a local relay and imageproxy like a little local deployment of nostrudel, that i use here, would probably help a lot, especially the imageproxy, though the app should be keeping a cache of images (ya know, cache management isn't that hard but mosta y'all never did serious systems programming, and i only just got the excuse to build a cache)

second thing that would help would be a new field in filters that lets you send a bloom filter of all the event IDs you don't want to be sent in a request

i've filed this as an issue/enhancement request on nip-01 and i'd appreciate it if more people would recognise what a massive boost it would make to mobile users 
 Can you talk more about the bloom filter? How do you get a list of event IDs you don't want to see in the first place? 
▲ ▼
 probably from the local relay. but passing a bloom filter up seems like a solution looking for a problem. send up event ids. too many? use since/until. too big? use better encoding. still too much? write a for loop. 
▲ ▼
 clients have to have a model of a relay in them, some are more explicit in this than others. #nostrudel lets you literally run a relay on localhost:4869, which i have done extensively with my relay, and from this i saw just how chatty this whole thing is, unnecessarily

i've already come up with a way to reduce the traffic aside from this addition (and yes, a bloom filter is less bandwidth but maybe just having an extra "not these" field in the filter would work too, arguable) - that is, and i'm going to add this to my relay, it keeps track of events it sent out recently to clients, and not send them twice in some time window

also, you really haven't actually seen how much times this data gets requested by clients, i can tell 
▲ ▼
 I connect to the nostr network through a proxy relay that I wrote and operate, so I’m aware. 

I’m also cautious of premature optimization, particularly when solutions are advocated very matter of fact. 

I agree bloom filters would help reduce data tx, but it feels like a spoiler on a honda civic.  
▲ ▼
 with that said, I encourage you to explore and implement it! all iteration is good iteration.  
▲ ▼
 yeah, i'm not implementing it

i've figured out that i can have the relay track what events it sends out and not send them out every time they match a filter

within some time window that seems reasonable, somewhere between 5 and 30 minutes i figure

doesn't have to be bloom filters, that is just a compact representation of a list of 32 byte, ie 64 character event IDs that you can represent as 1 bit, pretty much, with a low rate of false positives in the bloom filter

if you got 500 results and have them cached, then that's more than most relays are ever going to send, and that's - hypothetical 500 results already known, 16kb of data in your hypothetical filter that is intended to reduce bandwidth 
▲ ▼
 ones that match the filter, in the local cache

#nostrudel has already started to move in this direction with the local cache relay, use the same query format for in-app data and then you can first check that, and then construct a bloom filter from the IDs that match, and add that to the filter before you send that filter out 
▲ ▼
 Bloom filter  pop in my mind while reading your postr, but i do not habe an exact idea on how to apply it to this case 
▲ ▼
 Since people invariably will want to use many different nostr clients, any ideas that a single client can use (caching events locally, doing any kind of negentropy with the relay, etc, etc) will not be sufficient.  Any client-specific solutions will not be enough.  We need a NIP based solution.

I've been saying for as long as I can remember that we need what I call a "client proxy".  People have pushed back by imagining I'm talking about some central service people sign up for which would become another point of censorship - NO!  I'm talking about something that you run on your local computer, or your local network, or a data center where YOU control it ... and all your various nostr clients funnel everything through it so as to avoid events being downloaded over and over again.

The protocol from client to client proxy might have to be outside of nostr proper. It just has to add which relay the client would have asked directly.

For example, the client can say "Get this filter from these relays" and the proxy can (being smart) say "Hah, I already have that from relay A, here you go" while it asks relays B and C and caches those results. 
 There was one time I forgot to turn my wifi back on and couldn't doordash for a month because nostr ate up all my mobile data....
😭
nostr:nevent1qqsy7cx4nnwppugdkewnql7x58mnt89yxxuczce9xrxqhx64js6zghcpz4mhxue69uhhyetvv9ujumt0wd68ytnsw43qygzsm98u9kzcp35zkpc62shck8335gqtq5yt4w26xwl0pp2a72qavvpsgqqqqqqsw2v33t 
▲ ▼
 I'm with @M. Dilger there. We need basically a proxy relay that lives on the device so you make requests exclusively to that but it has to be more complex than a common relay as you need a way to instruct it to get data from specific other relays and it needs some smarts of what was requested already. 
▲ ▼
 Youre going to have to let us know what client you are using so we can get to the bottom of this, and 20MB/s the internet speed dedicated to this activity, its not too helpful in diagnosing this, you could download 1MB @ 1000MB/s, it just would just be instant.