Oddbean new post about login | logout Everyone repeats this but I still don’t understand how our mobile clients are going to connect to 100s of relays for users with big follow lists. How could that not be a disaster for battery, data use, scroll speed, etc etc?
Relay proxies and dvms. There is a whole world of optimizations out there that haven't been tried
A relay proxy is a simple solution, but if we are honest it is just moving the problem to another place. You, as the user, still need to run the proxy relay or trust someone else to run it for you.
Yes, but it does improve on the performance problems you were mentioning.
That is to say, a smart proxy can select relays, cache relevant data, etc. Completely agree on the privacy questions
Yes, I quite like the proxy model (it’s essentially what filter.nostr.wine is with additional features) though it has centralization/privacy costs. I’m not sold that people will be able to run these effectively on their own though. I’m interested in this discussion though, not just trying to naysay. I would like to hear the counter arguments. I made another post to Jeff nostr:note1q8yqvh246yprxwth3lnwnduz5ctfjgcajjqzcqgps2tjd4z3c0ts6m576w
It’s not that they are connecting to 100s of relays and pulling down huge quantities of data. They surgically connect and grab or publish a few notes and then disconnect. My guess is that many users will still stay connected with a small number of larger relays but it’s not a requirement and it will be a limiting factor on how big relays need to be.
There is plenty of overhead to establishing a connection, making a REQ, then disconnecting. Even if the REQs are made strategically (and they certainly aren’t today on most clients) this still introduces a ton of additional latency. Small personal/community relays will also tend to be slower and unreliable, further increasing latency and performance issues. If as you say majority of users stay connected with a small number of larger relays (tends to be my belief), doesn’t the gossip/outbox just become a waste of resources unlikely to yield many new notes with terrible tradeoffs? I could be wrong of course, but to me the long term outlook of public (anyone can read) nostr relays seem more like medium-large hubs not small personal/community data stores. We still get the advantage of having many in different locations with different policies run by different operators with different risk tolerances. Private relays (with NIP-42 auth) are different and can serve a completely different purpose.
Come up with something better.
Did you read my follow ups to Jeff? I’m looking for counter arguments not to necessarily throw away the idea. I hope there is better feedback from the advocates besides “come up with something better” nostr:note1q8yqvh246yprxwth3lnwnduz5ctfjgcajjqzcqgps2tjd4z3c0ts6m576w nostr:note1jyhedh98dy63d533npak5jd0garzd6e582u5xhqsn5tcqs7nae6sjr8waq
Come up with something better is the exact right response. Essentially at the moment there is a pretty stark choice between decentralization and performance / efficiency. You can get performance and efficiency by aggregating and thus centralizing data. That's Twitter. Or you can have the basic nostr model of data spread far and wide under no point sources of control with the trade-off that it's never going to have the efficiency of a monolithic database and the need to invent new optimizations that make it work acceptably well. The gossip / outbox model seems like the current best basic optimization that retains real decentralization (without which nostr is absolutely nothing interesting at all) and lets clients find stuff. Come up with something better.
The best what outbox model can achieve (if implemented extremely and exceptionally well, which requires huge amount of man-hours) — is to become a niche technology for “ultra censorship resistance guys” (which it will unlikely to become because there are hundreds of other things needed to become censorship resistant)
Gotta say I’m pretty blown away by this response. If the most simple and obvious arguments have no rebuttal, then we indeed desperately need to come up with something better. I assumed those pushing it had at least thought past level 0. The choice isn’t gossip model or one single database. Look around, what we have now is already much better than that.
Sorry I massively oversimplified because I think it's a big picture issue. Of course there is a lot of grey. But essentially there are currently three big picture models. There's one big database (which of course is not really one db, but again simplifying) aka Twitter, there's semi decentralized where independent servers federate aka mastodon, and there's fully decentralized like nostr currently is trying to be for the most part. I'm not trying to be dismissive or sarcastic. I've tried to come up with something better and I can't. But that doesn't mean someone else couldn't. Maybe you will. I hope you do.
I agree that without decentralization, nostr becomes uninteresting. So centralizing on the same big relays is not an option. The only alternative to outbox/inbox that I know of is blasting events everywhere, and that has the worse downside of massive resource usage and its scalability becomes more and more difficult the larger the network grows. I think the downsides of the outbox/inbox model aren't a big deal. They only arise on mobile if you have low battery and are trying to follow a god-awful number of people, or if you want privacy but for some inexplicable reason aren't operating over tor and so you are pissed that your IP address leaked to a server you didn't pre-approve of (even that can be solved with whitelisting).
If there isn't a better alternative, then we need to accept the downsides.
I think this is right. Unless someone comes up with some magic - which I don't rule out - it's a trade off. I think a lot of people are not going to be satisfied with anything other than a Twitter clone. And I really think that failing the magic the only way to produce a performant Twitter clone is via some version of a big db controlled by a single entity. Personally, for me, censorship resistance is way more important than all kinds of things that Twitter has that I don't care about so for me that trade-off is easy to make. Since I got involved with nostr I've been ranting against doing things that Twitter does that I think are impossible to do (properly) in a meaningfully decentralized protocol. Like counting likes and followers. I mean, of course you can count them as long as you are satisfied with a count that is always wrong and sometimes way wrong and totally wastes resources to compile. Personally, I'm here to read what certain people have to say. I don't give the slightest shit how many people follow them or how many people like what they say. But what I want from a system like this is different from what others want. I won't ever need to follow more than a few dozen people.
I personally use a desktop computer over a starlink interface and I follow about 170 people. Right now I'm connected to 31 relays, which is far fewer connections than this computer can make. So it is working great for me. But I recognize that younger people tend to only have a smartphone and fewer people use desktop computers. And I recognize some people want to follow thousands of other people. That is a difficult thing to make work well in a direct fashion under the outbox/inbox model. But it can work indirectly. One solution is using a client proxy (which @hodlbod and I speak about briefly on his podcast that will be coming out in a week or so) which is basically an internet server which acts like a client on your behalf on the outside, and presents like a relay on the inside so that your mobile app only needs to make one connection. That can be done right now with zero changes to the NIPs, using the inbox/outbox model, and it solves that mobile phone problem far better than (1) blasting events everywhere, or (2) everybody centralizing on the same relays. Sure, most people won't be running their own client proxy, they will sign up as a customer to a client proxy service. But they can easily move to a different one if the one they are using starts censoring them. And us hacker do-it-yourself types retain the option of running direct. @Mazin
I appreciate this post, thank you for taking the time. Obviously, I’m a big fan of proxy relays as a general concept (thats what filter.nostr.wine is) and we will likely offer this type of individual proxy relay as a service eventually. I’m not sure that it leads to decentralization that is much better than us just having say 50 or 100 “big” relays where we can also “move to different ones if they start censoring us”. There will likely be far fewer proxy relay providers than public relays.
I don't expect people will spread out over thousands of relays. Things naturally gravitate towards centralization. We don't all use thousands of different web browsers or even thousands of different email providers (anymore)... people learn which ones are the best and people naturally gravitate to and centralize upon those. The same will be true of relays. Most people will use the main group of 40 or so relays (which might even whittle down to 10 or so over time, who knows) that everybody knows work well. And that begs the question "they why bother with the outbox model?" Because the outbox/inbox model allows that 5% of people who want to do their own thing, to roll-their-own, to have custody of their own notes, to be sure they are not being censored, to do so without losing their audience. I hope the fan-out never gets crazy large. I hope I'm right that most people will naturally tend toward using the same popular relays. But if it does get large, I think proxy solutions are going to be the way to manage it. I didn't realize filter.nostr.wine was such a thing, or that you were involved with it (so much going on in nostr to keep track of who is doing what).
That's funny. I also mostly use a desktop over starlink and use gossip (though I happen to be composing this on Amethyst on my phone at the moment) and I currently follow fewer than that so yeah, for me there is no performance bottleneck, no battery issues. I also run a personal relay based on strfry with my own minor modifications and I very much like the idea of client proxies. But I also grow most of my own food. And while friends who come over admire my garden and tell me how much they'd like to have one like it themselves, everyone knows that the vast majority of people are not gonna be spinning up a VPS to run and maintain a proxy relay. I think needing to even subscribe and pay for a proxy that someone else runs is going to be too big an adoption hurdle for nostr to get beyond fringe usage in the age of downloading an app to your phone that "just works" in about 15 seconds. I hope I'm wrong but that's how it seems to me. For me, the problem with even "50 or 100" big relays that everyone uses is that each and every one of those relays becomes a target for government's that think their job is to keep people from saying things they don't want them to say. If there is such a target set, no matter the intentions of the people operating those relays, the whole thing is too vulnerable. I think the only way it's not vulnerable is if relays are tiny, cheap, stupid and ubiquitous so that it's effectively impossible to police them by sheer numbers. I recognize that world has all the performance and scaling issues we are talking about and it also may not work in the sense of getting past fringe adoption. But again, I'm here for the free speech absolutism, not for the stickers, badges, likes, follower counts, etc.
so no jurisdictional relay arbitrage?
From this point of view I don't see how to solve the problem. Information flows in the digital world in bubbles. The idea of an omnipresent connection where everything appears to be connected and we know everything that is happening instantly is a fallacy. Hodlbod's explanation of how Twitter achieves this apparent ubiquity demonstrates that this data is manipulated. Trying to achieve this result will simply perpetuate the error and generate unnecessary technical problems.
Client proxies only need to serve smallish groups where the intersection of the relays used by the people followed by the people they serve isn't too large. Then we can have many client proxies.
Most of my opinions about nostr's architecture come from a conversation I had with a friend of mine who used to work at Twitter. He explained that Twitter is only able to deliver relevant content quickly by pushing it to a huge network of special purpose caches. This is easily done with nostr relays in theory, the hard part is coordinating how caches (relays) get primed. I think some combination of push and pull will be necessary - if indeed the problem can be solved without central coordination.
So like having a Smart Relay?
I definitely think that's part of it
What else do you think we could need? (Nymble is looking to commission a smart relay in the near future and would like to get some perspective on what else should be built into it)
Well-established strategies that people can expect smart relays to implement. DVMs that can perform aggregation/analysis/discovery on relays, including AUTH-protected ones. Clients that can use smart relays. Conventions for using different relays differently. Lots of things, mostly based on establishing good conventions and a shared mental model for this stuff.
DVM is definitely something we want to build. We were also thinking of setting up a page (at a .bit domain controlled by the client) that would act as the canonical source of information for the .bit name holder so that clients can be guarantied to receive every note the people they follow have ever posted (or at least the ones they want them to see). An added benefit is that it seems that clients would be able to pull information faster with such an architecture.
If Nostr strives to be a Twitter and deliver what Twitter delivers, it will fail. The beauty is in not having an algorithm. If people insist on wanting to be directed, I think they are not the right people for this type of freedom, which requires a commitment together. Aristotle said that there are slaves by nature. I don't think Nostr should become a slave network.