Oddbean

Nodes will rate limit them, nodes that don't have rate limits will keep dropping older data. It is a sloppy network, you should never expect the DHT to give you perfect reliability. But just like you don't query the DNS root servers directly, you don't need to query the DHT directly, you can use relays with bigger cashes and more robust rate limiting and only rely on the DHT for things you haven't seen before. obviously that also means that you shouldn't update your Pkarr too often, because long TTL is a big part of the reliability and scalability

which 3 or 4 relays should I query to know what are your preferred servers? how can I possibly know which relays did you post your list on? wouldn't that by default force us all to post and read from the same 3/4 relays to have reliable resolution? wouldn't that make them target? and what happen when they censor you or go offline? where do I go to find you now? if you try to solve this, you will end up reinventing DHTs, but you have ZERO chance of making a DHT even 1% of the size of Mainline. So your options are: 1. use Mainline DHT 2. make a worse more vulnerable version of Mainline DHT and use that instead.

Yes indeed. Nostr is still better than the status quo, because 3-4 relays is better than one central server. But you cant compare 3-4 relays with 3-4 million nodes. I have always considered nostr resilient like RAID, but the censorship-resistance is over hyped. If you want censorship resistance we now have a bullet proof way with millions of nodes, and decades long track record. Solution is to let nostr be nostr, and use the right tool for the right job.

The NIP-65 records, which is the Nostr's DNS in a way, should be in all relays. Clients should spread that record far and wide. There are indexing relays like purplepag.es that you can use all fallback, but usually there is no need to.

If you think this is a sufficiently censorship resistant alternative to millions of self organizing nodes, so be it, you know your threat model best. Pkarr is a bet on the biggest and the most battle tested network, that is old enough that is most likely to survive for at least as many years.

NIP-65 is likely enough and I agree that it is going to take at least 30 years to replace DNS, but Nostr could serve as the test ground for pubky. If you can host all Nostr events via pubky records, faster than what relays can do, then Nostr gets better and pubky gets the testing station that it desperately needs to take on the bigger system.

I am long time fascinated by mainline DHT, and it's cool you are experimenting with it. Can we talk about browser js not being able to access DHT directly? User needs to set a custom dns server, which is quite a barrier, and is the middleman that can censor or be censored, is that right? We could have a custom relay that could query dht for user's outbox relays and fetch requested events from there, acting as dht bridge for web apps. But then it's no different from existing indexer relays that clients hardcode for discovery. We could also have a custom dns server resolving npubs to their outbox relay. But again users won't change their dns settings. I go back and forth on this and still don't see what is fundamentally solved by this use of dht, at least for the web. Am I missing something?

Add home servers to the mix, and fast caching from pubky relays. It's a very good system. I'm just now looking that pubky keys can also be tor v3 addreses, did you know? Use base32 encoding, add a 2 byte checksum and version number=3. Now host tor from your own machine over your npub (if it works). VERY exciting.

This a common question: if you need to use a bridge between browsers and Mainline DHT then how is this better than Nostr relays? 1. Native apps obviously are a thing, and arguably they are giving the web a run for it's money, so why dismiss that? 2. relays/resolvers for Pkarr are not only useful for browsers, they also add some reliability and reduce the load on Mainline which then becomes like the DNS root servers 3. Finally to your point, the upside is that you as an app developer can use your own relay to support your browser customers, and I can use my own relay to support my users, and both our relays are going to magically connect, not by gossiping with untrusted strangers, and not by forming private swarms, but by simply using Mainline as the backbone.

I am struggling a bit to explain this, but I will keep trying until I find the best words: DHTs needing a bridge to browsers doesn't equate them to Nostr relays, anymore than Bitcoin needing an Electrum server makes it equivalent to a bank. Anyone can build a bridge, and they still work with each other without knowing (or trusting) each other, that is the definition of social scalability, right? nostr:note17fgpyt2we2wy5yfrwfzkqmmmg7w7nz69lpa4mgedwq9svy074kvqcf5e45

Thanks for the input. Native apps don't really have to rely on dns like web apps and are already much less restricted and harder to censor at transport level. Those are much better censored at app store level and hence my focus on the web apps. Pkarr being cache seems like a bug, not a feature - if resolving HAS to use cache middleware to be reliable then it's less decentralised, and 10m nodes turn to 1. It looks like unless this becomes a web standard browsers won't benefit fundamentally. I thought about this much less than you did so this isn't a criticism, just thinking out loud.

you focusing on browser apps is fine, but you are wrong that native apps don't benefit from censorship resistant DNS, that is like saying Mastodon is fine on Native. No we still need users to be able to migrate from one cloud provider to another without breaking all their URLs. As for cache, everything needs to be cached on every possible layer, there is no scalability otherwise, DNS only works thanks to TTL. Focus, your criticism is about bridges being necessary between the web and the DHT. Pkarr clients can use multiple relays in parallel, and if you think will so can Nostr, I remind you of the most important point: at scale there will be so many nostr relays, it will be impossible to find peoples data unless you both share some of the relays (centralisation pressure) or if Nostr relays are gossiping with each other (basically like Bitcoin nodes). I am submitting to you, that 1000s of Pkarr relays (one by every app developer and then some that users can add) are better than Nostr relays, because they don't have the same centralisation pressure (any relay is equivalent to any other), and Mainline is at the extreme edge of quality of overlay network, and any gossiping between Nostr relays will be guaranteed to be inferior, because gossiping is inferior to DHTs in this usecase, and if they form a DHT, it will be so small that it is much more vulnerable to Sybil attacks, and if they grew to millions of nodes, then we are back at Pkarr so what the hell? 😀 Think about it a bit and try to draw the line to the extreme of scalability. In general, in **pemessionless** overlay networks, scale really really matters because otherwise you have no defense against Sybil. So without millions of nodes, you are going yo be vulnerable to Sybil attacks, forcing Nostr relays to whitelist relays they want to gossip with, and now we are back at not all relays are equal, and your relay your run on your umbrel or vps is worthless because no one wants to gossip with you. all of that assuming that Nostr relays are going to evolve into an overlay network in the first place, which would be ironic because Nostr clearly started judgmental of p2p and wanting to keep relays only serving clients. So please tell me, ignoring Native apps mobile and desktop, how is Pkarr relays not better than Nostr relays then? Also, their API is literally 2 endpoints GET and PUT, and their CPU cost is almost nothing, they are so dump :)

I am curious though if you might me misunderstanding pubky and thinking we want to put all data on the torrent network with p2p etc. That is NOT what we are doing. We strictly use Mainline to find the Homeserver (which is the Outbox in Nostr world, except it is a normal https server like WebDav). All data is on that server, we don't use Torrent at all. Others might choose to use WebTorrent for some use cases like peertube, but we don't think personal mutable data has enough demand for torrent to be helpful, and obviously not reliable

Yes, if you separate the data from the transport (a bit like nostr events) you see the completely decentralized nature. So you are not reliant of DHT but you benefit from the property of DHTs, namely, verifiable censorship resistance.

Yes, if you ship your app in a virtual machine you have to live by their rules :) I could have hardcoded the relay IP instead of domain, but then it wouldn't have worked within https apps. If you dislike this dependency on ICANN, build native apps, and or help us convince browsers to support pkarr TLDs (will take years).

You realize you can just you can just ask ICANN to adopt this, or something close. Some bits will have to be cleaned up, though, first. I happen to meet them because they do their plenary in my home town every 2 years. They've been wanting to redesign DNS for ages. And in fact last time I met them they were talking about namecoin and how there was a way to spread it. And they were asking if they should put .arpa on the end of all their domains but not show that in the browser, which would let everyone have their own resolution. I said that was a bad idea! ICANN people are actually pretty cypherpunk if you get to know them, they consider DNS to be an old system and they would update it if there was a good solution.

Then it just means that you are in the pre-seed stage. You are just playing with money with no regard for it. Anyone that has even grown anything knows that growth is all about proving you can do the next step over and over again. In my eyes, you have a LOT to prove. You talk a big game but talk is cheap. And these days even code is cheap. Demo apps are not proof of anything. > Web servers need proofing? You are not proving Web servers, you are proving your pitch. You are not pitching Web servers. Those are already out there and have already been proven. You are pitching a very specific use of Web servers. That must be proved it not only works but that there is nothing better that can easily replace it.

you went a step far, things don't prove the absence of theoretical better alternatives. but I always appreciate being called out to my face. I wonder what of the combination of Pkarr and web servers for hosting data seems to be the most vulnerable? it would be very helpful for more experienced developers to tell me "this where you will probably fail", since if they turned out right, they win, if I did a good job, then it's thanks to their heads up.

This is correct, even when one makes the most conservative bet they can think of, they still have to build the thing. For the record, I am never asking anyone to build stuff for me, or use my work, I only aim to explain what IS. What you ought to do is whatever the hell you want really. nostr:note1t3d7wnxs8vdrzj4vquyg0lhszrwm0jcntzza37k2jz0tcnq5thuqsgvrta