why haven’t more clients implemented the gossip model? nostr:note19hm09s5dzf0dfmdtt2atlkp22mfz234xh3t3qkz7fx6n2st3fcysspwpht


Because it’s hard and it doesn’t shine

But as nostr fragments it will become increasingly more obvious when a client has done the superficial work and not the hard part


Because it’s pretty complicated to do right. nostr:npub1l2vyh47mk2p0qlsku7hg0vn29faehy9hy34ygaclpn66ukqp3afqutajft did us all a huge favor by adding it to NDK but even that implementation could probably improve.

Soon though. Soon.


outbox model (aka gossip) is a moving goal-post because we don't yet know what's the full approach

NIP-65 is a starting point, not the finish line.


:fire:


opens up many attacks unless you do lots of hardening


I think an infographic might be in the cards


We have it 🐙


I think there are lots of reasons. Here are a few…

- It’s complex to implement and requires a hardened ingester (sourcing notes from relays you didn’t select may be malicious)

- Performance and bandwidth penalties. Significant connection overhead increase.

- Clients who support NIP-65 often just copy kind 3 style 20 read/write relays in their list instead of the 2-4 the spec recommends

- Changes are still being made to NIP-65


bc it’s a server side issue?


Putting one or two relays in a note that’s stored on a ton of other relays is nice for discovery (this is the gossip model), but it’s imperfect…

It’d really be wonderful if you could put your list of relays on-chain, so everyone could easily lookup what relays your notes are currently on.

It wouldn’t be as expensive as paying a transaction fee for every new post, since it’d just be a fee each time you update your relay list… but block space is still the limiting bottleneck.

Nevertheless, it’s either blocks or a DHT for finding what relays belong to a given npub. But DHTs have their own set of trade-offs…

What I’d really like… and am putting off until GitNestr is ready… is a way for a relay to signify all the users/npubs it hosts data for & compact that into a ZKP that the relay puts on-chain.

This way anyone could scroll through these ZKPs on-chain to see if a given npub’s posts are stored on any of those relays — it offsets the transaction fees to the relays.

Trade-offs everywhere, but I know we’ll find new ways… the more options the better.


"Worst is better" dynamics maybe?

I've been around before Amethyst, when there was few clients, like branle 😹 and it seems to me that a lot was being built at the moment the gossip model arrived.

I asked Vitor about that a while ago and he said that implementing that would require doing Amethyst from scratch I think


Does Primal use it?


All of these methods have something in common — they flip the discovery model on its head, similar to gossip:

Read posts from the relays other users write to, instead of only reading from the list of relays you write to. 👀


You got to store your NIP-05 data somewhere. Why not put the relay list there, too?


I opened this issue on the damus repo the other day that documents some of the issues and how we’re planning to solve it.

https://github.com/damus-io/damus/issues/2041


True, though that’s a bit more centralized albeit possible.


Actually in the original/current NIP-05 spec but was ditched. https://github.com/nostr-protocol/nips/blob/master/05.md


Can be improved like you can have alt DNS servers. Just a matter of definition. Could be just file-hash pointers but there needs to be some point of entry to start the query.


It should be with dvms, a client would request to find an npub and it's relays. A client could also post a specific kind of note to a group of relays asking to find an npub, and a dvm or relays could answer that note with a answer, a bit like nsec bunker.


today we gossip about nazis -


Mostly because of dev complacency... Myself included.


Because Bitcoin runs on secp256k1 and that makes it holy.


Also Ed25519 can't do DH so it can't be used for encryption, whereas secp256k1 does signature and DH.

Isn't that ill-concieved DM scheme relying on secp256k1 DH.

There are ways to convert an Ed25519 key into an X25519 key for DH, libsodium has something there, but just saying.


Distributed systems engineering isn't in your league is it.


How dare you run late with this. The boss @jack is onto you! Work faster to please him.


not yet, but  nostr:npub16c0nh3dnadzqpm76uctf5hqhe2lny344zsmpm6feee9p5rdxaa9q586nvr is very aware of it and moving in its direction in due time


Forgive me if this is stupid
'Unpaid relays that have recently successfully accepted a note' could be a useful list for a client to choose from, in a way that favors the least commonly used relay among them, to increase decentralization.


good night 🌉


Hmm what about NIP-05 with decentralized DNS? I think ZeroSync will enable that on bitcoin without needing to be a full-node, but we could test it with namecoin in the meantime.

The relay IP hosting the NIP-05 data can be updated on-chain. When you connect to it, you download the user’s list of currently selected relays.

A user could put this decentralized DNS in all their notes. Could even have two or three decentralized DNSes and put all of them in every note you post.

Downside is the user has to buy the domain and pay to update the IP when needed, but it does feel more intuitive at least.


why haven’t more clients implemented the gossip model? nostr:note19hm09s5dzf0dfmdtt2atlkp22mfz234xh3t3qkz7fx6n2st3fcysspwpht


all the movies I watched in the 90s were on vhs, I didn't give a fuck about beetamax or lazerdisk
can't imagine why


Save yourself the pain, and use Pkarr.org
nostr:nevent1qqs9wy6nneg7z8n6mqcf6z6nvv5j5znglsln2nf505hn8ytj89fc45spp4mhxue69uhkummn9ekx7mqzyzprg8ug9dh2hnft5lc7ly92m9su7p6279deaaz2p8ua928ml0n2yqcyqqqqqqg2my8p6


Because it's a secret 🙊


What are the kinds of approaches to solving this problem?  Is there a blog post or Nostur article that summarizes some of them for us folks?￼@

Also can we see those articles on Freeform or Damus client? I only see them on Nostur client. Are they not in the standard?  nostr:nprofile1qqswum4p82uluhz2dr40nvdrflspffntgqghc58w9fs57nx6jkdkuaqd5rvcc


What's that?


Do you mean longform posts? Damus supports them to a degree. Nostur, Amethyst, and Coracle do a good job along with Habla and YakiHonne. I don’t think Primal shows them at all.


What's wrong with using blaster for relay list replacable events per the gossip model? Why the need for on-chain data?


Onward 🫡


Lmfao


there is a difference between the uploaded gossip and the built gossip. i rarely bother but we are retooling hollywood. ai is extremely capable of extrapolating content from a single idea if they have a point of origin. if there isn't a design of narrative surrounding the gossip it doesn't drive. if no one tells the story, a snippet picked up stays stagnant. they need a storyteller to weave the tale. hollywood was literally gasping it's dying breaths because no one was weaving - it cannot just be bits. the data is all there but without a serious reason to put it all together, they cannot because of safety stuff. i don't mind being the source because my footprint is literally everywhere, so they now have a springboard.


You can use the x-coordinate on the 25519 curve and do X25519 with your Ed25519 public keys, using the algorithm "always chose the positive x coordinate". Signal does this. In fact in secp256k1 we have the same situation.


We aren't at the perfect NIP-65 solution yet, and NIP-65 will change slightly (mostly via additions), but we have to move in this direction even if it is hard. It is no use to be stuck at fixed relay lists and not be able to find the content of the people you follow.


The issue is relying on relays to propagate your NIP-65 note to ALL other relays. It might not propagate to ALL of them if you’re just trusting it to propagate. Negentropy with interval-based syncing will help though, which were coding in Go.

The blockchain OTOH is the ultimate central repository (that’s ultimately decentralized) that any relay can use to check which relays you’re committed to.

I think Gossip is a great start that doesn’t use on-chain data. Just saying, if we want to take it farther there’s a way to do it without needing to pay on-chain fees everytime you post. The blockchain is valuable if it can be made cheap for users.


Thanks!  I haven’t seen them in other clients.


It’s a bit difficult but the last few months I’ve been on and off working on adding it to Nostur and NostrEssentials.


Stop taking shit from billionaires who don't even use your app. Outbox model does not work well on mobile, you should just wait this out.
What you have been building has been amazing.

Also a reminder that #amethyst is the only native app with both orbot and amber support.


A personal relay should implement the gossip model. Have the client only point to your personal relay, and that forwards to the world and gossips around to fetch notes.


Where can I learn about the gossip model?


Is that what this does?

https://libsodium.gitbook.io/doc/advanced/ed25519-curve25519


The influx of new hot chick bots to nostr  should do the job nicely


This is what I found about "gossip model" after consulting ChatGPT 3.5. Any expert, please correct or add extra info/context as I'm also interested in learning more about it:

"The gossip model is a communication model used in decentralized systems, such as decentralized relays. In this model, nodes in the network communicate with each other by gossiping or spreading information in a peer-to-peer manner. When a node receives new information, it shares it with a few randomly selected neighboring nodes, which in turn share it with their neighbors, and so on. This process continues until the information has spread throughout the network.

In the context of centralized vs decentralized relays, the gossip model is often used in decentralized relays to disseminate information across the network without relying on a central authority or server. Each node in the decentralized relay network acts as a peer, contributing to the dissemination of information by gossiping with other nodes. This approach provides resilience to failures and censorship, as there is no single point of control or failure in the network."


Thanks, that makes sense!


why haven’t more clients implemented the gossip model? nostr:note19hm09s5dzf0dfmdtt2atlkp22mfz234xh3t3qkz7fx6n2st3fcysspwpht


The last few days on developer nostr have involved quite a kerfluffle over the outbox model, blastr, banning jack, and many related misunderstandings. This post is an attempt to lay out my thoughts on the matter in an organized and hopefully helpful way.

# What's wrong with outbox?

It all started with a post from [jack](npub1sg6plzptd64u62a878hep2kev88swjh3tw00gjsfl8f237lmu63q0uf63m) asking why more devs haven't implemented the outbox model. There are many answers to this question, not least having to do with there being two standards for user relay selections, and ongoing changes to NIP 65. But I don't want to talk about compatibility here.

nostr:nevent1qydhwumn8ghj7argv4nx7un9wd6zumn0wd68yvfwvdhk6tcprfmhxue69uhhq7tjv9kkjepwve5kzar2v9nzucm0d5hszymhwden5te0wfjkccte9enrw73wd9hj7qpq2uf488j3uy084kpsn594xcef9g9x3lplx4xnglf0xwghyw2n3tfqqnrm02

Mazin responded with some numbers which estimate how many connections the outbox model requires. Too many connections can become expensive for low-power clients like mobile phones, not to mention some privacy issues stemming from nosy relays.

nevent1qythwumn8ghj76twvfhhstnwdaehgu3wwa5kuef0qyv8wumn8ghj7cm9d3kxzu3wdehhxarj9emkjmn99uq3samnwvaz7tmrwfjkzarj9ehx7um5wgh8w6twv5hsqgrn7l6zj7ht6ruyk76vvvtkfs4xrhyzc3tm64l3eyfvd40y26sz0gshmunh

I have some [minor disagreements](nostr:nevent1qyvhwumn8ghj76r0v3kxymmy9ehx7um5wgcjucm0d5hszxnhwden5te0wpuhyctdd9jzuenfv96x5ctx9e3k7mf0qy2hwumn8ghj7un9d3shjtnyv9kh2uewd9hj7qpqxqjdcpdjn9fp958zuunzhk8ztce6kqg05wmwnq7j7y3w6u07n3nsfpeeuh) with Mazin's numbers, but I basically agree with his point — a purist outbox model, where a large proportion of nostr users run their own relays results in a high number of connections to different relays. I brought this question up late last year in my [interview with Mike Dilger](https://fountain.fm/episode/ALgeriqGnypw1Qhqvf5W) and in a conversation with fiatjaf, who convinced me that in practice, this doesn't matter — enough people will use a handful of larger hubs that there will be a good amount of overlap in relay selections between most pubkeys.

To articulate this more clearly: the goal is not "personal web nodes", which is a pipe dream the Farcasters and BlueSkys (BlueSkies?) of the world aim at, but a more pragmatic mix between large hubs and smaller purpose-built relays. These small relays might be outlets for large publishers, small groups, or nerds who also run their own SMTP servers and lightning nodes.

The point of the outbox model is that these small nodes be _possible_ to run, and _discoverable_ from the rest of the network so that we can preserve the censorship-resistant qualities of nostr that brought us here in the first place.

# Blast It!

It's no secret that I've long been a critic of Mutiny's blastr relay implementation. My main objection is that the blastr approach doesn't account for the hard limits involved in scaling smaller relays. If the goal is to cross-pollinate notes across all relays in the network, all relays will require the same size database, and contain all notes in the network. This works right now (sort of), but as the network grows, the relays running on a $5 VPS are going to have their disks fill up and will inevitably fall over.

nevent1qyvhwumn8ghj76r0v3kxymmy9ehx7um5wgcjucm0d5hszxnhwden5te0wpuhyctdd9jzuenfv96x5ctx9e3k7mf0qythwumn8ghj7un9d3shjtnwdaehgu3wvfskuep0qqs07jr9qx49h53nhw76u7c3up2s72k7le2zj94h5fugmcgtyde4j9qfrnwxj

Not only that, but the content breakdown on any given relay by default becomes an undifferentiated soup of "GM", chinese notes, bots, bitcoin memes, and porn. Blastr makes it impossible to run an interesting relay without implementing write policies.

Which is actually fine! Because that's always been true — servers that allow anonymous uploads always get abused. Tony is just helpfully pointing out to us that this is no less true of nostr relays. I only wish he could have waited a little longer before mounting his attack on the network, because lots of hobbyists are interested in running interesting relays, but the tools don't yet exist to protect those servers from unsolicited notes.

One other note on blastr — Tony at one point described blastr as a relay proxy. This is an interesting perspective, which puts things in a different light. More on proxies later.

# Ban Jack?

Here's a thought experiment: how might we actually "ban blastr"? @Pablof7z suggested to me in a conversation that you could configure your relay to check every note that gets published to your relay against the big nostr hubs, and if it exists on any of them to simply delete it. Of course, that would result in your relay being basically empty, and the hubs having all of your content. That's game theory for you I guess.

Another approach that was floated was to encourage users to only publish to small relays. In theory, this would force clients to implement outbox so users could still see the content they were subscribed to. Fiatjaf even [posted two identical notes](nevent1qydhwumn8ghj7un9d3shjtnhv4ehgetjde38gcewvdhk6tcprfmhxue69uhhq7tjv9kkjepwve5kzar2v9nzucm0d5hszymhwden5te0wp6hyurvv4cxzeewv4ej7qpq44nafdr27wwd0cgs29r2ddg7zz94gww2a8ym6uhf3p76h33uxj9ssn68l6), one to his personal relay, and one to a hub to see which would get more engagement. The note posted to the mainstream relay got 10x more replies and likes than the more obscure note.

nostr:nevent1qyd8wumn8ghj7urewfsk66ty9enxjct5dfskvtnrdakj7qgmwaehxw309aex2mrp0yh8wetnw3jhymnzw33jucm0d5hszymhwden5te0wp6hyurvv4cxzeewv4ej7qpqdc2drrmdmlkcyna5kkcv8yls4f8zaj82jjl00xrh2tmmhw3ejsmsmp945r

Of course, this is thwarted by blastr, since blastr not only replicates notes posted to it, it also actively crawls the network as well. So the next logical step in this train of thought would be for hubs to encourage people to use small relays by actively blocking high-profile accounts.

nostr:nevent1qydhwumn8ghj7argv4nx7un9wd6zumn0wd68yvfwvdhk6tcpzdmhxue69uhhyetvv9ujue3h0ghxjme0qyd8wumn8ghj7urewfsk66ty9enxjct5dfskvtnrdakj7qpqpjhnn69lej55kde9l64jgmdkx2ngy2yk87trgjuzdte2skkwwnhqv5esfq

This would of course never happen (Damus is one client that hasn't implemented NIP 65, and they also run the biggest relay), but it was a fun thought experiment. At any rate, the silliness of the suggestion didn't stop certain people from getting offended that we would "disrupt the free market" by "forcing" our opinions on everyone else. Oh well.

# Death to Blastr

In reality, even though blastr makes it a little harder to adopt outbox in the short term, its days are numbered. Eventually, relay operators will start to feel the pain of unsolicted notes, and will either shut their relays down or look for tools that will help them curate the content they host.

From my perspective, these tools take two forms — read protection and write protection. This is something I alluded to in my [talk at Nostrasia](https://www.youtube.com/watch?v=R-5DHymkfzw) last November.

Write protection is straightforward — already many relays have access control lists based on active subscriptions, invite codes, or just static whitelists that determine who is allowed to post to a given relay, or what event authors are represented there. This approach effectively prevents blastr from using relays as free storage, which is a huge improvement.

Read protection is more tricky, because anything publicly readable will be scraped by blastr and replicated to unauthenticated-write relays across the network. In most cases, this is ok, but there are use cases for relays to exist that host a unique collection of notes oriented around some organizing principle. Unfortunately, with blastr in action (or any scraper that might exist), the only way to do this is to actively protect proprietary content. There are a few approaches that can work to make this happen:

- IP-based access control lists
- AUTH-based access control lists
- Stripping signatures when serving events
- Storing and serving encrypted content

Each of these approaches has its own set of trade-offs. But depending on use case, any of them or a combination of them could work to allow relay operators to carve out their own piece of the nostr-verse. In fact, this is a big part of what Coracle is about — the [white-labeled version of the product](https://info.coracle.social/) confines certain notes to proprietary relays, with optional encrypted group support.

Enough of my polemic against blastr. Let's talk about how to make the outbox model actually work.

# Hints are pointless

Right now, clients that implement the outbox model rely pretty heavily on relay hints to find related notes — whether user profiles, reply parents, or community definitions. The problem with hints is that they are prone to link rot. Many of the relays that were set up a year ago when nostr took off are no longer online, and yet they persist in user relay lists, and in relay hints. These hints can't be updated — they are set in stone. What this means is that a _different_ mechanism has to be used to find the notes the hints were supposed to help locate.

Because of this, I've come around to the position that hints are basically pointless. They are fine as a stopgap, and might be appropriate for certain obscure and ill-defined use cases where relay urls are the most durable address type available. But they provide basically no value in supporting the long-term robustness of the network.

What are durable, however, are pubkeys. Pubkeys are available pretty much everywhere, except in event id hints — and there is a [proposal](https://github.com/nostr-protocol/nips/issues/1101#issuecomment-2001940642) in the works to add a pubkey to those too. The cool thing about pubkeys as hints is that once you have a pubkey, all you need to do is find that person's kind 10002 inbox/outbox selections, and you should be able to find any note they have published.

This goes with the caveat that when users change their relay selections, or rotate their key, they (or their relays) should be sure to copy their notes to the new relay/pubkey.

The question then is: how do I find a given pubkey's relay selections?

There are already several mechanisms that make this reasonably easy. First of all, [NIP 65](https://github.com/nostr-protocol/nips/blob/master/65.md) explicitly recommends publishing relay selections to a wide range of relays. This is a place where the blastr approach is appropriate. As a result, relay selections are _usually_ available on the most popular public relays. Then there are special purpose relays like purplepag.es, which actively seek out these notes and index them.

These indexes are not confined to relays either. It would be trivial to create a DVM that you could ask for a pubkey's relay selections, optionally for a fee. Alex Gleason's [proxy tag](https://github.com/nostr-protocol/nips/blob/master/48.md) could also be used to indicate indexes that exist outside the nostr network — whether that be torrents, DHT keys, or what have you.

The best part is that this doesn't negatively impact the decentralization of the network because in principle these indexes are stateless — in other words, they're easily derived from the state of the public part of the nostr network.

# Just do it for me

Looping back to where we started — the complexity and technical challenges of implementing the outbox model — there is a simple solution that many people have experimented with in different ways that could solve both issues at once: proxies.

As I mentioned above, Tony thinks of blastr as a proxy, and he's right. More specifically, it's a write-proxy. This is only part of its functionality (it also acts as an independent agent which crawls the network. EDIT: apparently this is not true!), but it is an essential part of how people use it.

Another kind of proxy is a read proxy. There are several implementations of these, including my own [multiplextr](https://github.com/coracle-social/multiplextr) proxy, which is outbox-compatible (although it requires a wrapper protocol for use). The advantage of a proxy like this is that it can reduce the number of connections a client has to open, and the number of duplicate events it has to download.

Proxies can do all kinds of fancy things in the background too, like managing the outbox model on behalf of the client, building an index of everything the user would be likely to ask for in advance to speed up response times, and more.

One interesting possibility is that a NIP 46 signer could double as a proxy, reducing the number of round trips needed. And since a signer already has access to your private key, this kind of proxy would not result in an escalation in permissions necessary for the proxy to work.

# It's simple

The number of cool and creative solutions to the content replication and indexing problem is huge, and certainly doesn't end with blastr. Just to summarize the next steps I'm excited to see (to be honest, I want to build them myself, but we all know how that goes):

- More clients supporting outbox
- Outbox implementations maturing (Coracle's still has some issues that need to be worked out)
- A shift from relying on relay hints to relying on pubkey hints + relay selection indexes of some kind
- Proxy/signer combos which can take on some of the heavy lifting for clients of delivering events to the right inboxes, and pulling events from the right outboxes

Let's get building!

Disambiguating the Gossip Model. Advocating for relay proxies. Separating the Truth from the Jokes. Ban Jack! Down with Blastr!

This is a long form article, you can read it in https://habla.news/a/naddr1qvzqqqr4gupzp978pfzrv6n9xhq5tvenl9e74pklmskh4xw6vxxyp3j8qkke3cezqq2hxenh2cchynnpdag4sepkx4gxyj2dt9nk6pn2mxs