Oddbean new post about | logout
 nostr:nprofile1qqsw9n8heusyq0el9f99tveg7r0rhcu9tznatuekxt764m78ymqu36cpz4mhxue69uhhyetvv9ujuat50phjummwv5hszymhwden5te0wahhgtn4w3ux7tn0dejj7qg4waehxw309an8yetwwvh82arcduhx7mn99uuwx66a, any chance that Haven Outbox is rejecting kind 10002 events (other tha. Due to rate limiting)? I don't see anything obvious in the code. 
 FYI, this is true for any replaceable event. If the relay sends an outdated event, the app will immediately send a new version if the app has one. 

For relays, the recommendation is to either always accept replaceable event updates, even if people are not paying anymore, OR delete old versions from your database. 
 this is one of the "directory" event kinds that i talk about, events that should be broadcast everywhere, because they provide information about the addresses of things that need to be accessible everywhere

they are small compared to follow lists, i think the proper relay behaviour is to accept them if it has old copies also 
 Some of them are massive (20+) relays because some clients just import everything from kind3 automatically. We need to find a way to discourage that.  
 turning these lists into CRDTs is key to cutting down this... follows, relay lists, mailboxes, all of them could be add/subtract ops one per time

i forget the nip number, maybe 87? there is some proposal for this that applies this and extends the capability of lists but i'm not sure if relays have been added to that 
 Vitor. Got it. Quick quesiton, event kind 10002 is signed by my own key when using Amethyst right? (Just checking as currently Haven only accepts events signed by the Outbox's relay owner).  In my case 10002 is the only spammy event. I don't think that neither Citrine nor Haven are rejecting the events AFAIK (but I will double check). One way or another I think that Amethyst should implement an exponential backoff policy. As it is, it's basically spamming "non-compliant" relays. 
 Your 10002 is signed by your own key, but we download 10002s of the authors of every like, zap, post, reply, report, edit + every pubkey mention in any post. So, for each post that appears in the screen, there is likely a 1-100 10002s from other folks being downloaded. 
 Got it. But is Amethyst trying to download or broadcast these (as in, other people's 10002 relay lists) from/to my own Outbox relay? As in, is my interpretation that Amethyst should only be trying to write my own 10002 events to my Outbox relay and downloading 10002 events from everyone else's Inbox relay correct? Or should my Outbox relay be expected to accept other people's 10002 events? 
 No, it only reverts back to the relay that sent the outdated event. So, this usually only happens when somebody had permissions to insert, the relay received a bunch of 10002s and then the relay closed that permission.

That being said, I would keep a copy of everybody's 10002s in my outboxes because it helps clients figure out where to send things if your relay has seen them before. 
 Makes total sense. Thank you. So in Haven's case this likely isn't the problem, as in, the Outbox relay only ever accepted events signed by me, and as far as I can tell, it accepts all events signed by me, including replaceable events (unless I consume all my "tokens", then it starts rate limiting). So I'm still at loss about what's triggering the 10002 event write "loop".

How does Amethyst check if my Outbox relay latest 10002 event is up to date? (Can you roughly point me to this code  in Amethyst?) 
 ohhh I just logged into your relay and it sent ALL the past versions of your own 10002 events. There are 44 10002 events coming down to Amethyst just for yourself. For each one of the 43 outdated events, Amethyst replies with the new one. 

nostr:nprofile1qqsw9n8heusyq0el9f99tveg7r0rhcu9tznatuekxt764m78ymqu36cpz4mhxue69uhhyetvv9ujuat50phjummwv5hszymhwden5te0wahhgtn4w3ux7tn0dejj7qg4waehxw309an8yetwwvh82arcduhx7mn99uuwx66a something is wrong with haven's replaceable code. A filter by kind 10002 should only return the latest event, not all versions.

You can test it here: https://lightningk0ala.github.io/nostr-wtf/query

with filter: [{"kinds": [10002]}] 
 Yep I saw this too, I will have a fix for it this week. Ty gentz 
 Hi nostr:nprofile1qqs827g8dkd07zjvlhh60csytujgd3l9mz7x807xk3fewge7rwlukxgpz4mhxue69uhhyetvv9ujuerpd46hxtnfduhszrnhwden5te0dehhxtnvdakz7qgswaehxw309ahx7um5wghx6mmd9usjfpck, I’m sorry to bother you, but I’ve just confirmed that Citrine is also experiencing the “store more than one replaceable event per kind and pubkey” issue as described by nostr:nprofile1qqsyvrp9u6p0mfur9dfdru3d853tx9mdjuhkphxuxgfwmryja7zsvhqppemhxue69uhkummn9ekx7mp0qythwumn8ghj7anfw3hhytnwdaehgu339e3k7mf0qyghwumn8ghj7mn0wd68ytnhd9hx2tch2deau below.

nostr:nevent1qqszpw6fuypscee284eap44kjcrusvdl97jrhpd0xkrwq3wfmcc9ndspypmhxue69uhksctkv4hzuctrvd5k7mre9eek7cmfv9kz76twvfhhsq3qgcxzte5zlkncx26j68ez60fzkvtkm9e0vrwdcvsjakxf9mu9qewqxpqqqqqqzrnj7sq

I have the delete events option enabled, but Citrine is still returning more than one kind 10002 event for my pubkey.

https://image.nostr.build/edaff6505d4269d91fe2129d0a70f43993aa8dc5267ac661a1608ae35ad79abb.jpg

Since folks may not want to open Citrine to the internet, I wrote a quick script with `nostr-tools` (sorry, I’m not a JavaScript person at all) to reproduce the problem below. Just replace the IP address and pubkey. My local Citrine instance is currently storing and returning several different kind 10002 events for my pubkey.

```javascript
import WebSocket from 'ws'
import { Relay, useWebSocketImplementation } from 'nostr-tools/relay'

useWebSocketImplementation(WebSocket);

const relay = await Relay.connect('ws://192.168.1.1:4869');
console.log(`connected to ${relay.url}`)

const pk = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"

let eventCount = 0

relay.subscribe([
  {
    kinds: [10002],
    authors: [pk],
  },
], {
  onevent(event) {
    console.log('got event:', event)
    eventCount++
  },
  oneose() {
    console.log('Got', eventCount, 'events')
    relay.close()
  }
})
```

Could you please have a look when you have a chance?
nostr:nprofile1qqsw9n8heusyq0el9f99tveg7r0rhcu9tznatuekxt764m78ymqu36cpz4mhxue69uhhyetvv9ujuat50phjummwv5hszymhwden5te0wahhgtn4w3ux7tn0dejj7qg4waehxw309an8yetwwvh82arcduhx7mn99uuwx66a and nostr:nprofile1qqsrhuxx8l9ex335q7he0f09aej04zpazpl0ne2cgukyawd24mayt8gprfmhxue69uhhq7tjv9kkjepwve5kzar2v9nzucm0d5hszxmhwden5te0wfjkccte9emk2um5v4exucn5vvhxxmmd9uq3xamnwvaz7tmhda6zuat50phjummwv5hsx7c9z9 are also currently working on fixing this in HAVEN + katru.

#devstr 
 All events will return more than one version, you should try to get the latest event filtered by created_at.  
 Not according to nostr:nprofile1qqsyvrp9u6p0mfur9dfdru3d853tx9mdjuhkphxuxgfwmryja7zsvhqppemhxue69uhkummn9ekx7mp0qythwumn8ghj7anfw3hhytnwdaehgu339e3k7mf0qyghwumn8ghj7mn0wd68ytnhd9hx2tch2deau. To be fair, NIP-01 states:

> for kind n such that 10000 <= n < 20000 || n == 0 || n == 3, events are replaceable, which means that, for each combination of pubkey and kind, only the latest event MUST be stored by relays; older versions MAY be discarded.

So, my interpretation is the same as yours (i.e., Amethyst / clients should be filtering for the latest event instead of expecting relays to return only the most up-to-date event). Still, upserting replaceable events by kind and pubkey is a reasonable suggestion. Either way, I just want one side to fix this so that Amethyst stops spamming my relays :D.
 
 You could try to delete the older events using kind 5. 
 Yep, this is a good mitigation strategy. But if nothing changes on either side, unfortunately, Amethyst will still write new events every time I change my relay settings, and we’ll soon be back to the same problem. 
 Until greenart7c3 can work his magic, here's another one of my awful scripts to delete all kind 10002 events from a relay, following nostr:nprofile1qqs8h057dewjchkpp2etmjkvd9chzr7j09m5ra4jsvq4ls60usya73qpz4mhxue69uhkummnw3ezummcw3ezuer9wchsz9thwden5te0wfjkccte9ejxzmt4wvhxjme0qy88wumn8ghj7mn0wvhxcmmv9ue77sja's advice above. It works with Citrine but not with Haven (Haven is silently ignoring delete events).

nostr:nprofile1qqsw9n8heusyq0el9f99tveg7r0rhcu9tznatuekxt764m78ymqu36cpz4mhxue69uhhyetvv9ujuat50phjummwv5hszymhwden5te0wahhgtn4w3ux7tn0dejj7qg4waehxw309an8yetwwvh82arcduhx7mn99uuwx66a, is supporting NIP-09 (Event deletion requests) something you feel strongly against, or is this just something that haven't been implemented yet / possible bug?

My impression from this line is that delete events should be working: https://github.com/bitvora/haven/blob/b51529221e3df04de22b73840e2b3cc3ed6b898c/main.go#L247

```javascript
import WebSocket from 'ws'
import { Relay, useWebSocketImplementation } from 'nostr-tools/relay'
import { finalizeEvent, getPublicKey } from 'nostr-tools/pure'
import * as nip19 from 'nostr-tools/nip19'


useWebSocketImplementation(WebSocket);

const relay = await Relay.connect('ws://192.168.1.1:4869');
console.log(`connected to ${relay.url}`)

const sk = nip19.decode('nsec1aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa').data
const pk = getPublicKey(sk)

let eventCount = 0

let eventTemplate = {
  kind: 5,
  created_at: Math.floor(Date.now() / 1000),
  tags: [],
  content: 'Deleting all kind 10002 events',
}

relay.subscribe([
  {
    kinds: [10002],
    authors: [pk],
  },
], {
  onevent(event) {
    console.log('got event:', event)
    eventTemplate.tags.push(['e', event.id], ['k', '10002'])
    eventCount++
  },
  async oneose() {
    console.log('Got', eventCount, 'events')

    if (eventCount > 0) {
      const signedEvent = finalizeEvent(eventTemplate, sk)
      console.log('Deletion event:', signedEvent)
      await relay.publish(signedEvent)
    }

    relay.close()
  }
})
``` 
 what are 10002 events and why delete them? 
 10002 events are events containing a list of your Inbox/Outbox relays—see: https://nostr-nips.com/nip-65

I’m deleting them because some relays are retaining outdated events and lists. When Amethyst downloads an outdated list from a relay, it sends a new 10002 event with the latest list. Since relays aren’t deleting old events and Amethyst isn’t filtering to show only the most recent event, users who switch relays frequently (like me) end up receiving a ton of redundant 10002 events, often triggering rate limits, etc.

Not sure if I’m explaining this clearly, but I hope it makes sense. 
 Thanks, I'll fix this.
I did this way so people could restore old lists in case some client replaced it with a empty list but I started worked on other things and never finished this. 
 Last update of the day. Some folks asked for a version of my crappy script that preserves the latest relay list, and someone also suggested adding a “license” to it. So here you go. Thanks, nostr:nprofile1qqs8h057dewjchkpp2etmjkvd9chzr7j09m5ra4jsvq4ls60usya73qpz4mhxue69uhkummnw3ezummcw3ezuer9wchsz9thwden5te0wfjkccte9ejxzmt4wvhxjme0qy88wumn8ghj7mn0wvhxcmmv9ue77sja, for the idea.

```javascript
// Remove all outdated kind 10002 events
// SPDX-License-Identifier: Unlicense
// SPDX-FileCopyrightText: 2024 Anthony Accioly <anthony@accioly.social>

import WebSocket from 'ws'
import { Relay, useWebSocketImplementation } from 'nostr-tools/relay'
import { finalizeEvent, getPublicKey, sortEvents } from 'nostr-tools/pure'
import * as nip19 from 'nostr-tools/nip19'

useWebSocketImplementation(WebSocket);

const relay = await Relay.connect('ws://192.168.1.1:4869');
const sk = nip19.decode('nsec1aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa').data
const pk = getPublicKey(sk)

console.log(`connected to ${relay.url}`)

const kind10002Events = []

relay.subscribe([
  {
    kinds: [10002],
    authors: [pk],
  },
], {
  onevent(event) {
    kind10002Events.push(event)
  },
  async oneose() {
    const eventCount = kind10002Events.length
    console.log('Got', eventCount, 'kind 10002 events')
    try {
      await removeAllOutdatedKind10002Events();
    } catch (error) {
      console.error('Error removing outdated events:', error);
    }
    relay.close()
  }
})

async function removeAllOutdatedKind10002Events() {
  if (kind10002Events.length === 0) {
    console.log('No kind 10002 events to remove')
    return
  }

  const [firstEvent, ...outdatedEvents] = sortEvents(kind10002Events);
  console.log('Most recent kind 10002 event:', firstEvent);

  if (outdatedEvents.length === 0) {
    console.log('No outdated kind 10002 events to remove')
    return
  }

  const deleteEvent = finalizeEvent({
    kind: 5,
    created_at: Math.floor(Date.now() / 1000),
    // Event tags with id and kind
    tags: outdatedEvents.flatMap(e => [['e', e.id], ['k', '10002']]),
    content: 'Deleting outdated kind 10002 events',
  }, sk)
  console.log('Delete event:', deleteEvent)
  try {
    await relay.publish(deleteEvent);
    console.log('Deleted', outdatedEvents.length, 'kind 10002 events');
  } catch (error) {
    console.error('Error publishing delete event:', error);
  }
}
```

Haven doesn’t honour delete requests at the moment. One way to work around this is to delete the Outbox database and re-import your notes from well-behaved relays. Once you have a single kind 10002 event in your Outbox relay, avoid changing your relay settings (easier said than done, but it’s the only workaround I’ve found until either Amethyst or Haven/Khatru is updated).

nostr:nevent1qqs8pj23nwav4e352086d7eawq4w28lqus7yp4l6ey8vvj7uvux97tgpypmhxue69uhksctkv4hzuctrvd5k7mre9eek7cmfv9kz76twvfhhsq3qa6we08n7zsv2na689whc9hykpq4q6sj3kaauk9c2dm8vj0adlajqxpqqqqqqzv39a9a

#devnostr #kind10002writeLoop #amethyst #haven #citrine 
 nostr:nprofile1qqsyvrp9u6p0mfur9dfdru3d853tx9mdjuhkphxuxgfwmryja7zsvhqppemhxue69uhkummn9ekx7mp0qythwumn8ghj7anfw3hhytnwdaehgu339e3k7mf0qyghwumn8ghj7mn0wd68ytnhd9hx2tch2deau, nostr:nprofile1qqsw9n8heusyq0el9f99tveg7r0rhcu9tznatuekxt764m78ymqu36cpz4mhxue69uhhyetvv9ujuat50phjummwv5hszymhwden5te0wahhgtn4w3ux7tn0dejj7qg4waehxw309an8yetwwvh82arcduhx7mn99uuwx66a 
 nostr:nprofile1qqswa8vhnelpgx9f7arjhtuzmjtqs2sdgfgmw77tzu9xankf87kl7eqprdmhxue69uhksctkv4hzuctrvd5k7mre9eek7cmfv9kz7qgwwaehxw309ahx7uewd3hkctcpzdmhxue69uhhwmm59e6hg7r09ehkuef00yxkae nostr:nprofile1qqsyvrp9u6p0mfur9dfdru3d853tx9mdjuhkphxuxgfwmryja7zsvhqppemhxue69uhkummn9ekx7mp0qythwumn8ghj7anfw3hhytnwdaehgu339e3k7mf0qyghwumn8ghj7mn0wd68ytnhd9hx2tch2deau I am running these reqs and only getting one from my relay, can you please show me some screenshots of this not working? 
 Nevermind I see it on desktop but not phone  
 Check the notes I’ve tagged you in over the past few days. If you want to reproduce the problem yourself, just add a relay to your Inbox/Outbox list using Amethyst, save it, and sign the new 10002 event. Repeat this a few times and either use Vitor’s filter above or my "Citrine fix" script. Both will return more than one kind 10002 event.

Also, it seems Haven is accepting but silently ignoring Kind 5 Deletion Requests, so my script doesn’t fix the issue. Deleting the Outbox database and reimporting notes from well-behaved relays does fix the problem. 
 Delete is fixed now on the inbox relay

Fiatjaf still working on kind 10002