Oddbean new post about login | logout Amethyst receives so many events per second that I am truly considering moving the event's cryptographic verification to the GPU. GM.
GM Vitor. Have a nice weekend.
Wat. Is that really possible. I don't know GPUs at all so this might be an in-house that's lost on me.
in-joke
Technically it is. But I don't know if the OS will like it. We will see.
It's just like mining in the GPU.
I'm not sure mobile GPUs do general purpose kernels, but I'm not an expert 🤷🏾♂️. Also GPU computations tend to like working on array-like data. Is handling Nostr events similar?
I used to convert any dataset into a texture and then run a shader ok the GPU to convert the texture into pixels and then take those pixels back to the CPU. In nostr, I just need to create a texture with all the 1000 events we receive every second and call verify to run them all into the graphics pipeline and get the White (verified) or Black pixels in the at the end. This is how we did it before CUDA became a thing. It's not a good general purpose procedure, but when you have so many things to run in parallel, it could be worth it.
If I understood it correctly, essentially you encode into a rendering problem, compute, and decode back. I'm not aware of this technique. If you have a link, would love to read about it. The encode decode overhead is the main bottleneck, but it could work, specially considering today's mobile SoCs are pretty powerful. One thing to consider would be how uniform is the API from one device to the next.
GM 🔥 What is the performance impact of verification vs. JSON parsing ?
if the question is "schnorr signature verification" and your code assumes the ID is correct, then it's just a simple schnorr signature verification against the signature bytes if your code validates the ID is maybe not correct, you have to convert to the canonical form and derive the hash of this, so, it depends on how deep you want to do the validation the fact is that right now on the network, the only events i've seen that are now frequently failing to generate correct IDs are zaps that seem to be associated with either mutiny or alby lightning wallets, and they are unusual events because they contain embedded JSON inside the tags the signatures are then on these in correct ID hashes and thus they are actually invalid events nobody is checking for this afaik but i've reported it to alby and mutiny but no idea if they have fixed it, it looks like they have, because i don't see this ID check fail anymore
Amethyst's checks IDs and schnorr in every event, even duplicates to make sure the seenOn relay is accurate (the event must be valid in that relay).
this needs to be standard, good to hear
It has been like that since our first version. There has never been unverified events on any Amethyst, ever. This is not hard. Especially if the client only works with a few NIPs and has reduced exposure to different event kinds.
Você nunca me respondeu sobre fazermos uma campanha para o Rio Grande do Sul aqui.
Muito difícil para mim garantir um bom uso dos recursos daqui de fora :(
Tenho carteira disponível para fazer a campanha e distribuir diretamente nas dioceses de contato. Publiquei uma nota sobre um pouco da história do Rio Grande do Sul e alguns contatos diretos para doação. Minha carteira está disponível para intermediação. https://nostter.app/nevent1qqsgyp6md2rnvkznakj8wktp6373gs545lxnvqc66p6hgwue3wnxc2gwzvnwg
How many are there approximately? Are there any duplicate events? GM, Vitor :)
We can do 43,000 id hash check + schnorr verifications a second on the CPU (all threads, blocking everything else) of a Pixel 8. Can you do it faster? Amethyst receives on average 7000 events per second from 13 relays relays on WiFi. The startup collection is about 63,000 events for my user. If the number of relays increase, we see even more.
Oh, 43k per second is already fast. I am also trying to optimize this. My current approach is to minimize validation actions. I will cache the event ID, and if the same ID and signature appear, the signature validation will not be performed again. Additionally, I will try to fetch the events at the most appropriate time, avoiding fetching multiple events at the same time as much as possible.
Is it too late to switch to ed25519 with blake3?
Or... hear me out: nostr ASICS
Huh?
It's like GPU mining for Nostr :)
Wen ASIC event readers? nostr:nevent1qqszfsgzvuxsq2wzlusx4wa9d0l436c7f2ruw8kdfvq29767magewmqpz9mhxue69uhkummnw3ezuamfdejj7q3qgcxzte5zlkncx26j68ez60fzkvtkm9e0vrwdcvsjakxf9mu9qewqxpqqqqqqzznc0eg
Badly optimized code indeed. Now he's going to put the bad code into the GPU and then starts torturing our small machine.
We can do 43,000 id hash check + schnorr verifications a second on the CPU (all threads, blocking everything else) of a Pixel 8. Can you do it faster? Amethyst receives on average 7000 events per second from 13 relays relays on WiFi. The startup collection is about 63000 events for my user. If the number of relays increase, we see even more. Happy to do what you think can be faster than what we already do.
In disk? Phones can only do 1400 insertions per second. Usually loading this info is slower than verifying again. Keeping a cache of all events we receive won't fit in memory. We have to diacard/rotate them periodically.
I don't save anything on disk because that should be the work of a local relay. But that number uses SQLLite on Android. Standard stuff.
I did, but it was last year. Things might have improved a bit. But I don't expect much. Mobile SSDs under Android are not as fast as they could be, especially in lower end phones.
