Minimal perfect hashtables are awesome. I use it in btcs (my bitcoin script interpreter) for mapping opcode names to the opcodes themselves when parsing. It works great when you have a fixed dictionary, like the words in a note. It also uses minimal space so we can pack it nicely within nostrdb as an optimized datastructure for checking if a note contains a word. #nerdstr
Sorry ill stop nerdposting now
https://cdn.jb55.com/s/02fd7ed5300180c8.pngnostr:note1wr7pg65thhnk9r80w0j5a9j9t6aqupd62el0fydlpr4dnn25wuqqpjavy9
researching a way to build a minimal perfect hashmap of words to store alongside the binary note in nostrdb so that nostrdb clients (damus ios, damus notedeck, zaptream rust) can do O(1) mute word matching, which is pretty important for performance when you have 100s of mute words.
yeah that would save a lot of space! its just a nice compression mechanism. next step would be to find the probability distribution over words and represent an article as prefix-coded bit vector, huffman style :P
this would effectively be a compressed database. I like this idea a lot. I kinda want to try it... but the database would be completely different.
but tbh, most the storage space seems to be from keys and signatures anyways. i guess you can have a dictionary over pubkeys, but .... eh
what I'm proposing should be way faster than anything in this video, because in a *minimal* perfect hashtable I don't even need to do a compare in many cases, since if the word is out of the range it can be rejected outright.
I have realtime requirements in notedeck (144fps+), so I like to have important things precomputed by nostrdb ahead of time, like fast an small datastructure for testing mute words.
I could do this outside of nostrdb in a separate worker pool, but my ingester threads are already worker pools and it seems like this will such a common operation, so having a small and optimized data structure on each note makes sense to me
I also have this for note blocks, which are the parsed sections of a note, these are also stored in the database. this way each frame I can simply walk and interact with these structures immediately after they come out of the local database subscription and enter the timeline.
AI superposition, polysemanticity and mechanistic interpretability is fascinating. we have a chance of seeing what artificial neural networks are actually "thinking" using autoencoders to extract monosemantic features from polysemantic neurons.
Using these techniques we might be able to detect if AIs are being desceptive by peering into their brains, which will be useful if they try to enslave and/or kill us.
These terms probably makes no sense if you've never heard of them, I definitely didn't, but chris olah explains it well. Highly recommend the lex fridman podcast with him and other anthropic employees. if you have a spare... 5 hours.
https://podcasts.apple.com/ca/podcast/lex-fridman-podcast/id1434243584?i=1000676542285
Same. In this case, it’s the idea of neural networks taking advantage of the sparsity of the embeddings to encode more features than just the dimensionality of the vector space (the set of orthogonal vectors). I probably can’t do it justice in a nostr note after a few whiskeys.
I imagined it like a Fourier transform. Distilling individual features from a combined signal shared between multiple neurons.
The reason you need to do this is the superposition hypothesis: that multiple neurons are encoding more features than just the orthogonal vectors.
Chris kind of covers something along these lines. He argues these ai systems are fundamentally simple systems, like how we see uniformity across our brains. But this simple system can grow bigger and more complex abstractions. It’s quite beautiful when you think about it
yes I noticed damus doesn't load any blossom video. its easy to blame damus but we use the default ios media player, so unless utxo wants videos to not work on ios at all, I suggest returning these headers and see if it fixes it.
fwiw im not sure if it is the issue, its just the biggest thing i saw that was different between the headers of the server responses. Would be good to know which header specifically is the issue, assuming both servers are returning the same data and there isn’t some other subtle issue.
These are both pretty important for mobile, since sometimes we refuse to download things that are too big in low data mode, and ranges are nice for incremental downloads. Could mention it as a “SHOULD”
you could also detect when someone on your contact list "joins" a kind20 app by noticing if they post a kind 20. then the app could suggest: "hey, looks like one of your contacts is posting photos, you want to follow them on your kind20 follow list?"
cows are amazing biological machines.
The protein upcycling capabilities on the latest version for converting low quality plant proteins into high quality ones 👌
incredible stuff when you think about it.
nostr will have the best tech and the most devs, people won’t be able to ignore that for long. other clients on other protocols will run into heavy handed moderation issues due to centralization of power, government pressure, and lack of user keys. This is the main risk I see for mastodon, bluesky and big tech, regardless of current user numbers.
As long as people care about freedom and building a sovereign presence in cyberspace, nostr is inevitable.
It’s possible most people don’t actually care about these things, but at least we’ll have the best solution for people who do. Those people are the coolest to hang out with anyways.
I am one of the most followed accounts on here, and get a very large number of DMs and interactions. Out of the 2+ years I’ve been on here I have never had a death threat or any serious threat at all. It’s probably the nicest online experience I have had on any social network, so sorry if your claim seems hard to believe.
Maybe it’s because your first interaction with me was hostile and calling everyone on nostr a fascist. Have you tried being nicer? Do you think you would have a good experience in real life if you went up to people and started comparing them to hitler ? Sounds like a you issue.
I don’t see why it’s necessary to bring up your political leanings or sexuality, why does that matter? Other than opening yourself up to attack by assholes and trolls. I grew up on the internet. I’m 35 now, I have spent over 20 years here. I quickly learned that if you put out anything that trolls can latch onto, where they know it will get you, they will do so. This is a universal fact about humanity when it comes to posting online, especially anonymously. No tech solution can fix it. The best thing you can do is ignore them, if they can’t get a rise out of you they don’t get their dopamine.
Sure you can surround yourself with people who aren’t like that and try to heavily moderate, but it doesn’t make this basic fact go away.
notedeck should reach theoretical optimal performance. Loading the database is just a single mmap calI. There is no heavy serialization needed. The initial queries just scans the local index for note ids (u64). Once we have those, its a quick btree lookup into virtual memory for a flatbuffer-like binary note pointer. Effectively zero copy queries for every note in the database.
Will be very surprised if someone can make a faster client than this. This will be faster than any sqlite or postgres client. Notice the cold start time. It’s instant to load, even for databases that are terabytes in size.
On top of this, we skip the entire web stack for rendering and render directly to the gpu on every platform, we can render 1000s of frames per second with hundreds of nostrdb queries per frame.
People may think I’m exaggerating for marketing or promotion but i’m really not. Any web client will never be faster than notedeck, because the web does not have access to virtual memory capabilities. This is a the reason I am calling it a next-gen client.
Someone once said damus was slow and I took that personally. nostr:note10hgpd27h3kuq7ufs90m23kdvmj0p3f4m20t8cpqdew6zk6r68n4qan5alu
really not happy with the state of custodial lightning wallets. does that mean I want to make my own? probably not? maybe? would rather someone else do that so I don't have to.
Also when you need to quickly refer to a key without relays, most people will simply create an empty nprofile, which is fine for uniformity but will be a non-stable identifier when comparing to your nprofile with relays. I tend to look at the end of my npub when identifying it.
and we wouldn’t have had the duplication between nprofile and nevent, but the great centralized nostr nips council deemed it so and call npub a “horrendously bad idea”. Come on 🙄
leaning toward nostr:npub?r=relay.damus.io&r=relay.primal.net
No percent encoding needed unless explicit ws://
hex would be really bad, i moved away from hex for a very specific reason: its easy to mix up your private and public key if they are hex. I accidentally pasted my private key as hex on the network
most clients expose npub, so copying npub around is more descriptive, its compatible with nostr: mentions, etc. no need for a custom scheme for each entity
design is very important, the design of a client can encourage toxic behaviour and ruin the experience for lots of people. Why do you think people say nostr is a pleasant place?
if i created a “fuck you” button that sends “fuck you” to people as a reply, this is an example of a design that would make this place less pleasant. I have a lot of power and i’m trying to wield that in a way that makes this a fun place to be.
If you disagree with this, you or anyone can make a client that implements these things, noone is stopping you. In fact, i believe amethyst does implement this? Which is why I seem to get cowardly anon replies that attack me all the time.
Oddbean new post about login | logout
Notes by jb55 | export