I’ve been playing with the new nostrdb query engine and this gave me an idea:💡

When you get text search results, it would be neat if your timeline jumped to that point in time. Kind of like when you search for old things on telegram.

In other news, i finished tag queries in nostrdb today, finalizing the last thing I needed for the switchover to nostrdb in iOS. Going to start on this tomorrow 👀.

I started building nostrdb 6 months ago, turns out building a high performance database from scratch in C takes some time, what a wild ride it’s been. Now the cool stuff begins 🤠

Things that are possible now:

- contact list restoring
- fast account switching
- full local search history
- fast seeking to any point in time, infinite local scrolling
- efficient syncing between nostrdb and strfry nodes (soon)
- full validation and rogue relay protection, gossip model is viable now.
- note history cloud backups
- properly saved bookmark storage
- full scrollable dm history
- dm and note read states
- proper follow counts
- proper note stat counting
- follow / unfollow tracking
- fast nostrscripts for custom lists and filtering

Guess i better get to work! #nostrstandup

https://jb55.com/s/be168357bab172b2.png


Thank you nostr:npub1xtscya34g58tk0z605fvr788k263gsu6cy9x0mhnm87echrgufzsevkk5s for all you do 🫡


This looks promising; we just have to get it right once. It can solve many problems, from speed to reliability, and most importantly, it can do it in the right way.


Aww man, I kinda liked spotty follower counts, good reason to ignore it.


A lot of those items are working on Nostur 🤔

Curious to see the performance difference though.
nostr:note1gh77azc93d8dx6y9xcxagyptl3n7nzg9n7r6uhws6mxn377pl49saegar3


i like the #nostrstandup tag! going to start following it


Killing it! What was behind your decision to write it in C?


Yes nostur has a coredata database. I found it too slow which is why i wrote nostrdb


I also wanted a database I could use in damus android+notedeck, if I built on coredata it would be a bunch of duplicated and wasted time. It’s nice I can use this in my multi-platform client as well. I want to use this as a way to quickly build new native nostr clients without having to do all the hard stuff each time.


Yeah - I get it and makes total sense.

DId you ever measure the difference between coredata vs. nostrdb when considering to build it?


Yes damus use to use coredata for our profile database. It was extremely slow and had many race conditions and contention issues. Switching to nostrdb made profile search so much more stable and faster. It’s actually live on the appstore version. The remaining work is replacing in-memory and from-network notes with nostrdb. There is a lot of duplicate work that is happening atm because notes are processed twice. So the perf boost should be substantial. I am also going to rework timeline ui rendering so that it’s not as slow, as that is the slowest part of damus atm.


Cool thing about nostrdb is that it is safe to  query it concurrently across threads. reads don’t require locks because of lmdb’s mmap + copy on write mechanism.

Most queries are fast enough to run on the main thread (0.01ms for pulling 10 notes), but it’s nice that you can safely offload queries to threads if you need to. Fulltext queries can sometimes take 70ms because of the compressed key index I’m using, so I leverage threads for those situations.


Nice 💪


legend

legend


Can you add note/reply drafts too?


Looking forward to Nostr completely obsoleting Telegram at some point.


Put it in my veins.


From: nostr:npub16sqzxgzz38dw8ttfan67vjynnz8pprmuhulu5m0e0pkvyaw5vflqudnz6j


It’s already here. Exiting Damus forgets the draft.


I know, I want it stored for later tho


There’s a 300,000 sat #bounty on scheduled posts

https://github.com/damus-io/damus/issues/1241


Storing notes at the client I think is important. It makes it possible to have optimizations, indexes, caches to query data and to sync with relays in flexible ways, including the 'outbox' model.

nostr:nevent1qqsytl0w3vzcknkndzznvrw5zq4lcelf3yzelpawthgddnfcl0ql6jcpz4mhxue69uhhyetvv9ujumt0wd68ytnsw43qygpjuxp8vd29p6ancknaztql3eajk52y8xkppfn7au7elkw9c68zg5psgqqqqqqs2wjjdw


I guess simple storage/recall is a prerequisite to this, right?

I’ll put up 100k sats


Great point. Added as pre-req.

https://github.com/damus-io/damus/issues/1241#issuecomment-1883749250


I’ve been playing with the new nostrdb query engine and this gave me an idea:💡

When you get text search results, it would be neat if your timeline jumped to that point in time. Kind of like when you search for old things on telegram.

In other news, i finished tag queries in nostrdb today, finalizing the last thing I needed for the switchover to nostrdb in iOS. Going to start on this tomorrow 👀.

I started building nostrdb 6 months ago, turns out building a high performance database from scratch in C takes some time, what a wild ride it’s been. Now the cool stuff begins 🤠

Things that are possible now:

- contact list restoring
- fast account switching
- full local search history
- fast seeking to any point in time, infinite local scrolling
- efficient syncing between nostrdb and strfry nodes (soon)
- full validation and rogue relay protection, gossip model is viable now.
- note history cloud backups
- properly saved bookmark storage
- full scrollable dm history
- dm and note read states
- proper follow counts
- proper note stat counting
- follow / unfollow tracking
- fast nostrscripts for custom lists and filtering

Guess i better get to work! #nostrstandup

https://jb55.com/s/be168357bab172b2.png


almost all the clients on the web do this, some worse than others

coracle's compose field remembers if you accidentally unselect it in a reply but once you change context it forgets also

the main compose button , now post+ up the top right, idk if it remembers prior inputs, probably not

primals is really easy to drop out of the input field and then poof it's gone

ah yeah, i don't really use any others, they are all shit xD


That’s impressive. Can I ask a few questions?

Is the db for one’s notes and followers profiles? Or is it everything close to a relay copy?

What’s the expected size of the db?


Sounds amazing 🤩
Thank you for all your hard work!
🤜⚡🤛


🙂 https://image.nostr.build/575767d72b3ec57ac17f5ac2f235135b728b8101391b7a04795d6993523adca4.png