Oddbean new post about | logout
 I'm thinking about extraction of relay data into different databases for different purposes.

I'm not a DBA. I'd love to hear what smarter folks think.

I can see use for -

Some sql for some basic storage and caching and BI/data vis.

Some graph db for some lookups or graph vis on social networks or on tags posts.

Some vector db to train some AIs on the network.

I have the feeling that basic relay dbs today are not useful for these purposes as their focus is relaying. 

But I wonder if that is just a totally acceptable separation of concerns.

E.g. let relays focus on relaying and add whatever other db you need for whatever other purposes you have. 

Then I wonder how to make existing relays interop with other dbs for other purposes. Some kind of standard? IDK.

What do our smart scaling heads think? Am I right or wrong track? 
 Any DB is fine.  Nostr content is just a text string.  And some text meta data.

Depends on your use case really.  mongo should be a decent choice. sqlite for small things.

Nostr should not be used as a database, but everyone does is.  Because it's convenient.  But you end up hitting walls.  Like no real indexes, or query language, no ability to do large follow lists, AND/OR facilities, joins, secondary keys, composite indexes etc. 

You might want to have a look at #ditto which is the first relay to have a modern database.  Nostr as a DB is 50 year old technology. 
 I hear you. I've looked a little at Ditto and my opinion at first glance is it's not as modular as the rest of the network tries to be. 

So, yes it has a DB, but depending on my purpose maybe not the one I need. 

And it has data BI and tracking but maybe not the one I need.

And it has a client but maybe not the one I need. 

And it has a user rights and identity setup, but maybe not the one I need.

I find myself seeking more modularity, like to pick up pieces ala cart and configure them (for now, and I may be wrong!)

I might be misunderstanding. Trying to get my head around it.  
 #ditto has multiple clients including nostr and mastodon clients

You, dont have to use ditto, you can build your own.  But ditto is the only solution that has been largely accepted, welcomed, and funded by the community.  

You are welcome to make your own solutions  But, those are all hard things to achieve and took a lot of work.  You have to figure out whether you want to do that work, and take the (quite high) risk that your solution is rejected by the community.  Most things are, because there's a rule of having only one way of doing things.

I agree modularity is a good thing.  All depends how much work you want to put in, whether you want to reuse or recreate a solution, and what your use cases are.  For myself, I've gone the modular path, and will reuse ditto as a component in a bigger picture. 
 Re your other points, I agree 💯%. I wonder if that's an OK separation of concerns and easy to add later (vs being a fundamental issue with the design of the network) 
 Technically it is easy to add.  The harder part is getting people to agree.  And agree on a solution that scales.  Http managed to get all of that in its early design.  99% of other protocols dont, and remain niche.