They're already getting crushed by the running costs necessary for AI, especially data storage, analysis, and energy. They're trying to get the processors cheaper, since that's the easiest pressure point, hence the recent NVIDIA crash.
Nostr distributes all of those costs. Since your server doesn't have to do all analysis for everyone, but only cater to your own use case and your own data set, and can run asynchronously -- results periodically updating a data file on the server or generating an event for your (private) relay-- it's relatively cheap and you can buy one of the older or less-powerful GPUs for $700-$1500, or just run it on your gaming PC.
We literally just listed the embedding spec in the NIP repo this week, and you've already lost patience. 😅 Like, give us at least a year, okay?
https://github.com/nostr-protocol/nips/blob/master/README.md
https://www.heise.de/news/Sprach-KI-selber-betreiben-c-t-uplink-9326997.html
As they say, in this German discussion, you can direct the AI to integrate your own data set into the prompt. You don't necessarily have to train the AI directly.
But even training will quickly be possible on smaller machines.
Yes, I know (this is what I kind of do most of my days - that integration of own data sets into prompts and responses, talking RAG and all). Still, though, I wonder whether / how Nostr will fit in here. So far, we see these two tendencies of companies / customers either opting for Microsoft Azure / OpenAI - to push cloud migration strategies they had in mind for ages but no chance to get done - or waiting for smaller, local models they can run on-prem with an acceptable amount of hardware. Still, though, with the out-of-the-box approach, using this kind of stuff in Azure is extremely attractive with all of the "features" it offers combined so this is hard to address for a lot of people.
Azure carries the burden of generalism. We can have n different models running on n different servers addressing n different user bases.
They went straight to the centralized AI because that fits their centralized storage and corporate structure, but we have distributed storage and lots of smaller project teams. We can also have some teams build models in the cloud and then open-source them.
It's not all worked out yet, but I think this is the direction it's headed.
We could come up with an open-source version of the Azure bundle, but based upon relays and events. With a self-hosted or professionally-hosted variation.
I could see relay operators and media server operators moving in that direction. It's going to be the age of backend developers and sys admins, I think.
They should probably get together with the DVM/AI-CI guys and come up with System DVMs, that give you a custom package or interface to configure, with a nice wizard, or something.
Most people would probably start out hosted and then eventually try to run one, themselves, or do the computation-heavy stuff hosted and the rest self-hosted. Or whatever. LOL No idea.
Wen Nostr #AI bundle?
nostr:note1wad8d68jl865fg9zrntvd9awhu8nzhpw3phewh6eerwrd05vpdssndl6xv
I mean, it starts with really "no duh" applications, like my media server allowing me to tag 20 selfies I've uploaded, and then use them as a basis for generating AI pics.
But what about a community that has a private relay, where they've pulled in or created a bunch of events related to the same topic, and want to tune or prompt an AI to search through their data set. Where is that AI? Is it an add-on service to the relay, from their hosting service? Or is it self-hosted and the relay operator lets them plug it in? Or they self-host everything? Or can they plug in one from the media service provider, that they've been using to digitalize their document images of old books?
I'm just sort of ranting incoherently, but it's like...
Yo. Are y'all gonna make me set this all up, for our own project, when basically everyone and their mother is going to want this? 😅
And, like, would it be possible to somehow come up with a Standard Nostr Model, that is specialized for events and relays? Does that even make sense or is the data storage system irrevelant to the model?
We have thousands of people and bots, here, that could label data.
One of those bots were alive in wss://nfrelay.app already publishing NIP-32 labelling event 😄
nak req -k 1985 wss://nfrelay.app
Free labels to be used.
Been seeing those around, yeah. I think there's a couple of label bots running around. 😅
@Tagr-bot might be the popular one . The report is clearly visible due to kind 1984 usage 😄
I have that one muted, so I wouldn't see it, but I see lots of others. They're all over GitWorkshop, which makes for a confusing thread, as they appear as replies. Need to write an issue about that.
For what I think (though mostly unrelated), we're not even necessarily missing a good AI hosting here. We're missing a good alternative that covers most of the use cases people cover with Azure yet is not provided by Microsoft. An increasing amount of companies and quite some organizations, as I repeatedly have to learn, trust in that stuff. Teams is pretty everywhere. Sharepoint and Onedrive integrates with this. AD and Cloud SSO is used to handle AAA. In some cases, corporations even run virtual desktops on that stuff. Azure computer resources are used to deploy microservices that speak to other "standard" third-party components in Azure (such as Azure-hosted postgresql, elasticsearch, ...). It's a technical sh_t show if you look closer but it still gets you quite far in terms of integration, and their AI stuff tightly integrates with that both on a user level and on an application level. I have no real good idea how to get rid of that, except for really strong regulation (which I don't see happen anytime soon). It's concerning to the max (even though this was a massive deviation from AI on Nostr...).😶
I'm doubtful that we will make much influence on the corporate front, at least for some years. In my experience, it's just too well integrated at this point. So many specialised apps are written for Windows, so it makes sense that they just go all in on Microshit. I begrudgingly use it at work and just accept that it'll probably be that way until I retire. I'm sure we will have another CloudStrike, except far worse, before people even begin to question the stupidity of having most major infrastructure systems in one shitty basket that just keeps growing.
German government is trying to go open-source, to save money, but the older public employees keep causing trouble.
Are they the same idiots responsible for selling the Bitcoin and shutting down the nukes?
Yeah, I know. My last job was rolling out Office 365 and co.
You forgot to mention Power BI. 🤦♂️ Bane of my existence.
Uh. I tried to forget about Power BI. But then again, I try the same for Sharepoint.🙈
The last class in my CS degree revolves around PowerBI. Lol It's a bullshit class, but I might replace it with a C++ class or something.
They want you to learn it, to service the corporate victims and make $$$.
To be fair, that kind of is what colleges are designed around. It would be neglectful for them to not help people get employed. Not why I'm there though so fuck it.
Yeah, maybe some coding class is a better choice at least from a mental health perspective...🙈
And we don't actually need to always use LLM, anyway. You can do a lot with basic machine learning, if you're a dev.
Oddbean new post about login | logout