Oddbean new post about | logout
▲ ▼
oliver | 3 months ago (raw) | root | parent | reply | flag 
 Check out @Start9 / StartOS with FreeGPT. Also Venice AI is worth looking at from privacy preservation perspective.
▲ ▼
NotBiebs | 3 months ago (raw) | root | parent | reply | flag +2 
 doesn't it take up a lot of resources to run? doesn't seem like a good idea to run it on a start9 along with a bitcoin/lightning node.
▲ ▼
Start9 | 3 months ago (raw) | root | parent | reply | flag +1 
 The newer 7B models like Llama3 or Mistrel work great on well-resourced devices like the ones we sell.

No real harm in running it alongside Bitcoin and Lightning, but you also don't have to. It's your server, only install what you want.
▲ ▼
NotBiebs | 3 months ago (raw) | root | parent | reply | flag +1 
 So it runs smoothly on the Server One with both a bitcoin node and an AI chat app hooked up to Llama 3.1 8B? Is OpenUI available on Start9?

https://github.com/wandb/openui
▲ ▼
Start9 | 3 months ago (raw) | root | parent | reply | flag 
 Yeah that model runs just fine. Nothing compared to ChatGPT speeds but it's affordable consumer hardware. No, nobody has packaged OpenUI for StartOS yet, looks cool though.