Oddbean new post about login | logout "Micro$oft BitNet a blazing-fast 1-bit LLM inference framework that runs directly on CPUs. You can now run 100B parameter models on local devices with up to 6x speed improvements and 82% less energy consumption—all without a GPU!" https://github.com/microsoft/BitNet
I wonder if you can plug it into llama on @umbrel or @Start9 on a system with some GPUs and use it to train data. I have not looked at the llama or the implementation of it on these systems, but if it is based on llama and faster, I bet there is a good use case for optionally running it. Do these installs allow training or are they just running models?
I assume the 1-bit models are retarded?