Having followed Apple for close to three decades, I knew this would happen as soon as the first llama dropped. They tend to have a longer horizon and miss out on a lot of practical opportunities while lining up for something most people aren't thinking about yet. Given their surprising advance in unified memory and decision to make over-powered but under-featured mobile devices, preparing to run something heavy in the near future is a reasonable conclusion.
But, their advantage is going to be limited by their ecosystem. iOS is a lot more constrained than an open source OS, and while it can be a hot mess, it's also possible to lock those down too. It just takes time and code. Perhaps code written by LLMs.
Yesterday I downloaded a local LLM app to my iPhone 15 Pro Phi3 mini was throwing 15 tokens / sec. Don't get sidetracked by Apple, everyone is going the same place, and very soon.
Oddbean new post about login | logout