Oddbean new post about | logout
 Having followed Apple for close to three decades, I knew this would happen as soon as the first llama dropped. They tend to have a longer horizon and miss out on a lot of practical opportunities while lining up for something most people aren't thinking about yet. Given their surprising advance in unified memory and decision to make over-powered but under-featured mobile devices, preparing to run something heavy in the near future is a reasonable conclusion.

But, their advantage is going to be limited by their ecosystem. iOS is a lot more constrained than an open source OS, and while it can be a hot mess, it's also possible to lock those down too. It just takes time and code. Perhaps code written by LLMs. 
 Yesterday I downloaded a local LLM app to my iPhone 15 Pro Phi3 mini was throwing 15 tokens / sec. Don't get sidetracked by Apple, everyone is going the same place, and very soon. 
 Which App?  
 Thanks! You inspired me to search github for an android equivalent app. Found this one but I have not tested it, still downloading Q4_K_M 
https://github.com/nerve-sparks/iris_android