This is huge! Now watch the LLM API costs dropping even further. 

These papers almost feel like cheat codes, & why closed companies like OpenAI don’t publish their important works anymore.

Mind-boggling that it is even possible.

Full research paper: https://t.co/Kvj3lRTONE?s=09

Abstract:
https://image.nostr.build/955e3e7a4e37cf74162329132f5d0d29afb1ebefa56fca46e859bdcca83a9eb6.jpg 

 Check this too 🤯
https://huggingface.co/papers/2312.11514 

 11.69x faster? 😮
https://image.nostr.build/3baa05385a11708f26060004d847f6acf7aabf6ae21fc3c212145c31aa632428.jpg#m=image%2Fjpeg&dim=988x571&blurhash=i6R%7B%23%3FIU%25Mt7-%3Bof%25Mof-%3B%7EqM%7BIUxuWBofRjRjWB_3RjIUxuayt7RjRjfQM%7BM%7BRjxut7ofWBWBt7IUM%7Bj%5Bj%5Bt7j%5BWBWBj%5B&x=37bcc4bd89385a0c51abd076a5fec0b3f7aa700fc799c76afaa9c8d88b481dd7 

 Not your model
Not your prompt 

 Link to the code - https://github.com/SJTU-IPADS/PowerInfer