Why the cost of training AI could soon become too much to bear
==========

The cost of training the most advanced AI models may soon be too much to bear. OpenAI's GPT-3, which had 175 million parameters, cost over $4.6 million to train. GPT-4, released a year ago, has an estimated 1.8 trillion parameters and a training cost of around $100 million. Experts forecast exponential cost growth in training AI models, with the amount of computational power needed doubling roughly every six months. The hardware cost of training one new frontier AI model could theoretically surpass the entire gross domestic product of the U.S. around the mid-2030s. The availability of training data, expanding data centers, and energy consumption are potential limiting factors. However, smaller, fine-tuned models can outperform large models like GPT-4 in specific use cases. An open-source approach and collaborative work can help address cost issues.



https://fortune.com/2024/04/04/ai-training-costs-how-much-is-too-much-openai-gpt-anthropic-microsoft/