Oddbean new post about | logout
 I don't agree at all and I'm not sure our current understanding of the human brain is in line with what you're saying. Humans don't go about the world predicting the right word to say next. There is understanding of what was said to them to help formulate the response. Also the choice of words you use may change based on who you're talking to, the socio-economic context you're in, etc. There is certainly an element of memorization for learning language and even how we engage with the world but it is much more complex than simply predicting or curve fitting.

If human brains worked this way, GPT models would be much better than they are and wouldn't need millions of training data points  and thousands of learning hours to mimic what we do. There are obviously very large differences in how we think and how these algorithms work.