Oddbean new post about login | logout Our current civilization is built on the written word. Knowledge was built on this to scale and spread. Fossil fuels provided compute and social mechanisms to spread exponentially quicker in the last 150 years. Here we are. The idea of Gray Goo might be on point, but settled in a different domain. What if self-replicating knowledge tools ( #generativeAI ) turn knowledge and our ability to deal with peak system complexity into #graygoo . The end result is the same as bio/nano/ai tech.
This scenario works by cognitive zombification. First, we become reliant on determining direction via compute (reporting, logistics, operations, prediction). Then, we willingly dedicate resources to displace human cognition from the process. Stability would be something like the first Matrix movie, which I think is the true allegory of the first movie. The real gray goo, powered by zombie resources and hope, is when knowledge eats itself, out of control, with zombies operating/maintaining.