Why There Will Be No AGI
·
Published in
·
10 min read
·
Aug 4
Photo by Amanda Dalbjörn on Unsplash
August 2023
There is a lot of debate about the capabilities and limitations of large language models like GPT-3, GPT-4, Claude, and Llama. Do they display emergent capabilities ? Do they merely display memorization but not generalization powers ? Is it correct to imply that they have reasoning abilities ? Do they display human-level natural language understanding ? How do we even define human-level natural language understanding ? Will it be ever possible to get rid of the hallucination problem ? Is the Natural Language Processing field obsolete (in a Fukuyama End of History style) ?
The November surprise
First, let me say something that is not so original, because it echoes a lot of the thoughts I have seen around on LLMs. It is incredible to see what next word prediction models can do. No one anticipated that we would be able to do so many useful things with them, until OpenAI opened Pandora’s box and released ChatGPT in November 2022. It is certainly a testament to the power of language and specifically context (given that attention is all we need), that we can do summarization, translation, and question answering among other things with the same GPT model.
There is so much power in language, so much wisdom, knowledge. There is such immense power in written language. It’s beautiful that we are able to distill a kind of inner structure of human knowledge by merely attempting to predict the next word in a sentence, based on the context. This implies that there is a considerable amount of standardization in the way we say or write things. Therefore, true novelty is quite rare, unpredictable, and precious. The space created by language, with its words and grammar, is infinite, yet simultaneously constrained by logic and reality, by human experience and what we’ve collectively gleaned from it since the beginning of time. This implies that our collective “semantic space” (the space of all the meaningful sentences or texts we can create) is more structured and predictable than we previously thought.
I find all of this fascinating. It’s as if there are certain arithmetic or algebraic laws of language that we don’t fully understand yet, but which make some sequences of words meaningful and human-like, while others are not. This is a testament to the power of mathematics…
