This is the third post about my argument to try and convince the Future Fund Worldview Prize judges that "all of this AI stuff is a misguided sideshow". My first post was an extensive argument that unfortunately confused many people.
(The probability that Artificial General Intelligence will be develop)
My second post was much more straightforward but ended up focusing mostly on revealing the reaction that some "AI luminaries" have shown to my argument
(Don't expect AGI anytime soon)
Now, as a result of answering many excellent questions that exposed the confusions caused by my argument, I believe I am in a position to make a very clear and brief summary of the argument in point form.
To set the scene, the Future Fund is interested in predicting when we will have AI systems that can match human level cognition: "This includes entirely AI-run companies, with AI managers and AI workers and everything being done by AIs." This is a pretty tall order. It means systems with advanced planning and decision making capabilities. But this is not the first time people predicted that we will have such machines. In my first article I reference a 1960 paper which states that the US Air Force predicted such a machine by 1980. The prediction was based on the same "look how much progress we have made, so AGI can't be too far away" argument we see today. There must be a new argument/belief if today's AGI predictions are to bear more fruit than they did in 1960. My argument identifies this new belief. Then it shows why the belief is wrong.
Part 1
- Most of the prevailing cognitive theories involve classical symbol processing systems (with a combinatorial syntax and semantics, like formal logic). For example, theories of reasoning and planning involve logic like processes and natural language is thought by many to involve phrase structure grammars, like for example Python does.
- Good old-fashioned AI was (largely) based on the same assumption, that classical symbol systems are necessary for AI.
- Good old-fashioned AI failed, showing the limitations of classical symbol systems.
- Deep Learning (DL) is an alternative form of computation that does not involve classical symbol systems, and its amazing success shows that human intelligence is not based on classical symbolic systems. In fact, Geoff Hinton in his Turing Award Speech proclaimed that "the success of machine translation is the last nail in the coffin of symbolic AI".
- DL will be much more successful than symbolic AI because it is based on a better model of cognition: the brain. That is, the brain is a neural network, so clearly neural networks are going to be better models.
- But hang on. DL is now very good at producing syntactically correct Python programs. But argument 4. should make us conclude that Python does not involve classical symbolic systems because a non-symbolic DL model can write Python. Which is patently false. The argument becomes a reductio ad absurdum. One of the steps in the argument must be wrong, and the obvious choice is 4, which gives us 7.
- The success of DL in performing some human task tells us nothing about the underlying human competence needed for the task. For example, natural language might well be the production of a generative grammar in spite of the fact that statistical methods are currently better than methods based on parsing.
- Point 7. defeats point 5. There is no scientific reason to believe DL will be much more successful than symbolic AI was in attaining some kind of general intelligence.
Part 2
- In fact, some of my work is already done for me as many of the top experts concede that DL alone is not enough for "AGI". They propose a need for a symbolic system to supplement DL, in order to be able to do planning, high level reasoning, abductive reasoning, and so on.
- The symbolic system should be non-classical because of Part 1 point 2 and 3. That is, we need something better than classical systems because good old-fashioned AI failed as a result of its assumptions about symbol systems.
- DL-symbol systems (whatever those are) will be much better because DL has already shown that classical symbol systems are not the right way to model cognitive abilities.
- But Part 1 point 7 defeats Part 2 point 3. We don't know that DL-symbol systems (whatever those are) will be much better than classical AI because DL has not shown anything about the nature of human cognition.
- We have no good reason, only faith and marketing, to believe that we will accomplish AGI by pursuing the DL based AI route. The fact that DL can do Python shows that it is good at mimicking symbolic systems when lots of example productions are available, like language and Python. But it struggles in tasks like planning where such examples aren't there.
- We should instead focus our attention of human-machine symbiosis, which explicitly designs systems that supplement rather than replace human intelligence.
I don't think I quite follow what you consider to be the reductio. In particular, I don't see why your argument wouldn't also go through with humans. Why doesn't the following hold?
Biological Learning (BL) is an alternative form of computation that does not involve classical symbol systems, but instead just a bunch of neurons and some wet stuff, and its amazing success at producing human intelligence shows that human intelligence is not based on classical symbolic systems
The reductio is specifically about Python. I show that the argument must conclude that Python is not symbolic, which means the argument must be wrong.
So your alterenative would be that BL shows that Python is not based on classical symbol systems.
We don't know how human cognition works which is why the BL argument is appealing. But we do know how Python works.
But humans made python.
If you claim it's impossible for a non-classical system to create something symbolic, I don't think you get to hide behind "we don't know how human cognition works". I think you been to defend the position that human cognition must be symbolic, and then explain how this arises from biological neural networks but not artificial ones.
Yes, humans made Python because we have the ability for symbolic thought.
And I am not saying that non-classical systems can't create something symbolic. In fact this is the crux of my argument that Symbolic-Neuro symbolic architectures (see my first post) DO create symbol strings. It is the process with which they create the strings that is in question.
If you agree that bundles of biological neurons can have the capacity for symbolic thought, and that non-classical systems can create something symbolic, I don't understand why you think anything you've said shows that DL cannot scale to AGI, even granting your unstated assumption that symbolic thought is necessary for AGI.
(I think that last assumption is false, but don't think it's a crux here so I'm keen to grant it for now, and only discuss once we've cleared up the other thing)
Biological neutrons have very different properties from artificial networks in very many ways. These are well documented. I would never deny that ensembles of biological neutrons have the capacity for symbol manipulation.
I also believe that non-classical systems can learn mappings between symbols, because this is in fact what they do. Language models map from word tokens to word tokens.
What they don't do, as the inventors of DL insist, is learn classical symbol manipulation with rules defined over symbols.
Could you mechanistically explain how any of the 'very many ways' biological neurons are different mean that the the capacity for symbol manipulation is unique to them?
They're obviously very different, but what I don't think you've done is show that the differences are responsible for the impossibility of symbolic manipulation in artificial neural networks.
I think I may have said something to confuse the issue. Artificial neural networks certainly ARE capable of representing classical symbolic computations. In fact the first neural networks (e.g. perceptron) did just that. They typically do that with local representations where individual nodes assume the role of representing a given variable. But these were not very good at other tasks like generalisation.
More advanced distributed networks emerged with DL being the newest incarnation. These have representations which makes it very difficult (if not impossible) to dedicate nodes to variables. Which does not worry the architects because they specifically believe that the non-localised representation is what makes them so powerful (see Bengio, LeCun and Hinton's article for their Turing award)
Turning to real neurons, the fact is that we really don't know all that much about how they represent knowledge. We know where they tend to fire in response to given stimuli, we know how they are connected, and we know that they have some hierarchical representations. So I can't give you a biological explanation of how neural ensembles can represent variables. All I can do is give you arguments that humans DO perform symbolic manipulation on variables, so somehow their brain has to be able to encode this.
If you can make an artificial network somehow do this eventually then fine. I will support those efforts. But we are nowhere near that, and the main actors are not even pushing in that direction.
That last comment seems very far from the original post which claimed
If we don't have a biological representation of how BNNs can represent and perform symbolic representation, why do we have reason to believe that we know ANNs can't?
Without an ability to point to the difference, this isn't anything close to a reductio, it's just saying "yeah I don't buy it dude, I don't reckon AI will be that good"
Sorry I think you are misunderstanding the reductio argument. That argument simply undermines the claim that natural language is not based on a generative phrase structure grammar. That is, that non symbolic DL is the "proper" model of language. In fact they are called "language models". I claim they are not models of language, and therefore there is no reason to discard symbolic models ... which is where the need for symbol manipulation comes from. Hence a very different sort of architecture than current DL
And of course we can point to the difference between artificial and biological networks. I didn't because there are too many! One of the big ones is back propagation. THE major reason we have ANNs in the first place, completely implausible biologically. No back propagation in the brain.
You seem really informed about detailed aspects of language, modelling, and seem to be an active researcher with a long career in modelling and reasoning.
I can't fully understand or engage with your claims or posts, because I don't actually know how AI and "symbolic logic" would work, how it reasons about anything, and really even how to start thinking about it.
Can you provide a primer of what symbolic logic/symbolic computing is, as it is relevant to AI (in any sense), and how it is supposed to work on a detailed level, i.e., so I could independently apply it to problems? (E.g. blog post, PDF chapter of a book).
(Assume your audience knows statistical machine learning, like linear classifiers, deep learning, rule based systems, coding, basic math, etc.).
Charles, I don't think it is necessary to understand all the details about logic to understand my point. The example of a truth table is enough, as I explain in my first post.
It's more like these deep learning systems are mimicking Python very well [1]. There's no actual symbolic reasoning. You believe this...right?
Zooming out and untangling this a bit, I think the following is a bit closer to the issue?
Why is this right?
There's no reason think that any particular computational performance is connected to human intelligence. Why do you believe this? A smartphone is amazingly better than humans at a lot of tasks but that doesn't seem to mean anything obvious about the nature of human intelligence.
Zooming out more here, it reads like there's some sort of beef/framework/grand theory/assertion related to symbolic logic, human intelligence, and AGI that you are strongly engaged in. It reads like you got really into this theory and built up your own argument, but it's unclear why the claims of this underlying theory are true (or even what they are).
The resulting argument has a lot of nested claims and red herrings (the Python thing) and it's hard to untangle.
I don't think the question of whether intelligence is pattern recognition, or symbolic logic, is the essence of people's concerns about AGI. Do you agree or not?
I'm not sure this statement is correct or meaningful (in the context of your argument) because learning Python syntactically isn't what's hard, but expressing logic in Python is, and I don't know what this expression of logic means in your theory. I don't think you addressed it and I can't really fill in where it fits in your theory.
Charles, you are right, there is a deep theoretical "beef" behind the issues, but it is not my beef. The debate between "connectionist" neural network theories and symbol based theories raged very much in the 1980s, 1990s. These were really nice scientific debates based on empirical results. Connectionism faded away because it did not prove to be adequate in explaining a lot of challenges. Geoff Hinton was a big part of that debate.
When compute power and data availability grew so fantastically in the 2010s, DL started to have practical success as you see today. Hinton re emerged victoriously and has been wildly attacking believers in symbolic systems ever since. In fact there is a video of him deriding the EU for being tricked into continued funding of symbolic AI research!
I prefer to stay with scientific argumentation and claim that the fact that DL can produce Python defeats Hinton's claim (not mine) that DL machine translation proves that language is not a symbolic process.
I literally read your post for over ~30 minutes to try to figure out what is going on. I don't think what I wrote above is relevant/the issue anymore.
Basically, I think what you did was write a narration to yourself, with things that are individually basically true, but that no one claims is important. You also slip in claims like "human cognition must resemble AGI for AGI to happen", but without making a tight argument[1].
You then point this resulting reasoning at your final point: "We have no good reason, only faith and marketing, to believe that we will accomplish AGI by pursuing the DL based AI route.".
Also, it's really hard to follow this, there's things in this argument that seem to be like a triple negative.
Honestly, both my decision to read this and my subsequent performance in untangling this, makes me think I'm pretty dumb.
For example, you say that "DL is much more successful than symbolic AI because it's closer to the human brain", and you say this is "defeated" later. Ok. That seems fine.
Later you "defeat" the claim that:
You say this means:
But no one is talking about the nature of human cognition being related to AI?
This is your final point before claiming that AGI can't come from DL or "symbol-DL".
Charles, thanks for spending so much time trying to understand my argument. I hope my previous answer helps. Also I added a paragraph to clarify my stance before I give my points.
Also you say that "You also slip in claims like "human cognition must resemble AGI for AGI to happen"". I don't think I said that. If I did I must correct it.