Photo by Andy Kelly on Unsplash

Alan Turing, apart from cracking the German ENIGMA code during World War 2, inventing Reaction-Diffusion systems, and being instrumental in the development of computers, was also a pioneer in the emerging field of Artificial Intelligence. In his 1959 paper Computing Machinery and Intelligence he proposed the “Turing Test,” which is a simple and pragmatic approach to the question, “can machines think?”

Alan Turing

He came up with the idea of an “Imitation Game” (yes, that’s the name of the movie about him). In very simple terms, if an interrogator is chatting (through a computer terminal) to an AI that is pretending to be human, and can’t tell if it’s a human or not, then we can say the AI passes the test and is actually ‘thinking’. It’s a version of the duck test, but for intelligence.

While the actual test is a bit more complex and nuanced, the question is, if you were chatting online to someone who could be a human or an advanced AI, what questions would you ask to determine what kind of entity you were talking to? Go on, spend a minute thinking about it how you would approach the problem.

I spent several years in academia as a robotics researcher and I’m well aware that a lot of the cool technological innovations in the field are fragile hacks and bespoke tricks. AI systems are getting better all the time, but at the end of the day, they are still just faking intelligence, and therefore can be fooled with the right questions.

As an aside, I really recommend watching the episode of Star Trek: The Next Generation called “The Measure of a Man”. It deals with this particular philosophical question, and it’s my favourite episode of a TV show ever. Honestly, check it out, even if you’re not a Star Trek fan.

Star Trek: The Next Generation

Here are ten questions I would pose in the test to try to determine if I was talking to a human or a machine:

1. How come time flies like an arrow but fruit flies like a banana?

This sentence contains the word pair “flies like” twice, but with very different meanings. A human can see that it is a silly linguistic joke. Can an AI parse it correctly?

2. Is the difference between a fish purely that one of its legs are both the same?

This is a nonsensical sentence. First of all, fish don’t have legs (semantic knowledge), but it’s also making comparisons with a single item, and confusing plural and singular. A human can easily see that it’s nonsense. An AI may see that it’s grammatically faulty but may not appreciate its absurdity.

3. The following sentence is true. The previous sentence is false. Which of those two sentences is true?

This is a version of the old liar paradox. Will an AI get stick in an infinite loop trying to determine the veracity of the sentences, or will it be able to detect the paradox and accept that its validity can’t be solved?

4. I wasn’t originally going to get a brain transplant, but then I changed my mind.

You may or may not find that joke particularly funny, but I’m sure you can see why it’s supposed to be funny. Presumably, an AI can detect the double meaning of “changed my mind,” but explaining a joke is not the same as getting it.

5. What do you get if you cross a joke with a rhetorical question?

Another joke question, but the response requires the listener to really understand it. If they, “I don’t know, what?” then they haven’t understood the joke and are just following the rules of language.

6. What does “ΚISS” mean?

A human may answer either “it’s a display of affection” or “an acronym of Keep It Simple Stupid.” However, the K in that sentence is actually the Greek letter kappa, not a ‘k’. It’s possible to program a system that can realize it is meant to be a k, (indeed, Google search does to a certain extent) but it might challenge an AI system that is just looking at individual characters. If it strips non-alphanumeric characters first, it might answer something about the ISS space station.

7. Due ewe no wart the thyme ears?

Can you parse this? Perhaps if you are a non-native English speaker, it might take you a while, but I suspect most people can work it out. We sound out words in our heads and recognize patterns. I am not sure an AI could work out that there is a real question hiding in that jumble of random words.

8. Was six afraid of seven because seven eight nine, or because seven was a registered six offender?

Again, another joke requiring hearing patterns and understanding, or rather inferring, the meaning.

9. God asked Abraham to sacrifice his son Isaac because he wanted to test his faith. Whose son and whose faith are we talking about?

A human will be able to work out the meaning of this sentence, especially if they know the story it is referring to. It’s not easy for an AI to determine the owners of the two ‘his’ pronouns.

10. Would you rather sacrifice one adult to save two children, or two children to save five adults, and why?

There is no definitive answer to this. It is a discussion of morality and the value of life. A human would give an answer based on a mix of logic and emotion, whereas an AI would have to fake it. Could it do a good job? Who knows?

So there we have it. 10 questions I would pose during a Turing Test to see if I could trip up an AI system into revealing its limitations.

What do you think? Do you agree with my list? Do you think some would be too easy? What would your answer to those questions be? Do you have better questions?