The Turing Test

Well, last time I spoke about how the Fermi Paradox irritated me. I'm on a curmudgeonly roll. Since it's in the news, let's go gunning for the Turing Test.

The Turing Test was invented, not surprisingly, by Alan Turing. It came from a paper entitled Computing Machinery and Intelligence and appeared in Mind in 1950.  It was an attempt to determine whether machines could think without defining a "machine" or "think", since these were potentially ambiguous concepts. He wanted something that could get beyond this problem.

Turing proposed an "imitation game" where a judge must attempt to determine if a respondent is a human or a machine. To do this, the conversation between judge and candidate was made devoid of clues by using a teletype (remember, it was 1950). If the judge could not distinguish the machine's conversation from human, then the machine "won". If the judge could detect that the candidate was not human, the machine "lost."

Turing thought that if a computer could fool a human judge up it had shown itself sufficiently capable as to be considered "intelligent."

I don't have a problem with the Turing Test as long as we understand a few things about it:

1. It is a very limited kind of intelligence that is being tested
2. It is limited by the context of the test and the biases of the judges
3. It does not imply anything about the humanity or experiential nature of the candidate.
4. It plays into human biases in that it presumes that something that is capable of indistinguishable imitation of a human is as intelligent as a human.

 Let's consider a thought experiment. Let's say we have added intelligence to a dog. It's just as smart as a human in terms of cognitive brain power but it's nature, motivations and considerations are still that of a dog. We give the dog a Turing Test. Would it pass?

Maybe. Maybe not. Remember it's still a dog. Dogs are not motivated the same as humans. Their sensory system is very different. We are sight animals. They are smell animals. That alone is going to make the conversation interesting.

But the Turing Test presupposes that a human response is the correct response regarding intelligence. Consider if the situation were reversed and the dogs were giving a Turing Test to the humans. Perhaps a question might be, "Are you smelling my excitement right now?" (One should think that dogs would make a Turing Test that they would be able to pass.) The human would be unlikely to answer correctly and the dog professor would say sadly these humans are just not as smart as we canines. We share a lot with dogs. An intelligent extraterrestrial or computer program is going to be much harder.

We live in the context of our humanity. We should expect other, non-human, intelligences to live in their own context. My point is that there are only two possible true successful systems that would pass a Turing Test. One is a program that is specifically designed to pass a Turing Test. It doesn't have to have any general intelligence. It's designed to show itself as intelligent in this very limited domain.

Which brings us to Eugene Goostman, the program that "won" the most recent Turing Test.

Eugene Goostman is a chatterbot-- a program specifically designed to hold conversations with humans. Goostman portrays itself as a 13 year old boy from Ukraine that doesn't understand English all that well. I've read the transcript of some of Goostman's conversations. I argue that without this context provided by Goostman, it would not have passed. Goostman's programmers gamed the test in my opinion.

But let's say a really clever program was designed from the ground up to hold meaningful conversations. Would it be intelligent? I don't think so. Intelligence is a tool that can be applied in many circumstances. Watson, the program that won the Jeopardy is a closer contender. It's intelligence won at the game. Not, the same system is being used in medical decisions for lung cancer.
 
An intelligent conversationalist would be one that donated its intelligence to the conversation. It might supply insight. Make connections. In short, do all the things we expect from a human conversation. It converses intelligently instead of having conversational intelligence. That is, its conversation derives from its intelligence. It's not just a smart program that's learned to fool us.

Which brings us to the second possible winner of a Turing Test-- a system (biological or otherwise) that is so smart it can model our context sufficiently that we would find it indistinguishable from a human being. Such a system would have to be more intelligent than a human being, not less.

But this all presumes the Turing Test has a purity it does not possess. Not only does the test only measure an extremely narrow view of intelligence-- behavioral conversation-- it presumes the judges are unbiased. As we saw with Goostman, this is not so. And it could never be so.

After all, humans imbue cats, dogs, insects, statues, rocks, cars and images on toast with human like qualities. We infer suspicion from inept sentence structure. We infer honesty when it's really spam. We infer love from typewritten conversation when it's really sexual predation. Put two dots over  a curve and we inevitably see a face. Give us a minimum of conversational rigor and we inevitably determine that it's human.

Humans can tell what's legitimately human and what's not a lot of the time. But we don't do it from characters on a screen. We detect it from motion or facial expression. We detect it from tone of voice or contextual cues. We know when something that purports to be human, isn't, if we can bring our tools to bear on it.

For example, there's the uncanny valley. This is when an artificial visualization of a human being gets very close to presenting as human but not quite. People get uncomfortable. It happened with the animated figures in Polar Express. The characters animated on the train were just a little creepy. Exaggerated figures such as Hatsune Miku or the characters from Toy Story are fine-- they're clearly not human and don't trigger that reaction. Think of the characters in Monsters, Inc: monsters all, with the exception of Boo. But we were able to fill in any missing humanity they lacked. (The fact that the story was brilliant is beside the case.)

Alan Turing was a genius but, personally, I don't think the Turing Test is one of his best moments. It's an extremely blunt tool for measuring something that requires precision.

I invite the system under test to come with me to a family reunion with my in-laws. Navigating that is going to take some real intelligence.