Transcript for Brian Christian on The Turing Test

Jim Fleming: But first, Bryan Christian.  He’s the author of “The Most Human Human,” what talking with computers teaches us about what it means to be alive.  The book chronicles his participation in one of the most famous philosophical experiments in the world, The Turing Test.  Christian explains it to Steve Paulson.

Bryan Christian: The idea behind The Turing Test comes from the British Mathematician, Alan Turing, who in the early 1950's as, what came to be known as the computer, was first being developed.  Scientists were already asking these really philosophical questions like, ‘can these machines actually think?  Could we someday build a machine that could think,’ and Turing’s answer to this was to kind of put philosophy aside and propose a practical test.  His idea is you convene a panel of scientists and you have them exchanging test messages back and forth, and what the scientists don’t know is whether the messages that are coming back on their screen are coming from a human, who’s sort of sequestered in a room down the hall, or whether they’re coming from a computer program that’s designed to pretend that it is a human sequestered in a room down the hall, so it’s up to these judges in five minutes of exchanging these typed messages to determine whether they think they’re talking to a real person or just a piece of software that’s designed to mimic human conversation. 

Steve Paulson: So it’s a conversation, sort of a very condensed conversation going for five minutes and the judge basically has to figure out within that time whether they’re talking to humans or computers?

Christian: Exactly, and it was Turing’s famous prediction that by the year 2000, these programs would be fooling judges roughly 30 percent of the time, and that as a result, we would, as he says, “speak of machines being intelligent without expecting to be contradicted.”

Paulson: There are a few curious things about this test.  One is that apparently success for the machine only requires 30 percent of the time that they fool the judges.  Why only 30 percent of the time?

Christian: Well, of course, if they were fooling the judges 50 percent of the time, that would mean the judges were doing no better than chance.  It was Turing’s goal, not really to mark a line in the sand and say, ‘as soon as machines cross this threshold, they become intelligent,’ but more to talk about sort of the Zeitgeister, the popular conception of computers, so he was more saying, by about this point in time, computers would be sufficiently good, that we generally regard them to be thinking. 

Paulson: Now one of the curiosities of the Turing Test is that it actually gives out two prizes, one to the most human computer and the other to the most human human...

Christian: That’s right.

Paulson: ...and you actually wanted to compete in the most human human category.  Why were you drawn to that? 

Christian: Well, Alan Turing’s famous prediction about the year 2000 did not come to pass, and in fact, the programs that compete every year in what’s called The Loebner Prize, which is a specific Turing Test that’s been running since the early 90's.  Even the best of these programs generally only fools one judge, if it’s lucky, but what really caught my attention was that in the year 2008, the top computer program fooled three out of a panel of 12 judges, which of course, is 25 percent, so it missed the famous threshold by just one vote, so I had this feeling of it seemed that humanity had kind of dodged a bullet or this narrow squeeze for our species and it really excited me to think about what it might be like to get involved and sort of personally intercede, if you will, on behalf of Homo Sapiens. 

Paulson: You were going to defend the honor if the human species.

Christian: Yeah, that’s right.

Paulson: Now I know that you talked to a number of people, or I guess they’re called Confederates, who had competed in previous Turing Tests, in terms of trying to be the most human human.  What advice did they give you?

Christian: There was a famous example in the early ‘90's of science fiction author, Charles Platt, who won the most human human award, and I want to say it was 1994, and when asked how he did it, his response was, ‘well, I was moody, irritable and obnoxious.’

Paulson: It doesn’t say much for human beings does it?

Christian: Well, no it doesn’t.  I mean it struck me as being this sort of bleak answer, and so to me, the question really becomes this sort of call to arms of what really does it mean to be human and to act human, and is just being myself enough or should I really prepare for this test the way that the program and the computer software is being prepared, specifically to take it?

Paulson: So give me an example of this.  I mean, what is, I don’t know, typical computer speak as opposed to human speak?

Christian: It’s hard to say.  There are a number of different types of chat bots basically, so there are some chat bots for example, that are designed to be extremely fluent on a very limited topic, so there was one in 1997 that spoke at great length about it’s opinions of Bill Clinton’s second term presidency, and so if you also had strong opinions on Bill Clinton’s second term presidency, then you found you could sort of carry on for a surprisingly long time, doing what seemed like exchanging political views, but in fact, the program just had a sort of giant index of things that it felt like saying about Bill Clinton and kind of stepped through them one at a time.  If you tried to change the subject, you would find that it resisted any attempt to talk about anything else.  There are other programs that are designed in more of what you might think of as a sort of crowd sourced or Wiki-like structure, so for example, the program Clever Bot, has basically just been hanging out on line for about 15 years talking to people and writing down, if you will, everything that people say to it and sort of cataloging this giant database of real things that humans say.  So when you interact with a program like Clever Bot, you find that it can respond to a really dramatically wide range of topics.  It’ll correct your grammar if you try to speak in a foreign language.  It’ll sing famous song lyrics back to you, but what you start to realize with a system like that, it’s not to much that you aren’t talking to a human, as you aren’t talking to “a” human, and so when you start to ask it questions about itself, you can sort of reel out the sense that you’re not talking to any one particular person, but that it’s a very strange collage of thousands of different people. 

Paulson: It seems that there’s a deep philosophical question in the computation that you participated in, what is it to be the most human human.  What does it mean to act as human as possible?

Christian: Yeah, I don’t know that I have a single answer to that, but the book really became an exploration of what are the things that make communication work?  I ultimately see the questions of the Turing Test as these practical questions of how do we connect with other people.  What is the process by which these two strangers come together and start to really mean something to each other or reveal something about themselves to each other, and so for me, part of what I think is really productive about the Turing Test is that the answers to how to do a good job in the Turing Test become answers to how to do a good job when you sit down to coffee with an old friend and want to sort of catch up.

Paulson: Now there are some remarkable stories of people who have been involved in the Turing Test, either as competitors, or actually, there’s one remarkable story, one of the co-founders of the Turing Test is a psychologist named, is it Robert Epstein?

Christian: Yeah.

Paulson: Tell me what happened.

Christian: Well, it’s fascinating.  Robert Epstein is a psychologist who is one of the co-founders of the Loebner Prize back in the early ‘90's, and in 2007 I think it was, he found himself signing up for this on line dating website and he began exchanging these lengthy love letters with a Russian woman named Evona, so he would tell her about his life and she would respond by telling him what it was like to live in Russia and how she was starting to develop all these feelings for him, and long story short, he realizes after four months of sending these letters back and forth that in fact, Evona does not exist.  She is a computer program, and to me it is this great irony that this is literally one of the founders of the Loebner Prize Turing Test, and even he was duped, so I think to some extent there is maybe a life lesson there, which is no one is totally safe. I also like to think that through his embarrassment, he has these serious questions to ask himself of, how was I able to go that long before realizing that I wasn’t dating a person,’ he was dating a piece of software.

Paulson: So take me to the test itself, the Turing Test, the competition.  What happened?

Christian: The test is being held in this scientific conference called Interspeech, so as I walk through the main doorways, I notice these throngs of scientists, everyone from folks from Google talking about translation, people talking about how to make fun menu systems less frustrating by recognizing certain things in your voice, these zombie-like mock ups of the human vocal tract emitting strange vowel sounds, and I make my way to the competition room, and the first thing that I notice is there are these tables where the judges are starting to gather.  The programmers are frenziedly making the last few updates, and they quickly usher me behind this big velvet curtain because I’m not supposed to really be seen by them, and we just sort of wait.  As the competition gets underway, the message starts to appear on my screen and it’s a judge talking and it says, 'hey, how’s it going,’ and the Turing Test was off and I had to figure out, okay, well I’m doing well, but of course, a computer would be able to say that it’s doing well, so how do you sort of break through the conversational ritual and get it to a place where you’re sort of improvising in real time.  That for me, was one of the big challenges.

Paulson: You did well.  In fact, you won the competition, right?

Christian: Yeah, I’m very proud to say that not only did my fellow Confederates and I avenge the narrow escape of 2008 with a shut out in 2009, but yes, in fact, I myself was named the most human human amongst the four of us.

Paulson: And what do you attribute your victory to?

Christian: I think to some extent, I felt I had prepared, and that preparation had paid off.  It was rewarding to think that by really kind of plumbing the structures and the nuances of communication, I was able to bring that knowledge to bear in the conversation by basically looking at all the corners that these programs have to cut and then emphasizing precisely those things which don’t fit into their approximations of language.

Paulson: What would you say was your best moment, looking back on this five minute conversation you had?

Christian: More or less every conversation began in this uniform way, ‘hi, how’s it going, where are you from,’ all the things that we’re used to, but what amazed me was, within that five minutes, if you really sort of worked at it, you could get into this fascinating and unpredictable space, so one judge and I had a long argument about The Rolling Stones, and another judge and I started trading stories about how our fathers and grandfathers were involved in technology and how that sort of influenced us growing up.  Another judge and I speculated that perhaps the reason robotics is such a male dominated field is that men can’t have children, so they try to create them mechanically, and so for me that was the thrill, that once you broke through that pattern, you found yourself in a totally unpredictable, totally unique situation.

Paulson: Philosophers and scientists have been speculating for centuries about what it means to be human.  You cite the Harvard psychologist, Daniel Gilbert, who has something called “the sentence,” which goes, "the human being is the only animal that “blank.”  Have you come up with your own answer to this sentence?

Christian: You know, I haven’t.  I mean it’s funny that Gilbert even says animal because I think one of the great turning points in the 20th century is that we’re now much more interested in what makes us different from machines than what makes us different from animals, which had been sort of the main philosophical quest of people like Aristotle and DeCart.  I don’t know, I will be very fascinated to see how the sentence continues to mutate and evolve over the next few years.  One thing, of course, that occurs to me is that you can turn the sentence on itself, namely say something like, ‘the human is the only animal anxious about what makes it unique.’

Fleming: Bryan Christian is the author of “The Most Human Human,” what talking with computers teaches us about what it means to be alive.  He spoke with Steve Paulson.