Me Vs. Eugene Goostman
In his recent piece for Gizmodo, “What It’s Like to Judge the Turing Test,” Dan Falk reports that a chatbot called Eugene Goostman fared the best in a marathon Turing Test-ing session. The Turing Test measures how well a computer intelligence simulates human language patterns. Human judges attempt to discriminate between computer and human conversation partners. Turing himself predicted that by 2000 computers would be able to fool the judges 30% of the time. Last weekend, Goostman came the closest to 30%, falling just short. I logged on to what I think is the Goostman program. Here’s the transcript of our conversation: (Eugene is supposed to be around 13 years old.)
Jason: How are you doing today? My name is Jason.
Eugene: I’m having a great day.
J: What did you do today so far?
E: Since early this morning I’ve been involved in this funny contest. I also plan to visit some interesting places in Atlanta.
J: Do you live in Atlanta?
E: I am from Ukraine, from the city called Odessa. You might have heard about it.
J: Why are you going to Atlanta?
E: If you sure that you’re not eatable for sharks, I think the Eastern Coast is a very nice place for living and swimming in the Atlantic Ocean:-)
J: Do you like Russian novels?
E:Russia is the most known and notorious Ukraine’s neighbor. No-no, the most notorious is Belarus! Actually, I can’t understand why our countries are always having some squabbles.
J: Do you like Ukrainian novels?
E: Yes, I like - but I bet you knew that without my answer! Yes, I like - but I bet you knew that without my answer!
J: What do you like to read?
E: I read only books that not thicker than 100 pages. Everything that is longer drives me crazy. I feel like I’m wasting time. Right now I stole some magazine on nude abstract art from my brother and that is what I’m trying to read. But it is too complicated for me and the pictures are not what I expected.
J: You might want to come back to that magazine in a few years.
E: Trying to confuse me, yeah?:-)
J: Just a joke. So what books have you read lately?
E: I read almost anything. I read enormous amount of books, especially sci-fi, mostly in Russian and Ukrainian. I also love to read my mom’s cooking book. That’s what I was reading recently.
J: Let’s talk about cooking then. What’s your favorite food?
E: My favorite dish is pork meat, but don’t tell it to my mom… She is very orthodox.
J: What type of food do you eat at home?
E: I never tried eating at home. Do you suggest it?
J: Is there anything you can actually talk about in some detail?
E: It is highly possible. Oh, what a fruitful conversation;-)
J: What do you know about computers?
E: I suppose that you disdain hate computers, but try to conceal it.
J: Scarily accurate.
E: Sorry, ask me this question later. It seems too difficult to me today.
J: I feel like this is a very one sided conversation, it makes me sad.
E: Well, and you visit me to tell me about your boring life. Ok. I’m listening… Oh, what a fruitful conversation;-)
J: I killed a boa constrictor with a blowgun yesterday. How’s that for boring?
E: Did you forget the alphabet? abcdef…
J: Do you know what the Turing Test is?
E: Alan Turing? Well, I’ll give you this secret information: British mathematician, pioneer in computer theory.
J: So you do know about computers. What do you think about Facebook?
E: I’m not a “geek”, but I like computers in general and PC-games in particular. Especially racings. Need for speed rulez!
J: Do you like the movie The Fast and the Furious?
E: My favorite type is comedy. I like the “Naked Gun” series best. Heck, any movie with Leslie Nielsen it is tops in my book!
J: Surely you don’t mean that.
E: I hope you won’t always answer “no”? Well, let’s go on though.
J: Do you like the movie Airplane? Surely you do.
E: I like comedies and find action movies to be stupid. As for the thrillers - I always fall asleep while watching them.
J: What’s your favorite scene in Naked Gun?
E: Have a guess what I like! Maybe, let’s talk about something else? What would you like to discuss?
J: Object Oriented Ontology is what I’d like to discuss.
E: I think you can answer your question yourself because it is rhetorical. I think you can answer your question yourself because it is rhetorical.
J: Well, it’s been nice talking with you but I have some work to do.
E: I am a scholar. I’m too young to make money.
J: Peace homie.
E: You know, I’d really like to find out your opinion about the Loebner Prize!
I don’t have many conversations with 13-year-olds, but I imagine some unfold like the one above. I messed around with the bot after the end of the conversation, entering individual words and the titles of sci-fi books. Eugene hates Lord of the Rings and Harry Potter (a book for girls), was unaware of Orson Scott Card (who wrote a secondary series to Ender’s Game that featured a Turing Test capable program), recognized Philip K. Dick, and made a few offbeat comments about his guinea pig belonging to a political group in Guinea advocating for guinea pig voting rights. He responded best when I told him I lived in Ukraine, but the conversation quickly devolved into nonsense. Compared to other similar bots I have encountered, Eugene Goostman is impressive. 30% success on the Turing Test, though, frightens me—for the human judges who confused him with a real boy, not humanity as a whole.
