We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
from PART TWO
Published online by Cambridge University Press: 12 October 2016
The 18th Loebner Prize, hosted at Reading University in 2008, began with an experiment using one-to-one text-based interrogation of thirteen machines. Six machines with the most conversational ability scores of 50 or over were invited to participate in simultaneous comparison tests; that is, each of the machines would be interrogated simultaneously with another hidden entity.
Five of the invited systems accepted and participated in the simultaneous comparison phase, with the best machine to be awarded the Bronze Medal of the Loebner Prize. Hugh Loebner allowed the authors to design the experiment around simultaneous tests featuring five machines, Elbot, Eugene Goostman, Brother Jerome, Jabberwacky and Ultra Hal.
Unknown to the judges, control pairs of both human were embedded in the machine–set-ups. This was to make sure the judges were paying attention to the utterances and not returning a result of one machine and one human each time without proper consideration.
Because one of the six invited entries could not attend the experiment, it was decided to use that entry's human comparisons for a control pair of machine–machine tests. The Elbot and Eugene Goostman machines received the most scores of ‘50 or over’ in the preliminary online one-to-one phase. Their developers agreed that their entries could participate in the machine control pair tests. This was the first time in Turing test implementations that control pairs of two humans and two machines had been embedded among machine– human pairs hidden from the interrogators.
The experimental set-ups
A panel of 24 judges, or two juries, was convened for the experiment. The rationale for this was:
(a) to evaluate Turing's five-minute-simultaneous-comparison imitation game;
(b) to test the hypothesis that a five-minute interrogation giving a thin slice of conversation is sufficient for differentiating the machine and the human;
(c) to test the hypothesis that, without being explicitly told of machine–human and control pairs of human–human and machine–machine, an interrogator's gut reaction would correctly identify the nature of each hidden interlocutor.
This experiment considered first impressions observations (Willis and Todorov, 2006) and thin slice experiments (Albrechtsen et al., 2009).
To save this book to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
Find out more about the Kindle Personal Document Service.
To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.
To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.
