April 2, 2012 - The United StatesUniversity of CaliforniaThe University of San Diego has released a research study claiming to provide the first "artificial intelligence system capable of passing the standard three-way test.Turing Testof empirical evidence."
The Turing Test was developed in 1950 by British mathematician and computer scientist Alan Turing, who called it "The Imitation Game. Turing envisioned that if a questioner could not distinguish between a machine and a human when communicating through text, then the machine might have human-like intelligence. In the three-way Turing test, the questioner is asked to talk to a human and a machine, and accurately identify the human.
According to 1AI, the study tested three AI models: openAI's GPT-4.5, Meta Llama 3.1 405B as well as OpenAI's GPT-4o. In the experiment, human participants engage in five-minute test conversations with a human and an AI system through a split-screen interface. At the end of each round, the questioner was required to determine which side was human.
The researchers evaluated the performance of these AI models under two conditions: a base instruction (NO-PERSONA) mode and an enhanced PERSONA mode, which directs the AI to simulate specific human behavioral traits. The results showed thatIn PERSONA mode, GPT-4.5 has a win rate of 73%, suggesting that questioners often mistake it for a humanLlama 3.1-405B has a win rate of about 561 TP3T; while in NO-PERSONA mode, GPT-4o has a win rate of only 211 TP3T.
In conversations, the questioner engages in mostly everyday small talk, 61% interactions involve asking about everyday life and personal details, and 50% interactions delve into social and emotional dimensions such as opinions, emotions, sense of humor, and personal experiences.
The study states, "If the questioner cannot reliably distinguish between a human and a machine, then the machine is considered to have passed the Turing test. Based on this logic, theBoth GPT-4.5 and Llama 3.1-405B pass the Turing test with PERSONA mode enabled."
The study authors believe that these AI systems hold the promise of seamlessly complementing or even replacing human labor in economic roles that rely on brief conversations. They further state, "More broadly, these systems may become indistinguishable substitutes for a variety of social scenarios ranging from conversations with strangers online to interactions with friends, coworkers, and even romantic partners."
