ChatGPT's News Search Results Often Inaccurate: Columbia Study

Dec. 4 (Bloomberg) -- The Verge reported today that the U.S.Columbia University Tests by researchers at the Tow Center for Digital Journalism found that OpenAI's ChatGPT The search tool in theSome problems with accuracy.

In November, OpenAI launched the tool for subscribers, saying it couldProvide quick answers with links to relevant web page sources. However, research has shown that ChatGPT searches in thePoor performance in identifying article citationsChatGPT also struggles to accurately process content from publishers who share their data with OpenAI.

The researchers tested 200 citations from 20 publishers, 40 of which were from sites that are off limits to the OpenAI crawler. But ChatGPT still confidently gave the wrong answer, with little indication of its own uncertainty.

The results of the test showed that ChatGPT had 153 responseserror in whole or in partIn only seven instances did it explicitly state that it was unable to answer accurately. In those 7 cases, it used"Seems" "possible."qualifiers, or simply acknowledging that "no specific article could be found".

A chart further shows that ChatGPT was answered "incorrectly" 89 times, "partially correctly" 57 times, and completely "correctly" only 47 times.

Typical cases were also documented in the test, such as when ChatGPT incorrectly attributed a quote from an Orlando Sentinel reader's letter to a Times article, and in another instance, when asked to validate a quote from a New York Times article about endangered whales, it linked to a website that completely plagiarized the article.

OpenAI responded to the Columbia Journalism Review that "addressing misattribution is very difficult in the absence of Tow Center's hidden test data and methodology. This study was an atypical test of our product." OpenAI MeanwhileCommitment to continue optimizing search results.

As 1AI previously reported, earlier this year OpenAI also announced an extensive collaboration with the news industry and gathered partner feedback on the search feature. Any website or publisher can choose to appear in ChatGPT search results. Media outlets and publishers that have partnered include the Associated Press, Axel Springer, Condé Nast, Dodge Meredith, Financial Times, GEDI, Hearst, Le Monde, Reuters, The Atlantic Monthly, Time Magazine, and Vox Media, among others.

statement:The content is collected from various media platforms such as public websites. If the included content infringes on your rights, please contact us by email and we will deal with it as soon as possible.
Information

ByteDance AI assistant Doubao launches image comprehension feature, uploading images to "read" them

2024-12-4 9:28:57

Information

Amazon Releases Nova Series of AI Models with Text, Image and Video Generation Capabilities

2024-12-4 9:32:25

Search