Research finds 52% answers to ChatGPT programming questions incorrectly

Over the past few years, computerprogrammerare turning to programs such as OpenAI's ChatGPT Chatbots like that to help themencodingsThis has hit sites like Stack Overflow, causing it to have to lay off nearly 301 TP3T employees last year. The problem, however, is that a team of researchers from Purdue University released findings at the Computer-Human Interaction Conference that found that 521 TP3T of the answers to programming questions generated by ChatGPT were wrong.

Research finds 52% answers to ChatGPT programming questions incorrectly

Source Note: The image is generated by AI, and the image is authorized by Midjourney

That's a staggering rate for a program that people rely on for accuracy and precision, and it highlights a problem experienced by other end users like writers and teachers: AI platforms like ChatGPT often produce completely wrong answers out of thin air. In their study, the researchers reviewed 517 Stack Overflow questions and analyzed the results of ChatGPT's attempts to answer them. "We found that 52% of ChatGPT answers contained misinformation, 77% answers were more verbose than human answers, and 78% answers had varying degrees of inconsistency with human answers," they wrote.

The team also linguistically analyzed 2,000 randomly selected ChatGPT answers and found that they were "more formal and analytical" while exhibiting "less negativity" - the - This is the monotonous and pleasant tone that AI tends to produce. What's particularly worrying is that many human programmers seem to prefer ChatGPT's answers. Researchers at Purdue University surveyed 12 programmers and found that they preferred ChatGPT by a margin of 351 TP3T and found no AI-generated errors at 391 TP3T.

Why does this happen? It could simply be that ChatGPT is more polite than online people. "Follow-up semi-structured interviews revealed that polite language, a refined and textbook approach to answering, and comprehensiveness were among the main reasons why ChatGPT answers appeared to be more persuasive, so participants lowered their guard and ignored some of the misinformation in the ChatGPT answers," the researchers wrote.

This study shows that ChatGPT still has major flaws, but that's a bitter consolation for people who were laid off from Stack Overflow or programmers who had to correct errors in AI-generated code.

statement:The content is collected from various media platforms such as public websites. If the included content infringes on your rights, please contact us by email and we will deal with it as soon as possible.
Information

Nvidia says 20,000 GenAI startups are developing on its platform

2024-5-24 10:49:24

Information

Krea AI video generation tool is now available to everyone. Krea Video officially enters public beta

2024-5-24 11:07:16

Search