Today, people can let AI writing tools help them with just a few taps of the keyboardWriting essays, composing rap songs, or writing playsHowever, for the time being, mankind cannot expect it to have "Shakespearean originality". For now, however, humans can't expect "Shakespearean originality". According to Science, a new study suggests that these AI works are still clearly derivative, at least for now.
To validate this idea, the researchers have developed a program that is capable ofMeasuring AI Creativityof the program.University of LondonMirco Musolesi, a computer scientist at the Academy who studies AI creativity, noted that assessing creativity is "a complex and interesting challenge" and argued that this new approach has been useful inAddressing linguistic noveltyThis aspect of the problem is outstanding.
Since the introduction of generative AI and big language models, there have been questions in the scientific community about their creative capabilities. While these AIs can quickly generate text that appears to be human-created, some scholars argue that they are not truly innovative and simply reorganize the content in the training corpus. Critics have compared them to "Random Parrot" (Note: Blind repetition of known text).
Yet quantifying this creativity is no easy task. Scientists usually take two approaches.Computerized detection of signs of plagiarism -- but "no plagiarism" does not mean "originality"; and secondly, to let theHuman ratingAssessing fluency and originality - but the processSubjective and time-consuming.
In response.University of Washingtoncomputer scientist Lu Ximing and his team have developed a tool called DJ Search. ItBoth objectivity and the ability to capture nuance. The tool extracts phrase fragments from the AI output text and looks for similar content in the database. It not only looks for exact matches, but also recognizes semantically similar expressions. With the help of AI algorithms to analyze word meanings, it determines synonyms through embedding vectors. Ultimately, it computesPercentage of unmatched portion in textto assess the novelty of the AI output.
The study found that humans inIn the field of poetry, fiction and speechsignificantly outperforms AI, respectively. 80%, 100% and 150%.
DJ Search can also compare human productions, such as The Hunger Games, which has 35% more linguistic originality than Twilight. research leader Lu will AI analogy for DJ: "TheySplice text like a DJ mixing musicThe first is that, as wonderful as it is, it'sThere is no substitute for a composer.”
Nanyun Violet Peng, a computer scientist at the University of California, Los Angeles, suggests that the future should beAssessing the originality of the overall narrative, rather than being limited to the linguistic level.