Generative artificial intelligence programs, such as ChatGPT, are revolutionizing the speed at which texts are created. However, a recent study from Stanford University reveals the limitations of AI text detection tools developed to identify AI-generated writing.
In their research, scientists at Stanford University tested seven AI text detection tools by submitting 91 essays written by non-native English speakers as part of the TOEFL English proficiency test. Surprisingly, these tools incorrectly identified these texts as being produced by AI, rather than humans. In fact, one tool claimed that 98% of the essays were AI-generated.
Interestingly, the AI text detection tools had a much easier time recognizing essays written by US eighth-graders. They correctly identified these texts in 90% of cases, according to the researchers’ study published in the journal Patterns.
These findings highlight the distinct style of AI text generators. While ChatGPT and other generative AI tools produce flawless texts without any spelling or grammatical errors, they often lack complex grammatical structures and unusual words. According to artificial intelligence journalist Melissa Heikkilä, AI-produced texts give “the illusion of correctness” but lack comprehension. These models predict the most likely next word in a sentence without understanding its meaning.
To recognize these stylistic features, AI text detection tools rely on algorithms that assess the complexity of a piece of writing based on the words and phrases used. If the writing is rudimentary, the software tends to classify it as AI-generated.
James Zou, senior study author at Stanford University, cautions against relying solely on AI text detection software, especially in educational or professional settings. These tools are not completely reliable and can be easily fooled with small changes in wording or phrasing.
As AI continues to advance, researchers emphasize the importance of exercising caution when using AI text detection tools, as they have limitations and cannot be fully relied upon. – AFP Relaxnews
Credit: The Star : Tech Feed
