In the past decade of academics, plagiarism checks became increasingly prevalent, serving as a crucial tool to uphold academic integrity. As we step into the current decade, a new era is dawning upon us with the rise of AI-generated content checks poised to take center stage.
Artificial intelligence now enables the generation of texts that mimic human writing to an astonishing degree. However, this development also raises concerns regarding the potential misuse of AI-generated content to deceive plagiarism detection systems and produce seemingly original works.
To tackle this emerging challenge, institutions, educators, and technology providers are joining forces to develop AI-driven plagiarism detection tools. They can effectively discern between genuine human-written content and AI-generated text. These advanced algorithms will delve into linguistic nuances, writing patterns, and other subtle indicators to identify instances of artificially generated content.
However, a study from Stanford University has shown that such AI-detection tools could be seriously wrong.
The failure of AI text detectors
The current tools used by colleges to identify AI-generated content might not be as foolproof as claimed. These tools boast an impressive 99% accuracy rate, but researchers from Stanford University argue that this number can be “misleading.”
Namely, these detection tools themselves rely on artificial intelligence for content analysis. As is the case with many other applications of AI, there is a (huge) potential for bias. These biases can influence the tool’s ability to differentiate between authentic student work and AI-generated content accurately.
Source: PRSA
James Zou, an assistant professor specializing in biomedical data science at Stanford University, conducted an experiment involving 91 essays written by non-native English speakers. The assignments were written for the Test of English as a Foreign Language (TOEFL), a well-known English language proficiency test. He tested these essays against seven of the most commonly used tools in colleges today.
Surprisingly, over half of the essays were identified as AI-generated by these programs. One tool even went as far as marking 98% of them as the work of bots.
To further investigate, the researchers decided to use essays written by native English-speaking eight-graders in the US to evaluate the same set of tools. In this case, over 90% of the essays were correctly identified as being written by humans.
How AI tools assess content
The researchers examined into the reasons behind the bias of AI detection tools towards non-native English speakers. They discovered that it was primarily attributed to a factor known as “text perplexity” within the content. “Text perplexity” serves as an indicator of how much a generative AI model is taken aback or perplexed while attempting to predict the subsequent word in a sentence.
Source: Medium
This metric measures the ease or difficulty with which the model predicts the next word in a sentence. A low perplexity indicates easy predictability, whereas a high perplexity signifies difficulty in prediction. Large language models (LLMs), including ChatGPT, produce text with low perplexity, then use it to distinguish between AI-generated and human-generated content.
The bias against non-native speakers arises because they often use common words and follow familiar patterns in their writing. Consequently, the content they create is more likely to be mistaken for bot-generated by these language models.
The implications of such biased content generation are significant. Applications, assignments, and other written work from non-native speakers may be falsely flagged as AI-generated. This may further lead to potential consequences like marginalization on the internet. This bias could also influence search engine algorithms, such as Google’s, which rely on similar tools to assess content.
In academic environments, this bias might compel students to resort to using AI-generated content to sound more human, jeopardizing their career prospects and affecting their psychological well-being.
The research findings were published in the journal Patterns, shedding light on the crucial challenges and repercussions associated with biased content identification and its impact on non-native English speakers.
