Language models, such as GPT, may exhibit biases, including biases against non-native English writers. These biases can be caused by the training data that was used to train the model. According to the researchers, computer programs commonly used to determine whether a text was written by artificial intelligence tend to incorrectly label articles written by non-native language speakers as AI-generated. The researchers warn against using such AI text detectors due to their unreliability, which could have negative consequences for individuals such as students and job applicants.
Researchers show that computer programs commonly used to determine if a text was written by artificial intelligence tend to falsely label articles written by non-native language speakers as AI-generated in a peer-reviewed opinion paper published on July 10 in the journal Patterns. The researchers warn against using such AI text detectors due to their unreliability, which could harm individuals such as students and job applicants.
“Our current recommendation is that we be extremely cautious and perhaps try to avoid using these detectors as much as possible,” says Stanford University senior author James Zou. “It can have significant consequences if these detectors are used to review things like job applications, college entrance essays or high school assignments.”
Our current recommendation is that we be extremely cautious and perhaps try to avoid using these detectors as much as possible. It can have significant consequences if these detectors are used to review things like job applications, college entrance essays or high school assignments.
James Zou
OpenAI’s ChatGPT chatbot, for example, can compose essays, solve science and math problems, and generate computer code. Educators across the United States are becoming increasingly concerned about the use of AI in student work, and many have begun using GPT detectors to screen students’ assignments. These detectors are platforms that claim to be able to determine whether text is generated by AI, but their dependability and effectiveness have yet to be proven.
Zou and his colleagues tested seven popular GPT detectors. They ran through the detectors 91 English essays written by non-native English speakers for a widely recognized English proficiency test known as the Test of English as a Foreign Language, or TOEFL. More than half of the essays were incorrectly labeled as AI-generated, with one detector identifying nearly 98% of these essays as written by AI. In comparison, the detectors correctly classified more than 90% of essays written by eighth-grade students in the United States as human-generated.
According to Zou, the algorithms of these detectors work by evaluating text perplexity, or how surprising the word choice in an essay is. “If you use common English words, the detectors will give you a low perplexity score, which means my essay will most likely be flagged as AI-generated. If you use complex and fancy words, the algorithms are more likely to classify it as human-written,” he says. This is because large language models, such as ChatGPT, are trained to generate text with low perplexity in order to better simulate how an average human speaks, according to Zou.
As a result, simpler word choices adopted by non-native English writers would make them more vulnerable to being tagged as using AI.
The team then put the human-written TOEFL essays into ChatGPT and prompted it to edit the text using more sophisticated language, including substituting simple words with complex vocabulary. The GPT detectors tagged these AI-edited essays as human-written.
“We should be very cautious about using any of these detectors in classroom settings, because there’s still a lot of biases, and they’re easy to fool with just the minimum amount of prompt design,” Zou said. The use of GPT detectors may have implications beyond the education sector. Search engines like Google, for example, devalue AI-generated content, which may inadvertently silence non-native English writers.
While AI tools can have a positive impact on student learning, GPT detectors should be improved and tested before being used. According to Zou, one way to improve these detectors is to train them with more diverse types of writing.
