Why AI Detectors Don’t Work – And What to Do Instead

Why AI Detectors Don’t Work

“AI Detectors… scan student writing then provide a determination about how much of the text is AI-generated. Problem: they are awful at this, issuing plenty of false positives and negatives. They may also penalize some styles which the academy often allows, including work by students for whom English isn’t their first language.” – Bryan Alexander, Teaching with AI: strategies against cheating, 7/14/2025

AI detectors like GPTZero and Turnitin claim to be able to detect content generated by AI with high rates of accuracy. Take those claims of accuracy with a healthy dose of skepticism, as false positives remain a problem with all of these tools and can lead to students who didn’t cheat being accused of dishonesty. False positives are the result of how these tools work: “AI text detectors do not work in the same way that plagiarism checkers do. Plagiarism checkers compare human-written text with other human-written text. AI text detectors guess the probability that a text is written by humans or AI… Let me repeat,  AI text detectors are guessing whether a text is written by AI or not” (Trust, 2023, emphasis added).

AI detection tools are especially inaccurate, and harmful, for students who are neurodiverse or for whom English is not their first language. A recent article in Bloomberg reported, “The students most susceptible to inaccurate accusations are likely those who write in a more generic manner, either because they’re neurodivergent like Olmsted, speak English as a second language (ESL) or simply learned to use more straightforward vocabulary and a mechanical style, according to students, academics and AI developers. A 2023 study by Stanford University researchers found that AI detectors were “near-perfect” when checking essays written by US-born eighth grade students, yet they flagged more than half of the essays written by nonnative English students as AI-generated” (Davalos and Yin, 2024, emphasis added).

Because of this, UMS IT has determined that it is best to disable the AI detection feature in our Turnitin plagiarism detection tool, and does not recommend using other AI detection tools at this time. 

What to Do Instead

  1. Include a Clear Statement About What You Consider to Be Appropriate Use of AI in Your Course Syllabus – For guidance on how to draft your AI syllabus statement, please visit our mini-guide Crafting AI-Ready Syllabus Statements.
  2. Discuss Academic Integrity and AI Policy With Your Students – A syllabus statement about your expectations for AI use is a good start, but likely not enough. Consider devoting some time at the beginning of the semester to engage with your students in a discussion of your expectations. This kind of dialogue can help to create a sense of shared responsibility and build trust – and help students better understand how using AI in inappropriate ways can undermine their own learning. Some potential questions to promote discussion can include:
    • “What ethical considerations do you think are important when using AI tools for educational purposes?”
    • “Have you encountered any biases in AI tools you’ve used? How did that affect your view of the tool?”
    • “What do you think is appropriate for instructors to use generative AI in teaching and evaluating students? What’s inappropriate?”
  3. Implement Frequent, Low-Stakes Assessments – If your course relies on one or two high-stakes exams or papers, consider replacing those with more frequent, low-stakes assessments. This approach reduces the anxiety that can lead students to cheat and offers regular opportunities for feedback.  Examples that can work in-person or for an asynchronous online environment include weekly quizzes, brief journal entries or video recordings reflecting on assigned readings, or short written assignments on local or current topics.
  4. If You Choose to Use an AI Detector, Use the Results to Discuss Your Concerns About Improper AI Use – If you suspect AI misuse in your class, discuss your concerns with the student. If they say they did not misuse AI tools for that assignment, you could ask them to explain their process, discuss any sources they used, or elaborate further on points they made in the assignment, which they should be able to do if they produced the assignment themselves. You could also ask the student to complete a follow-up assessment to demonstrate learning, such as an oral interview or a real-time writing assignment or exam.

If after you meet with the student you still believe AI misuse occurred, please reach out to the Dean of Student’s office at usmdeanofstudents@maine.edu for further resources and next steps regarding the Academic Integrity Policy.

References


Last updated August 12, 2025

css.php