Artificial Intelligence (AI) and the impact of ChatGPT has been a hot topic in the news recently.[1] The use of ChatGPT to author student papers has preoccupied academia. Educators fear that students can easily turn in assignments that were created by ChatGPT. That fear has motivated universities to impose harsh discipline when professors believe a student engaged in plagiarism by relying on AI to author a paper.
False accusations of plagiarism can lead to severe consequences, including failing grades, academic suspension, or expulsion. Before ChatGPT, educators typically supported plagiarism charges by locating the original source material and comparing it to the student’s paper.[2] Significant verbatim copying was strong evidence of plagiarism.
ChatGPT does not copy an existing text. It paraphrases and cites source material, just as a student should do. For that reason, works produced by AI cannot be detected by traditional plagiarism detectors.
Proving Plagiarism by AI
Some companies have attempted to fill the gap in plagiarism detection by offering flawed services that purport to detect the algorithm used by ChatGPT. The algorithm simulates human prose by predicting the next word that a human writer would use in the sentences it composes, based on the average word choice it finds in all the compositions in its database. Detection software determines whether a student’s word choices are so average that they are probably the product of the ChatGPT algorithm.
Unfortunately, students who consistently make average word choices can be flagged as submitting a paper written by ChatGPT. No evidence supports the false accusations beyond the assumptions made by software programmers. Innocent students have been accused of turning in plagiarized work simply because they wrote papers in the style of a paper that ChatGPT might have written.[3]
AI Plagiarism Detection Is Unreliable
While several websites have promoted AI detectors, many of the recent leading detectors have been shown to be unreliable. none The research company OpenAI, the creator of ChatGPT, designed an experimental tool called AI Classifier to detect AI-generated text test. OpenAI discontinued the product after finding that it had an accuracy rate of 26 percent.[4]
Turnitin is one of the leading tools that is marketed as an AI detector. While Turnitin initially claimed that its product returned false positives in fewer than one percent of the papers it checked, the company now admits that its detection scores are not always reliable.[5]
A computer scientist at the University of Maryland who has researched AI detectors concluded that “[c]urrent detectors of AI aren’t reliable in practical scenarios.”[6] He opines that it is “very unlikely that we’ll be able to develop detectors that will reliably identify AI-generated content.”[7]
The Future of Plagiarism Accusations
The false positives generated by AI detectors have been widely reported by media sources that focus on higher education.[8] That publicity, combined with challenges to the validity of AI detection that falsely accused students raised at disciplinary hearings, has persuaded some educational institutions that they are basing discipline on unreliable evidence.
It seems that colleges and universities have hit the pause button on accusations of plagiarism that are supported only by AI detectors and there has been a sharp decline in disciplinary charges based solely on claims of ChatGPT plagiarism. Hopefully schools have learned not to base shaky accusations on AI detection software that cannot provide a verifiable assurance of reliable results.
Scott Limmer concentrates his practice in the areas of criminal defense and representation of college and graduate students when charged with violating their school’s code of conduct.
- See, e.g., Chris Westfall, Educators Battle Plagiarism As 89% Of Students Admit To Using OpenAI’s ChatGPT For Homework, Forbes (Jan. 28, 2023). ↑
- While educators might recognize a plagiarized paper based on familiarity with the source material, they typically use a plagiarism detector that has been licensed by the university. The detection software compares a student’s paper to large databases of books, academic papers, and other student papers and identifies the percentage of the paper that matches source material. ↑
- Geoffrey A. Fowler, We Tested a New ChatGPT-Detector for Teachers. It Flagged an Innocent Student., Wash. Post (Apr. 3, 2023). ↑
- Benj Edwards, OpenAI Confirms that AI Writing Detectors Don’t Work, Ars Technica (Sept. 8, 2023). ↑
- Geoffrey Fowler, Detecting AI May Be Impossible. That’s a Big Problem for Teachers., Wash. Post (June 2, 2023). ↑
- Is AI-Generated Content Actually Detectable?, Univ. of Maryland (May 30, 2023). ↑
- Id. ↑
- See, e.g., Susan D’Agostino, Turnitin’s AI Detector: Higher-Than-Expected False Positives, Inside HigherEd (June 1, 2023). ↑