Researchers at the University of Reading have found that artificial intelligence (AI)-generated exam answers have gone almost entirely undetected and consistently achieved higher grades than those written by real students.
The study involved submitting answers generated by ChatGPT-4 on behalf of 33 fictitious students to the School of Psychology and Clinical Language Sciences’ examination system.
Researchers at the University of Reading have found that artificial intelligence (AI)-generated exam answers have gone almost entirely undetected and consistently achieved higher grades than those written by real students.
The study involved submitting answers generated by ChatGPT-4 on behalf of 33 fictitious students to the School of Psychology and Clinical Language Sciences’ examination system.
The study focused on exams that required short answers or essay responses. The short answer exams had a 2.5-hour window, while essays had an 8-hour window.
“Both were unsupervised take-home exams, so students could have easily used AI within those time frames,” Associate Professor Scarfe highlighted.
They speculate that some real students might have also successfully submitted AI-generated work during the study.
ChatGPT or human, content matters
However, the researcher suggests that question content, rather than type, can be a more significant factor.
“This could correlate with question types e.g. essays generally allow more complex reasoning to be demonstrated than multiple choice questions,” Scarfe said.
Notably, the study employed two distinct approaches for comparing ChatGPT-generated answers with those of real students.
“In comparisons in the paper, we either (1) compared all AI versus all student submissions (overall or specific modules), or (2) in the resampling analysis, randomly sampled from student submissions and compared it to the AI,” added Dr Scarfe.
Call for adaptation
“From an academic integrity standpoint, these findings are of extreme concern,” the researchers stated in the press release.
Professor Scarfe emphasized that the findings should serve as a “wakeup call” for educators worldwide.
“We won’t necessarily go back fully to hand-written exams, but the global education sector will need to evolve in the face of AI,” he said.
Professor Elizabeth McCrum, Pro-Vice-Chancellor for Education and Student Experience at the University of Reading, also acknowledged the transformative effect AI will have on education.
They have undertaken a huge program of work to consider all aspects of teaching, including making greater use of technology to enhance the student experience and boost graduate employability skills, she emphasized.
A broader discussion
Meanwhile, the researchers call for a broader discussion about the role of AI in society and the importance of maintaining academic and research integrity in this evolving landscape.
Source: IE
