An AI Tutor Lowered Final Grades in a 2,379-Student Trial. Only One in Seven Students Ever Used It.
Universities are racing to plug AI tutors into their courses on the promise that personal, always available help will lift grades. A new randomized trial from the University of Maryland points the other way, and the strangest part of the result is how few students actually used the tutor at all.
The study is a working paper, posted to EdWorkingPapers in October 2026 and not yet peer reviewed ([EdWorkingPapers](https://edworkingpapers.com/ai26-1598)).
What the trial found
Researchers randomly assigned 30 instructors, teaching 2,379 undergraduates in the fall 2025 semester, either to receive a virtual study assistant for their sections or not. The assistant was a GPT-4o tutor built into the course site and drawing on course materials. Courses ranged from biology and computer science to English, education and public policy ([UMD EDSI](https://edsi.umd.edu/publications/effects-course-integrated-ai-tutoring-student-performance-and-engagement-randomized)).
When the comparison was limited to sections of the exact same course, access to the tutor reduced final grades by 0.37 standard deviations and participation on the learning platform by 0.90 standard deviations ([working paper PDF](https://edworkingpapers.com/sites/default/files/ai26-1598.pdf)). Across the full sample the drop in participation was similarly large, while the grade effect was smaller and not statistically clear. Losses were larger for first generation students, at roughly 0.50 standard deviations against 0.22 for everyone else in the full sample.
The tutor almost nobody opened
Here is the puzzle. Only 14.63% of students in treated classrooms ever used the tutor. Of those, about 40% had a single session, and users averaged roughly four. Of 3,460 requests sent to the tool, 73.8% asked for a direct answer rather than help working something out ([working paper PDF](https://edworkingpapers.com/sites/default/files/ai26-1598.pdf)).
So how can a tool that most students ignored move a whole class? The authors argue that the trial probably measured more than the tutor itself. Putting an AI assistant inside the course space may have changed the wider AI environment of the class, and the tool may have substituted for the interactions students would otherwise have had with the course. Their summary is that the tutor may displace engagement with existing learning activities without improving performance.
Read it with both hands
This is not the only rigorous evidence, and it does not stand alone. A 2025 randomized trial in an introductory physics course at Harvard found the opposite: students using a carefully designed AI tutor learned more in less time, with an estimated effect of 0.73 to 1.3 standard deviations ([Scientific Reports](https://www.nature.com/articles/s41598-025-97652-6)). That tutor was built around a research based teaching design, while the Maryland tool was a general study assistant added to many different courses.
The Maryland authors also name their own limits. Grades are not independent learning tests, platform clicks capture only part of studying, sections can differ in how they grade, and the inference rests on a small number of clusters, 34 in all.
What the trial does make hard to argue is that adding an AI tutor is a free upgrade. Design, not access, seems to decide the outcome.
Takeaway: An AI tutor that mostly hands out answers can lower grades and participation even when most students never open it, so universities should test the design before they roll it out.