You are currently viewing Evidence of AI Tutors’ Effectiveness

Evidence of AI Tutors’ Effectiveness

While AI is being used to make self-tests, study aids, and a wide range of other learning materials, tutoring systems have arguably the most potential to improve student understanding and performance. Programmed to mimic human tutors, these systems engage in a Socratic dialogue with the student by asking questions and providing structure to guide them through the learning. Students can also ask the tutor to focus on particular areas where they struggle. In this way, AI personalizes learning by giving students the specific help they need.

Online AI tutors are available anywhere, anytime, allowing students to get help when they encounter difficulties in their studies rather than having to wait until scheduled tutoring sessions or office hours. They are not designed to replace instructors or human tutors but can act as a first place to turn for help. Students can start with the AI tutor and, if they still have questions, meet with their instructor afterward.

Despite such advantages, AI tutors are only slowly making their way into instruction, with many faculty and institutions wary of adopting them. One worry is that AI tutors will give students wrong information; another is that students will think instructors are shunting their teaching duties to a machine.

I doubt that this latter concern is on many students’ minds. Students receive a variety of learning materials beyond the wisdom that emanates from the instructor’s words in lectures, such as textbooks, videos, and articles, and an AI tutor is just another learning resource. Plus, students are already using AI to supplement their learning by creating practice tests and other learning content (Stein, 2025), and so they already see it as a legitimate and important learning tool.

But what about the issue of AI accuracy? Do these tutors truly help students learn, and how do they compare to human tutors? These questions have been examined in multiple studies, and the results are overwhelmingly positive. One study in Germany (Fischer et al., 2025) divided 334 university students into three groups to examine how (a) textbook-only learning compares to (b) textbook learning followed by AI tutoring and (c) textbook learning mixed with AI tutoring. The researchers included group B to see whether students benefited from getting “reliable” textbook answers before they turned to possibly unreliable AI tutoring. All three groups then took a test on the material.

The study found that group B performed at .21 standard deviations better on the test than group A, but group C performed at .34 standard deviations better than group A, a result that surprised even the researchers. It suggests that students don’t need an initial baseline of textbook information as blending AI tutoring with textbook instruction produces the greatest benefit. We live in a world where people whip out the ubiquitous smartphone for just-in-time answers to questions that arise throughout the day, and students use the same method in their studies. Offering an AI tutoring complement to that only improves learning.

Another study by economics professors at the University of Wisconsin–La Crosse (Babaei-Baderlou & Shakya, 2026) compared groups along two axes: use of an AI tutor and use of group study. The researchers put students into four groups to prepare for their course exams: (a) those who studied alone, (b) those who studied in groups, (c) those who studied alone with an AI tutor, and (d) those who studied in groups with an AI tutor. As in the prior study, the researchers had students take an exam after studying to determine which method produced the best scores.

The researchers found that the students who both used an AI tutor and worked in groups performed the best. Next best were those who worked alone with an AI tutor. Third best were the students who studied alone, while those who studied only in groups did the worst. This means that the top two groups were those who used the AI tutor. The fact that the group-study-only group did worse than the individual-study-only group is interesting. Perhaps group study sessions tend to degenerate into chit-chat about other topics, while an AI tutor acts as a kind of maypole that draws students back to the topic at hand.

These studies suggest that AI tutoring produces better learning than no AI tutoring. But what about human versus AI tutoring? It turns out that there are numerous studies of this topic, with one meta-review of 50 studies (Kulik & Fletcher, 2016) finding that intelligent tutoring systems can “match the success” of human tutoring, while another meta-review of 22 studies (Nesbit et al., 2014) found that intelligent tutoring systems offer “a significant advantage” over traditional classroom instruction. These systems from 10 years ago were mere infants compared to current ones, suggesting that the AI performance is likely better today.

What to do

All well-known LLMs, such as ChatGPT, Gemini, and Perplexity, offer the ability to create a custom tutor, but my recommendation is to start with NotebookLM. This Google product is designed for learning, and many people are already using it to learn topics on their own. It allows the user to upload sources, such as readings, as well as links to websites and videos for the system to use as its knowledge base, and then be tutored on that material. An instructor or student can simply connect all the course resources on a particular topic to a chat and then ask questions, be quizzed on the topic, get a summary, or just ask to be taught the information. The interface is both attractive and intuitive, and the variety of options offers a personalized learning experience to the user.

I have found NotebookLM, as well as the other systems, to offer excellent information on a topic, and I have yet to encounter a clear “hallucination.” The bigger issue is that an instructor needs to get over the feeling that they need to control all student learning. As faculty, we can fall into the trap of thinking that our own way of explaining a topic is just plain right and any other explanation is thereby wrong. But no two people will use the exact same words to explain a topic. It’s best to liken an AI tutor to a teaching assistant whose words are not identical to those of the instructor but who is nevertheless a valuable teaching aid.

References

Babaei-Balderlou, S., & Shakya, S. (2026, March 5). We designed an AI tutor that helps college students reason rather than give them answers. The Conversation. https://theconversation.com/we-designed-an-ai-tutor-that-helps-college-students-reason-rather-than-give-them-answers-276584

Fischer, M., Rau, H. A., & Rilke, R. M. (2025, December). AI tutoring enhances student learning without crowding out reading effort. IZA Institute of Labor Economics. https://docs.iza.org/dp18338.pdf

Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of intelligent tutoring systems: A meta-analytic review. Review of Educational Research, 86(1), 42–78. https://doi.org/10.3102/0034654315581420

Nesbit, J. C., Adesope, O. O., Liu, Q., & Ma, W. (2014). How effective are intelligent tutoring systems in computer science education? ICALT ’14: Proceedings of the 2014 IEEE 14th Annual Conference on Advanced Learning Technologies, 99–103. https://doi.org/10.1109/ICALT.2014.38

Stein, M. B. (2026, February 25). New Coursera report shows that 95% of students and educators are using AI on campus—but only a quarter of educators worldwide feel prepared to use it effectively. Coursera Blog. https://blog.coursera.org/ai-in-higher-education-report-2026


John Orlando is a professor of philosophy at Colorado Technical University and an educational consultant who leads workshops for faculty on how to use AI and other technologies to improve their teaching: jorlando2001@gmail.com.

The post Evidence of AI Tutors’ Effectiveness appeared first on The Teaching Professor.