PISA Warns AI Hurts Learning: How to Use It to Actually Learn More
OECD PISA data and new studies warn AI hurts learning when students outsource thinking. What the evidence says about using AI to actually learn more.
The OECD, the body behind the PISA assessments, has warned that unguided use of artificial intelligence risks weakening student learning rather than accelerating it. The caution builds on PISA 2022 data, published in 2023, showing that 65% of students across OECD countries report being distracted by digital devices during maths lessons. With generative AI now embedded in homework, essays and exam prep, education officials argue the pattern could repeat itself — at far greater scale.
The warning matters because the debate has shifted. The question is no longer whether students will use AI — surveys across the US, UK and Europe suggest most already do — but whether the way they use it builds knowledge or quietly erodes it. A growing body of peer-reviewed evidence now separates the two, and the difference comes down to design, not access.
- PISA 2022 found that 65% of students in OECD countries get distracted by digital devices in maths lessons (OECD, 2023).
- A University of Pennsylvania study (2024) found students with unrestricted ChatGPT access scored 17% worse on exams than peers who never used it.
- The same study showed a tutor-style AI, built to guide rather than answer, raised practice performance without damaging exam results.
- The OECD has confirmed that PISA 2029 will introduce a Media and AI Literacy assessment for 15-year-olds.
Why PISA Warns AI Hurts Learning
The OECD’s concern that AI hurts learning rests on a decade of PISA evidence about technology in classrooms. PISA 2022 data (OECD, 2023) showed that students spending more than five hours a day on digital devices for leisure scored roughly 49 points lower in mathematics — nearly two and a half years of schooling.
The organisation’s position is not anti-technology. Its analyses consistently find that moderate, purposeful use of digital tools correlates with better outcomes, while passive or excessive use correlates with worse ones. Generative AI sharpens that divide. Unlike a search engine, a chatbot can complete the entire cognitive task — reading, reasoning, writing — leaving the student with a finished product and no residual skill.
That distinction explains why the OECD is moving assessment itself. The organisation confirmed that PISA 2029 will include a Media and AI Literacy component, testing whether 15-year-olds can evaluate, question and direct AI systems rather than simply consume their output. It is the first time the world’s largest education benchmark will measure how students think with machines, not just without them.
What the Research Actually Shows
A University of Pennsylvania field experiment with nearly 1,000 high school students (Bastani et al., 2024) found that unrestricted ChatGPT access boosted practice scores by 48%, yet those same students performed 17% worse on the closed-book exam than a control group that never touched the tool.
The researchers titled the paper bluntly: «Generative AI Can Harm Learning.» The mechanism they identified is the one PISA analysts fear most. Students used the chatbot as an answer engine, copied solutions, and mistook fluency during practice for genuine understanding. When the tool disappeared, so did the performance.
Neuroscience is beginning to corroborate the behavioural data. A 2025 MIT Media Lab study using EEG monitoring found that participants who wrote essays with ChatGPT showed the weakest brain connectivity of three groups, and 83% could not quote a single sentence from a text they had produced minutes earlier. The researchers called the effect «cognitive debt»: effort saved now, capability lost later. The pattern echoes what a StudyVerso analysis of why AI study habits fail to lift grades documented among university students — heavy chatbot use with no measurable academic gain.
The Line Between Offloading and Learning
The same Pennsylvania study (2024) contained a second finding that reframes the debate: students given a «GPT Tutor» — the same model, but prompted to give hints instead of answers — improved during practice and showed no drop on the final exam. The damage was not caused by AI; it was caused by outsourcing.
Cognitive science has a name for what the tutor version preserved: retrieval effort. Learning consolidates when students struggle to recall and construct answers themselves. An AI that removes the struggle removes the learning. An AI that structures the struggle — questioning, hinting, correcting — behaves like the human tutors that decades of research rank among the most effective interventions in education.
| Dimension | AI as answer engine | AI as tutor |
|---|---|---|
| Student’s role | Copies and submits output | Attempts first, then gets guided feedback |
| Practice performance | Rises sharply (+48% in UPenn trial) | Rises moderately |
| Exam performance | Falls below non-users (−17%) | Equal or better than non-users |
| Skill retained without the tool | Low | High |
The EdTech market is already splitting along this line. Khan Academy’s Khanmigo refuses to hand over answers by design, and a wave of European apps — Spanish startups such as Modo Cheto alongside established players like Memrise — are experimenting with Socratic modes that force students to respond before the model does. Whether students choose those modes when a frictionless chatbot sits one tab away remains the open commercial question.
How Students Can Use AI to Learn More, Not Less
The evidence base points to a consistent rule: AI improves learning when it increases the student’s cognitive work per minute, and hurts it when it substitutes for that work. The OECD’s 2023 guidance on generative AI in education reaches the same conclusion — outcomes depend on pedagogy, not on the model.
Translated into practice, the research supports a short set of behaviours:
- Attempt every problem before opening the chatbot, so the AI corrects thinking instead of replacing it.
- Ask for hints, counterexamples and Socratic questions rather than finished solutions.
- Use the model to generate practice tests and quiz yourself — retrieval practice is the best-evidenced study technique in cognitive psychology.
- Explain your answer back to the AI and ask it to attack the weakest point.
- Close the tool and reproduce the work from memory before considering the session finished.
None of this is intuitive, which is why researchers keep finding the gap between how students believe they study with AI and what actually moves their grades — a gap explored in detail in this breakdown of common AI study mistakes.
What It Means for Students, Teachers and the Sector
For institutions, the PISA warning converts AI literacy from an elective into an assessed competence: by 2029, the OECD will benchmark countries on it. For students, the message is narrower — the evidence says AI hurts learning only under a specific, avoidable pattern of use.
Universities face the harder structural problem. Take-home essays and problem sets no longer certify individual ability, pushing assessment back toward supervised exams, oral defences and process-based grading. Teachers, meanwhile, inherit a new responsibility that few have been trained for: teaching students to manage a tool that is more patient than any instructor and more dangerous than any shortcut.
«Technology can amplify great teaching, but great technology cannot replace poor teaching.»
A decade after Schleicher wrote that line about laptops, it reads as a preview of the AI era. The variable was never the device; it was the pedagogy wrapped around it.
The unresolved question is who adapts first. If curricula, assessment and EdTech products redesign themselves around guided AI use before students consolidate the answer-engine habit, the OECD’s warning becomes a course correction. If they don’t, PISA 2029 may deliver the first international measurement of what four years of frictionless answers did to a generation’s ability to think without them.
