The Label Changes Everything

What a controlled experiment reveals about how students respond to feedback they believe came from a human vs. an AI

Duration: 30 minutes | Pre-class activity available for download 

Who This Is For: This lesson is designed for instructors, instructional designers, learning experience professionals and EdTech product managers who make decisions about how AI-generated feedback is deployed in educational settings. It is also relevant to computing educators working in hybrid environments where AI tools and human instructors operate side by side. If you have ever wondered whether labeling feedback as "AI-generated" changes how students engage with it, or whether presenting AI-drafted feedback under your own name is a neutral act, this lesson was written for you. Researchers in educational technology, human-computer interaction and applied cognitive psychology will find the experimental design and theoretical framing directly applicable to their work.

Real-World Applications

Large introductory programming courses at universities increasingly rely on LLMs to deliver personalized code feedback at scale, a task that human instructors cannot sustain across hundreds of students per semester. The research underpinning this lesson was conducted in exactly that context: a controlled experiment in which all feedback was generated by the same LLM but attributed differently across participant groups. The findings have immediate design implications for any platform that deploys AI-generated feedback and must decide how to label it. They are equally relevant to instructors who use AI to draft feedback that they then deliver under their own name, a practice that is growing in higher education and corporate learning environments.

The Problem and Its Relevance

When educators assume that labeling AI-generated feedback as human will increase student engagement, they are making a gamble that depends entirely on whether students actually believe the claim. In a three-condition experiment conducted at MIT, 46 percent of students who were told their feedback came from a human teaching assistant did not believe it, and those students produced the least complex code and spent the least time on task of any group in the study, performing worse than students who received openly AI-attributed feedback. The problem is not that AI feedback is ineffective: it is that a false or unbelievable claim about feedback authorship can actively harm learning outcomes, and that harm is proportional to the degree of distrust it triggers.

The second distinct issue is that educators and researchers have long assumed the motivational benefit of human feedback operates through social pressure, meaning students work harder because they feel watched and evaluated. The experiment found no evidence for this. Social presence scores were flat across all conditions, including among students who believed a human had reviewed their work. What actually differentiated high-performing students from low-performing ones was intrinsic motivation and perceived authenticity, not surveillance. This distinction matters because it changes the entire logic of how human involvement in feedback should be designed and communicated.

What the Research Found (and Why It Matters)

The experimental design

The study recruited 148 adult participants through an online research platform and assigned them to one of three conditions. All participants completed a self-paced creative coding tutorial using p5.js and received feedback generated by the same LLM at four checkpoints. The only differences between conditions were the attributed source of the feedback and the delivery timing. In the AI-Instant condition, feedback arrived after 5 seconds and was labeled as AI-generated. In the AI-Delayed condition, feedback arrived after 60 seconds and was also labeled as AI-generated. In the TA-Delayed condition, feedback arrived after 60 seconds but was labeled as coming from a human teaching assistant. This three-condition structure was designed to separate the effect of source attribution from the effect of delivery timing, a confound that prior studies had not controlled for.

What source attribution did

Students who believed their feedback came from a human teaching assistant spent approximately 28 percent more time on task and maintained more active focus than students who received equivalently timed AI-attributed feedback. These effects appeared in process measures, meaning how long and how actively students worked, rather than in the quality of what they produced at this stage of the analysis. The delivery delay, independent of source, did increase code length and complexity, suggesting that a natural pause before receiving feedback may support reflection. Source attribution and delivery timing operated through different mechanisms and produced different outcomes.

What happened when the attribution was not believed

Among the 50 participants told their feedback came from a human, 23 (46 percent) reported disbelief. This group showed a strikingly different behavioral profile from every other group in the study. They produced the least complex code, wrote the shortest programs, skimmed their feedback at nearly three times the rate of other participants, reported the lowest intrinsic interest in the tasks and demonstrated the lowest rates of autonomous effort. Their performance was significantly worse than that of students who received openly AI-attributed feedback, not merely equivalent. The researchers describe this as consistent with both self-determination theory and the trust violation literature: when students perceive that they are being deceived about the source of feedback, the experience does not simply neutralize the benefit of human attribution. It actively reduces engagement below what transparent AI attribution would have produced.

Why motivation, not surveillance, drives the effect

Post-survey items measuring social presence, including feeling watched, feeling evaluated and wanting to impress the reviewer, showed no meaningful differences across any of the four groups. Students who believed a human reviewed their work did not report feeling more observed than those who knew they received AI feedback. What did differentiate groups was intrinsic interest in the tasks and autonomous effort, meaning effort that students reported they would have invested regardless of whether anyone was reviewing their work. This finding is consistent with self-determination theory's account of relatedness: the sense that one's work is genuinely seen and valued by another person may support intrinsic motivation through perceived authenticity rather than through external evaluation pressure.

The Bottom Line

Transparency about AI feedback is not merely an ethical preference: it is a measurable performance variable. When students suspect that claimed human feedback is actually AI-generated, their learning outcomes fall below what they would have achieved had the AI been identified honestly from the start. An institution that uses AI to draft feedback and presents it as human-authored is not delivering a neutral experience. It is running a credibility test that a growing percentage of AI-literate students will fail on the institution's behalf.

The deeper implication challenges a foundational assumption in educational design: the idea that what makes human feedback motivating is the sense of being evaluated and held accountable. The evidence from this study points instead to authenticity and perceived genuine interest as the operative mechanism. This means that the question instructors should be asking is not "how do we make AI feedback feel more human" but "how do we preserve the conditions under which students believe their work is genuinely seen and valued." Those are not the same question, and they do not have the same answers.

#AIFeedbackAttribution #HumanVsAIFeedback #AIInEducation #SourceCredibilityEffect #ComputingEducationResearch