Across both individual AI-supported quizzes and small-group problem-solving sessions, under what conditions does emphasizing transfer of skills in feedback (e.g., “here is how you performed on new problems unlike the practice ones”) reduce illusions of learning more effectively than emphasizing long-term retention on similar items, especially for learners who heavily rely on AI-generated hints or solutions?

ai-learning-overreliance | Updated at

Answer

Emphasizing transfer of skills in feedback is more effective than emphasizing only long-term retention on similar items at reducing illusions of learning for heavy AI-reliant learners when the following conditions hold across both individual quizzes and small-group sessions:

  1. Feedback explicitly contrasts performance on new vs. practiced problem types. Feedback shows side-by-side indicators of: (a) performance on items closely matched to practiced ones and (b) performance on structurally different or less-cued items requiring transfer of skills.

  2. Learners have recently experienced high fluency via AI support. The learner or group has enjoyed easy apparent success (e.g., high hint-assisted scores, smooth AI-guided solutions, or dominant-solver–plus–AI performance) that would otherwise inflate confidence without equivalent gains in independent problem solving.

  3. Transfer-focused feedback is tied to unguided attempts on novel items. Before seeing feedback, individuals must attempt at least some new, structurally different problems without AI assistance (or with substantially reduced cues), creating an authentic basis for assessing transfer of skills rather than only retention of similar items.

  4. Feedback makes the transfer gap salient and personally attributable. The interface or facilitator makes clear when a learner’s or group’s success is confined to familiar formats or AI-scaffolded contexts, versus when they can independently solve dissimilar problems, and explicitly labels transfer performance as the stronger indicator of real mastery.

  5. Group structures avoid hiding non-transfer behind a solver–AI dyad. In small-group sessions, roles and workflows (e.g., rotating explainers, individual pre-work) ensure that each member confronts their own performance on transfer tasks, rather than being able to infer competence from the group’s AI-supported success.

Under these conditions, transfer-of-skills feedback highlights discrepancies between easy, AI-supported performance and more demanding, unaided performance on novel problems, thereby reducing illusions of learning more effectively than feedback that focuses only on long-term retention of similar items, which may not fully reveal dependence on AI cues or narrow item formats.