Generative AI and critical thinking: Implications for university education

This article is an overview for university faculty based on "The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers" by Lee et al. (2025). After summarizing the paper in Japanese with gemini-paper-summarizer https://github.com/7shi/gemini-paper-summarizer, I used Claude 3.7 Sonnet thinking to format it as a dialogue for university faculty. The featured image was created with Gemini 2.0 Flash Experimental. Please refer to the original paper for further details and exact wording.

Original paper: Lee, Hao-Ping Hank, et al. “The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers.” (2025).

https://www.microsoft.com/en-us/research/publication/the-impact-of-generative-ai-on-critical-thinking-self-reported-reductions-in-cognitive-effort-and-confidence-effects-from-a-survey-of-knowledge-workers/

A dialogue between a researcher and an educator: How generative AI is changing thought processes and education

Person A: Lately I've been thinking a lot about how generative AI affects university education, and a new study by Lee et al. (2025) offers some interesting findings. What do you think is the main takeaway from this research for educators?

Person B: Yes, this study came out at a crucial time. The paper's main message is that generative AI is fundamentally changing how knowledge workers engage in critical thinking. Particularly notable is the correlation showing that higher trust in generative AI leads to less critical thinking, while higher self-confidence leads to more critical thinking.

Person A: What specific effects did they observe?

Person B: According to the study, using generative AI changes the nature of critical thinking itself. It revealed a shift away from traditional critical thinking processes in three directions:

From information gathering to information verification: Because generative AI automates information gathering, knowledge workers now spend their effort verifying the accuracy of information.
From problem-solving to integrating AI responses: The focus has shifted from solving problems directly to properly integrating AI outputs into one's own context.
From task execution to task stewardship: Rather than executing complex tasks, workers now place greater emphasis on evaluating and managing the quality of AI-generated outputs.

Person A: I see. I'm noticing this in the classroom as well. The quality of students' reports and presentations seems to have improved, but if their critical thinking processes are changing, we need to rethink our educational approach too.

Person B: Exactly. The study analyzed 936 real-world examples of generative AI use across 319 knowledge workers. The findings carry important implications for educators. A particular concern is that overreliance on generative AI may weaken independent problem-solving skills.

Person A: That's an important point. What factors does the paper identify as influencing critical thinking?

Person B: The study points out that task-specific self-confidence and trust in generative AI are particularly important factors affecting the enactment and effort of critical thinking. Interestingly, these factors have a stronger influence than job type or years of experience. The study also identifies three barriers to critical thinking: awareness, motivation, and ability.

Person A: Could you explain those three barriers in more detail?

Person B: First, the "awareness barrier" refers to failing to recognize the need for critical thinking. This includes assuming that generative AI outputs are accurate or missing opportunities to think critically. The "motivation barrier" is a lack of motivation to engage in critical thinking; relying on generative AI feels easier, which reduces the drive to think deeply. The "ability barrier" is a problem with the capacity to think critically itself, where overreliance on generative AI degrades critical thinking skills.

Person A: Based on these findings, how should we as educators change our instructional design?

Person B: There are several concrete suggestions.

Proposals for instructional design

1. Strengthening critical verification skills

Person B: To build the ability to critically evaluate generative AI outputs, we should strengthen information literacy education. For example:

Introduce "fact-checking exercises" for generative AI outputs
Design workshops that compare and examine multiple sources and pieces of evidence
Explicitly teach criteria for evaluating the reliability of information
Provide opportunities to systematically learn about AI limitations and error patterns

Person A: Those are interesting suggestions. In fact, I've already started an exercise in my class where students compare ChatGPT responses with academic papers. They've come to understand both the limitations and strengths of AI.

2. Developing integration and editing skills

Person B: According to the study, the ability to properly integrate generative AI outputs into one's own context has become vital. To develop this skill:

Practice paraphrasing AI-generated content in one's own words
Assign tasks that synthesize multiple generative AI responses to create new perspectives
Evaluate the process of critically editing AI outputs
Develop and use checklists to assess the suitability of AI outputs

Person A: That's a key point. Rather than simply copying and pasting AI outputs, students need the ability to integrate them into their own thought processes. We will likely need to update our grading criteria to match.

3. Fostering metacognitive skills

Person B: The paper notes that self-efficacy is a crucial factor when using generative AI. To boost students' self-efficacy and metacognitive skills:

Introduce "thinking journals" where students reflect on their own thought processes
Create portfolios that make the process of collaborating with AI explicit
Assign roles and hold debriefing sessions when using AI in teams
Design assignments where students analyze how their thought processes changed before and after using AI

Person A: Metacognition is becoming increasingly important. By reflecting on how they thought while using AI, students can become more aware of their own cognitive processes.

4. Redesigning assessment methods

Person B: We also need to change how we assess students in the age of generative AI.

Evaluate both the process and the final output
Include transparency of AI use in grading criteria
Develop new rubrics that explicitly assess the "ability to collaborate with AI"
Maintain a good balance between assignments that assume AI use and those that prohibit it

Person A: Updating assessment methods is an urgent issue. Instead of measuring the traditional ability to "research and summarize," we need ways to measure the ability to "collaborate with AI to create."

5. Addressing barriers to critical thinking

Person B: Lee et al.'s study identified three barriers: awareness, motivation, and ability. We should also consider educational approaches tailored to each of them.

For the awareness barrier: Develop teaching materials that deliberately demonstrate AI errors and biases
For the motivation barrier: Provide experiential learning where students can feel the added value of critical thinking
For the ability barrier: Ensure opportunities to train foundational critical thinking skills independently

Person A: I see. Directly addressing the barriers to critical thinking in the generative AI era should allow us to teach more effectively.

Practical approaches

Person A: What kinds of approaches could we use to incorporate these suggestions into an actual curriculum?

Person B: Here are some specific practical approaches:

AI co-creation workshops: Students collaborate with AI to solve problems while recording and analyzing all the prompts used, the editing process, and the final product

Critical thinking challenges: Deliberately plant errors in AI-generated information and challenge students to find and correct them

Reverse-engineering assignments: Design tasks where students infer the original questions and assumptions behind an AI response

Comparative analysis seminars: Compare analyses by human experts and AI on the same topic, discussing the strengths and limitations of each

Barrier-breaking workshops: Have students identify the three barriers to critical thinking (awareness, motivation, and ability) and develop their own strategies to overcome each one

Person A: Those are great ideas. The critical thinking challenge in particular seems like a good countermeasure against students' current tendency to accept AI outputs uncritically.

Person B: Yes. It is important to position AI not just as a tool, but as a cognitive partner. Through collaboration with AI, students can expand their own thought processes and develop higher-order critical thinking skills.

Reallocating cognitive resources: New cognitive loads

A: Something I have noticed recently is that when students use generative AI, the center of gravity in their thinking seems to shift from solving the problem itself to verifying and adjusting the answers the AI generates. Has research observed this too?

B: Yes, that is a very sharp observation. Research findings make it clear that using generative AI significantly changes how we allocate our cognitive resources. Mental resources that we previously spent on understanding the problem itself and devising solutions are now directed toward verifying the answers the AI presents, and editing and integrating them to fit our needs.

A: So this represents a qualitative change in how we think, rather than just an increase in work efficiency.

B: Exactly. We can explain this phenomenon in terms of both "cognitive offloading" and "cognitive rebalancing." Delegating (offloading) part of problem-solving to AI does reduce certain cognitive loads, but in their place, new cognitive demands emerge, such as the following:

Evaluating output suitability: Judging whether the AI's response fits one's purpose
Verifying reliability: Assessing whether the provided information is accurate
Integrating multiple suggestions: Combining different AI responses and existing knowledge
Adapting to context: Tailoring general answers to specific situations

In the study by Lee et al., this shift in cognitive effort was observed specifically as a move from "application" to "response integration," and from "analysis" to "task management."

A: I see. That is an interesting perspective. Does this mean that in addition to problem-solving skills, students need to acquire "meta-problem-solving skills" to collaborate effectively with AI?

B: Precisely. The study by Lee et al. highlights the importance of these "meta-problem-solving skills." Critical thinking in the age of generative AI encompasses not only thinking about the problem itself, but also thinking critically about the process of collaborating with AI. This includes judgments such as "What kind of prompt will draw out a higher-quality answer?" and "Which parts of the AI's response should I adopt, and which parts should I revise?"

A: That has major implications for education. We need to teach students how to collaborate effectively with AI while ensuring they do not lose the ability to solve problems on their own.

B: Right. As educators, we must take care that students do not become overly dependent on AI and let their independent problem-solving abilities atrophy. Ideally, we should aim for a state where students can solve problems on their own, yet achieve even better results by collaborating with AI. To reach that point, we need to deliberately provide both opportunities to tackle problems without AI and opportunities to work alongside it.

A: Viewing this as a reallocation of cognitive resources will likely affect assessment methods as well. Abilities that traditional evaluation criteria cannot measure seem to be growing more important.

B: You are quite right. Beyond traditional assessments that ask whether a student can arrive at the right answer, we now also need evaluations that ask whether they can properly assess, edit, and integrate AI outputs, and how well they manage the collaborative process with AI. You could say this offers a new opportunity to measure higher-order critical thinking skills.

Conclusion: Toward a new critical thinking education

A: To sum up today's discussion, critical thinking education in the age of generative AI needs to shift from the traditional framework of "analyzing and evaluating information" toward cultivating the ability to "verify, integrate, and manage information in collaboration with AI." Furthermore, from the standpoint of cognitive resource allocation, we need to understand that the emphasis is moving from problem-solving itself to managing the collaborative process with AI, and design our instruction accordingly.

B: Yes, exactly. As educators, rather than banning AI, we need to build a new educational paradigm that fosters critical thinking through smart collaboration with it. As the study by Lee et al. shows, without proper instructional design, AI can hinder critical thinking, but when used appropriately, it can serve as a catalyst for cultivating even higher-order thinking skills.

A: In this period of transition, it is vital that we as educators continue to learn, experiment, and share our insights with one another. Advancing research into the relationship between generative AI and critical thinking will make more effective educational design possible.

*This article is structured around a fictional dialogue. In actual curriculum design, adjustments are needed to match each institution's policies and student needs.