Study says humans are switching off logic and blindly trusting AI chatbots
A new study reveals that people are increasingly abandoning their own reasoning and blindly trusting AI chatbots. It warns that while AI can boost performance when correct, it can also mislead users if they don't double-check its answers.
AI is helping humans. It’s making them work faster. It’s boosting productivity. However, amid this increasing reliance, there’s a flip side. AI may also be making humans lazier, pushing them to abandon their own logic and trust it blindly. A new study from researchers at the University of Pennsylvania suggests that, as the use of AI tools continues to grow, people are increasingly giving up their own reasoning and placing blind trust in AI chatbots.
The research describes this behaviour as “cognitive surrender”, a tendency where humans accept AI-generated answers with little to no scrutiny, even when those answers may be incorrect.
At the heart of the study, titled "Thinking—Fast, Slow, and Artificial: How AI is Reshaping Human Reasoning and the Rise of Cognitive Surrender", is a new framework called Tri-System Theory, which expands how we understand human thinking.
Humans are outsourcing reasoning from AI
Traditionally, decision-making has been divided into two modes: fast, intuitive thinking (System 1) and slow, analytical reasoning (System 2). The researchers argue that AI introduces a third mode — “System 3”, or artificial cognition — where reasoning is effectively outsourced to machines.
And humans are now said to be significantly putting their trust into AI. The research highlights that, unlike tools such as calculators or GPS, which people typically use with oversight, AI systems appear to encourage a deeper level of dependence. According to the study, when AI responses are delivered fluently and confidently, users are far more likely to accept them without question. In such cases, AI doesn’t just assist thinking, it can replace it.
To test this, researchers conducted experiments using Cognitive Reflection Tests (CRT), which are designed to distinguish between intuitive and analytical thinking. Participants were given optional access to an AI chatbot that provided correct answers only half the time, while deliberately offering incorrect responses in the remaining cases. Despite this, users frequently deferred to the AI.
According to the study, participants accepted AI-generated answers about 93 per cent of the time when they were correct, and still followed them nearly 80 per cent of the time when they were wrong.
Researchers note that this behaviour also had a direct impact on performance. When the AI was accurate, participants performed better than those relying solely on their own reasoning. However, when the AI was faulty, their performance dropped below baseline levels. In other words, users’ decision-making became closely tied to the quality of the AI, improving or deteriorating accordingly.
AI is building false confidence
Perhaps more concerning is the effect on confidence. The study found that participants who used AI reported higher confidence in their answers, even though the chatbot was wrong half the time. This suggests that AI not only influences decisions but may also create a false sense of certainty.
Overall, the findings show that people accepted faulty AI reasoning more than 70 per cent of the time, while actively overriding it in less than 20 per cent of cases. The study also found that people who naturally trust AI were misled more often. However, those who think more analytically were better at spotting mistakes.
That said, the researchers note that relying on AI isn’t always a bad thing. When AI systems are highly accurate, they can actually improve decision-making. However, the real risk comes from over-relying on AI tools without checking their answers.
AI is helping humans. It’s making them work faster. It’s boosting productivity. However, amid this increasing reliance, there’s a flip side. AI may also be making humans lazier, pushing them to abandon their own logic and trust it blindly. A new study from researchers at the University of Pennsylvania suggests that, as the use of AI tools continues to grow, people are increasingly giving up their own reasoning and placing blind trust in AI chatbots.
The research describes this behaviour as “cognitive surrender”, a tendency where humans accept AI-generated answers with little to no scrutiny, even when those answers may be incorrect.
At the heart of the study, titled "Thinking—Fast, Slow, and Artificial: How AI is Reshaping Human Reasoning and the Rise of Cognitive Surrender", is a new framework called Tri-System Theory, which expands how we understand human thinking.
Humans are outsourcing reasoning from AI
Traditionally, decision-making has been divided into two modes: fast, intuitive thinking (System 1) and slow, analytical reasoning (System 2). The researchers argue that AI introduces a third mode — “System 3”, or artificial cognition — where reasoning is effectively outsourced to machines.
And humans are now said to be significantly putting their trust into AI. The research highlights that, unlike tools such as calculators or GPS, which people typically use with oversight, AI systems appear to encourage a deeper level of dependence. According to the study, when AI responses are delivered fluently and confidently, users are far more likely to accept them without question. In such cases, AI doesn’t just assist thinking, it can replace it.
To test this, researchers conducted experiments using Cognitive Reflection Tests (CRT), which are designed to distinguish between intuitive and analytical thinking. Participants were given optional access to an AI chatbot that provided correct answers only half the time, while deliberately offering incorrect responses in the remaining cases. Despite this, users frequently deferred to the AI.
According to the study, participants accepted AI-generated answers about 93 per cent of the time when they were correct, and still followed them nearly 80 per cent of the time when they were wrong.
Researchers note that this behaviour also had a direct impact on performance. When the AI was accurate, participants performed better than those relying solely on their own reasoning. However, when the AI was faulty, their performance dropped below baseline levels. In other words, users’ decision-making became closely tied to the quality of the AI, improving or deteriorating accordingly.
AI is building false confidence
Perhaps more concerning is the effect on confidence. The study found that participants who used AI reported higher confidence in their answers, even though the chatbot was wrong half the time. This suggests that AI not only influences decisions but may also create a false sense of certainty.
Overall, the findings show that people accepted faulty AI reasoning more than 70 per cent of the time, while actively overriding it in less than 20 per cent of cases. The study also found that people who naturally trust AI were misled more often. However, those who think more analytically were better at spotting mistakes.
That said, the researchers note that relying on AI isn’t always a bad thing. When AI systems are highly accurate, they can actually improve decision-making. However, the real risk comes from over-relying on AI tools without checking their answers.
