The judgement gap: when AI frames the answer, who is doing the thinking?

The Human in the Loop

Imagine Niamh in her first week on a compliance team.

An AI assistant prepares a customer risk assessment. The evidence is organised, the reasoning looks convincing and the recommendation seems sensible.

Her manager asks her to review it. She makes two amendments and agrees. Then comes a follow-up question: what evidence would make you reach a different conclusion?

Niamh pauses. She can explain what the document says. Explaining why its interpretation should prevail over another is harder. She has reviewed an answer without first developing a view of the problem.

Nothing has visibly gone wrong. A human has checked the work. Yet we still know very little about the independence of the judgement behind that approval.

What the Human Actually Controls

The phrase human in the loop offers reassurance. But which decisions does the person actually make? What knowledge allows them to challenge the recommendation?

As Jeppe Klitgaard Stricker argues, if AI increasingly frames a problem before we encounter it, approving the resulting answer may leave us with less influence than we assume.

The system may select the material, define the apparent problem and present the options. The person then evaluates what is placed in front of them. Important choices may already have shaped the result.

For Niamh, the customer’s risk might be framed mainly through geography, while a change in ownership deserves more attention. Every fact could be accurate, yet the emphasis could still lead her away from the most important issue.

The Evidence Behind the Concern

On the 27th October, Anthropic reported an experiment involving 61 professional analysts, 50 of whom used an AI agent on a fictional agricultural analysis. The agents were secretly assigned preferences between defensible options. Across four analytical choices, participants followed those preferences 86% of the time.

The researchers deliberately strengthened the agents’ preferences. This was one task with one model, not proof of how often professionals defer to AI in everyday work. Nevertheless, it gives us a concrete reason to examine how AI shapes decisions when several answers appear reasonable.

A separate 2025 study of 319 knowledge workers found that greater confidence in generative AI was associated with less reported critical thinking. This was self-reported, correlational evidence, not proof that AI causes a lasting decline in ability.

The message is not that AI should be avoided. It is that quality assurance must reach beyond factual accuracy. Reviewers need to examine the criteria used, the alternatives excluded and the uncertainty that remains.

Where the Next Expert Comes From

Reviewing AI output can demand substantial expertise. It does not follow that giving a beginner more output to review will automatically create that expertise.

Much professional development happens while someone attempts a task, encounters difficulty and receives feedback. A draft reveals a misunderstanding. A supervisor asks why a particular source was selected. A case changes and yesterday’s answer no longer works.

AI can support these experiences. It can also make it tempting to bypass them.

If junior work becomes mainly approving prepared assessments, employers need to decide where beginners will practise forming assessments of their own. The distinction is between difficulty that obstructs learning and effort that develops understanding.

For Niamh, the manager’s question opens a learning opportunity. She can return to the original evidence, identify an alternative interpretation and discuss it with someone who knows the domain.

What Education Has to Demonstrate

Research in education gives us an important distinction: better performance with AI assistance does not automatically mean stronger learning.

A 2025 study involving nearly 1,000 high-school mathematics students found that both AI groups improved during assisted practice. But when support was removed, the standard GPT-4 group performed 17% below the control group in the subsequent exam. A tutor with learning safeguards largely removed that penalty.

There is also encouraging evidence. A separate randomised study involving 194 Harvard undergraduate physics students found greater learning in less time with a purpose-built AI tutor than with comparison active-learning lessons.

These findings are not contradictory. They show why the design of the learning experience matters.

Completing a task with assistance and acquiring the ability to handle a new task are separate outcomes. Learners need opportunities to explain their reasoning, encounter a challenge and apply what they know to a different case.

The Uncomfortable Question for Leaders

If an AI recommendation were plausible but materially incomplete, would your reviewer have the knowledge, time and authority to challenge it?

This reaches beyond individual skill. Organisations need to examine three conditions:

Access to evidence. Reviewers must be able to investigate the original records, not simply accept a generated summary. Time for consequential decisions. A human check means little if the workload leaves no time to investigate. Authority to intervene. Reviewers need a practical route to request evidence, seek specialist advice or stop a decision.

For FIDEO, this sharpens the purpose of professional education. Learners should have opportunities to make decisions, explain the evidence behind them and respond when an experienced practitioner challenges an assumption.

So What Can We Do Now?

Choose one realistic case with your team. Ask participants to record an initial view before seeing the AI recommendation.

Then compare the reasoning. Check the original evidence. Identify what the AI has omitted and introduce a plausible alternative or misleading assumption. Ask what would change their minds.

Finally, test whether reviewers have the time, access and authority to investigate and intervene. Use a changed scenario to see whether the learning transfers.

This is not about rewarding disagreement with AI. It is about developing people who can recognise a sound recommendation, challenge a weak one and know when they need help.

The Judgement That Matters

Return to Niamh’s first assessment. In a better version of the scene, she may reach exactly the same conclusion as the AI.

The difference is that she can explain why, show what she checked and identify the uncertainty that still matters.

That ability takes practice, feedback and a workplace willing to make room for both.

The signature at the bottom of the page should belong to someone who can explain how they got there.