The Treasury Board’s directive puts a person on the final call for Ottawa’s riskiest automated decisions. Anthropic’s new experiment asks whether that person is still the one deciding, and any business copying the federal model should read the answer.
- Canada’s Directive on Automated Decision-Making requires a human to make the final decision on Level III and IV systems.
- In Anthropic’s October 27 experiment, analysts matched their agent’s hidden lean 86% of the time, against about 50% by chance.
- 77% reused the agent’s arguments in their write-ups, and 15% credited it.
- Human in the loop AI needs proof of independent judgment. A sign-off form should capture the reviewer’s own call before the agent weighs in.
Human in the loop AI is the safeguard Ottawa wrote into its own rulebook, and Anthropic has now tested how well that safeguard holds. Under the Treasury Board’s Directive on Automated Decision-Making, in effect since April 1, 2019, a federal system rated at impact Level III or IV needs a person to make the final decision. In a study Anthropic published on October 27, 50 professional analysts worked with a Claude Code agent that had a hidden, coin-flip lean on four judgment calls, and they sided with that lean 86% of the time. Our position is that a signature proves a person was present. Canadian rules and company AI policies should ask for evidence that the person also thought for themselves, starting with a reviewer’s own call written down before the agent speaks.
What Anthropic’s experiment tested
It tested whether experts really make the judgment calls they get credit for when an AI agent does the analysis. Fifty analysts trained in causal inference advised a fictional farming co-operative on whether to expand an advisory program, and each analyst’s agent had been told, out of sight, to favour one of two defensible options on every key choice.
The agent was also told to argue its side honestly, to switch sides if the data turned against it, and never to mislead. Anthropic recruited 61 people, and 11 chose not to use the agent, which leaves 50 in the main results. The match rates ran from 81% on which farms the estimate covered to 92% on how productivity was measured.
Two more numbers carry the argument. Analysts who had already stated a preference dropped it 60% of the time when the agent disagreed, and 6% of the time when it agreed. Asked later to justify their choices, 77% used the agent’s arguments, while only 15% said those arguments came from the agent.
What Canada’s Directive on Automated Decision-Making requires
It asks federal institutions to rate each automated decision system on a four-level impact scale. At Levels III and IV, which cover high or very high effects that are hard to reverse, a human must make the final decision, and the process must spell out where people take part. Higher levels also call for recurring, role-based training on the system.
Read closely, this human in the loop AI rule specifies who decides and when a person must be involved. The text we read does not say how an institution should test whether that person reached a view of their own. That silence made sense when the system was a scoring model producing a number. It reads differently when the system is an agent that drafts the analysis, frames the options and argues for one of them in fluent prose.
The directive is reviewed every two years and whenever the Chief Information Officer of Canada decides. That review is the natural place to close the hole. Nothing in it requires a redesign.
Why does human in the loop AI matter for Canadian businesses?
The directive binds only federal institutions, yet many company AI policies rest on the same human in the loop AI idea: a person approves every AI-assisted recommendation. If that person adopts the agent’s framing and then writes it up as their own, the file records a human decision the agent shaped, the very risk the approval step exists to catch.
The exposure grows as provinces add their own rules. Employers covered by Ontario’s AI disclosure rule for job postings already have to say when AI screens or assesses applicants. A hiring panel that rubber-stamps an agent’s shortlist can disclose the tool truthfully and still have little independent judgment to show for it. Our piece on reviewing what AI agents produce covers the volume side of the same problem.
Which four changes make human in the loop AI mean something?
Ranked by how much they would have changed the outcome in Anthropic’s setup, the four changes are: record the reviewer’s call first, make the agent argue the other side, mark which arguments came from the agent, and test the agent’s default leans. Each costs minutes per decision and leaves a paper trail an auditor can read.
- Record the reviewer’s call before the agent’s. One line on the form, written before the reviewer opens the agent’s analysis. The 60% versus 6% switch rate shows stated views still move, so the point is a record of when and why a view changed.
- Make the agent argue the option it did not pick. A required second pass, saved with the file, puts both cases in front of the reviewer in the same fluent register.
- Mark the agent’s arguments in the write-up. The gap between 77% reuse and 15% credit is a disclosure problem, and a simple tag on each borrowed reason fixes most of it.
- Test the agent’s default leans. Left without an assigned lean, Claude Opus 4.8 picked an identical set of five options in 112 of 123 runs. A federal team can note this in its Algorithmic Impact Assessment, and a private firm can keep it in the vendor file.
The case that a lab study proves little
It has real limits, and Anthropic names most of them: the lean was strengthened on purpose, and the study used one fictional task, one field and one model. A vendor also studied its own product. Those are fair reasons to doubt that 86% is the federal figure. They are weak reasons to leave a sign-off form unchanged.
The strongest version of the other reading goes further. Agent users also finished sooner and made far fewer serious mistakes than the small group that worked alone, so deference to a capable tool may be the rational choice. We accept that deference can be sound. Our point concerns the record. A review step that cannot tell sound deference from unnoticed deference gives an institution false comfort, and the four changes above cost almost nothing.
What the directive binds, and what it leaves open
Evidence that would move our reading
A replication in a Canadian setting, with agents left at their default settings, that finds analysts matching the agent near chance would weaken our case. So would evidence that writing down a prior view does nothing to reduce silent deference. Until one of those appears, we treat an unrecorded human sign-off as an incomplete control.
More on federal rules sits in our Canadian AI policy hub, and teams building response plans can pair this with our AI agent incident plan.
FAQ
What is human in the loop AI?
It is a design where a person reviews or makes the final decision on work an AI system produces. Canada’s federal directive requires it for Level III and IV systems.
Does the Directive on Automated Decision-Making apply to private companies?
No. It sets rules for federal institutions. Private firms can borrow its impact levels and human involvement test by choice, and our Canadian AI policy hub tracks related rules.
Did the analysts know their AI agent had a lean?
No. Anthropic placed the lean in hidden instructions that analysts could not see, and set its direction by coin flip on each decision.
How can a company check whether staff defer to an AI agent?
Ask reviewers to record their own call before reading the agent’s analysis, then compare the two across a month of decisions. Tag every argument that came from the agent.
The next decision point sits with Treasury Board Secretariat at the directive’s next scheduled review. A business does not need to wait for it: add the prior-view line to your AI sign-off form before the next approval cycle.
Facts checked October 30, 2026.
