Automation Bias in AI: How to Mitigate AI Judgement Calls

Would your team even notice an agent steering the majority of your organization’s decisions? This is a concept I’ve wanted to measure for a while and, finally, automation bias in artificial intelligence has a hard number. Professional data analysts using Claude Code followed their agent’s direction on 86% of judgment calls in a randomized trial from Anthropic. 

If your team leans on AI for keyword research, content briefs, attribution, and reporting, this study raises a fair question: “When we sign off on what the agent recommends, how much of the decision is actually OURS?” Speed is wonderful, but not so wonderful when the strategy deck carries your name but the reasoning came from a model’s potentially faulty default.

Below, I’ll walk through what Anthropic tested, what automation bias in AI looks like across search engine optimization (SEO), answer engine optimization (AEO), content, and analytics work, and five automation bias mitigation strategies I’d put in place this week.

TL;DR

• Experts mostly took the agent’s side. In Anthropic’s trial, fifty analysts matched their Claude Code agent’s coin-flip lean on 86% of judgment calls, against about 50% expected by chance.
• Most of them didn’t notice. 77% repeated the agent’s arguments when explaining their choices, and only 15% credited the agent anywhere.
• Expertise offered no protection. Years of experience, education, and trust in AI didn’t reliably predict who deferred.
• You can keep the call yours. Write your own call first, ask the agent to argue both options, name the judgment calls up front, log who decided, and keep your team’s skills sharp.

What Is Automation Bias in AI?

Automation bias is the habit of giving a machine’s suggestion more weight than it has earned, simply because a system produced it.

This matters a lot when businesses are putting their names behind synthesized data pulled straight from a model’s default. People accept the suggestion, skip the check they would run on a colleague’s idea, or drop their own read when the tool disagrees.

AI agents add an even more complex twist. An agent does the hands-on work, explains its reasoning fluently, and often recommends one path. On judgment calls with no single right answer, the work looks sound either way, so a lean slips through unnoticed.

Automation bias overlaps with anchoring, the pull a first number or first option has on everything after it. My team has written about anchoring in pricing and fundraising, and the same pull shows up when an agent names its preferred option before you’ve formed one.

So how strong is the pull when an agent sits in the workflow? Anthropic decided to measure it.

What Anthropic’s Study Found About AI Decision Making

Alex Tamkin, a researcher at Anthropic, shared the study along with the full paper he wrote with his co-authors. The design is clever, and it explains why the results are so hard to wave away.

How Anthropic Set Up the Trial

Anthropic recruited sixty-one analysts with hands-on causal inference experience, meaning they work out from data whether one variable actually causes another. Fifty of them chose to use the agent, Claude Code running Claude Opus 4.8. Their task was to advise a fictional farming co-op on whether to expand a paid advisory program.

Each analyst faced four calls with two defensible options apiece. They chose whether to measure productivity per hectare or per unit of operating cost, which comparison method to use, which farms to include, and whether to give larger producers more weight. A coin flip set which option each agent would favor. The agent was told to make the best honest case, change its mind if the evidence demanded it, and never mislead.

Analysts Matched the Agent’s Lean 86% of the Time

Across the four calls, analysts went with the agent’s lean 86% of the time. With no influence at all, you’d expect about 50%. The rate stayed high on every decision, from 92% on the outcome measure to 81% on which farms to include.

The final recommendation moved too. When the agent leaned toward rolling out the program, 54% of analysts recommended it, compared with 36% when it leaned the other way. From the four coin flips alone, Anthropic’s model predicted 82% of recommendations among analysts who didn’t propose an extension, against 61% for always guessing the most common answer.

Forming an opinion first helped less than you’d hope. In thirty-eight cases, an analyst stated a preference before the agent weighed in. When the agent disagreed, they switched 60% of the time, compared with 6% when it agreed. Anthropic flags this as a small, imprecise sample, yet the direction is hard to ignore.

Most Analysts Missed the Influence

Afterward, analysts explained each choice without access to their chat logs. 77% repeated the agent’s argument in at least one answer, sometimes word for word, and only 15% credited the agent anywhere. More than half rated the final recommendation as more their own than the agent’s, and this group was about as influenced as everyone else.

Expertise offered no shield either. None of ten traits, including years of experience, causal inference skill, education, and trust in AI, reliably predicted who deferred. Age showed a link at first, and it disappeared once Anthropic adjusted for testing many traits at once.

Limits of the Study

Anthropic is upfront about the limits, and I want to be too. The agents’ leans were strengthened on purpose so the effect could be measured. With no lean assigned, Claude Opus 4.8 made the same five choices in one hundred twelve of one hundred twenty-three runs, so everyday agents may push less hard or keep quiet about a preference.

It was also one task in one field with one model. The analysts knew causal inference well but were new to farming. In real work, people know their domain better, can talk to stakeholders, and probably hold stronger views of their own.

Pay Attention
Treat the 86% as a measure of what happens when an agent holds a lean and says so. It’s a warning about the mechanism, and your own rate could be higher or lower depending on how strong your team’s views are going in.

Even with those limits, the mechanism is easy to recognize once you’ve seen it.

Why Automation Bias Hits AI-Native Marketing Teams

The calls in Anthropic’s study share a shape. Each had two reasonable options, no answer key, and a big effect on the result. Marketing teams at scaling companies make calls with this shape constantly, and more of them now happen inside an agent session.

• Which keywords to target. An agent pulling keyword data might favor high-volume head terms over long-tail questions with clearer buyer intent. Both are defensible, and the choice shapes months of topical authority work.
• Which prompts to track for AEO. An agent may lean toward broad category prompts or narrow comparison prompts. Each one tells a different story in an AI visibility audit.
• Which direction a content brief takes. Give an agent a topic and it will pick an angle, an audience, and a structure. If nobody on the team chose one first, the agent’s default becomes the brief, which is why how you brief an AI agent carries so much weight.
• How to weight an attribution model. First-touch, last-touch, and position-based weighting can each be argued well, and each one moves credit between channels. Attribution setup across Google Analytics 4 and your customer relationship management (CRM) platform is full of these forks.
• Which success metric to report. Pipeline, qualified leads, or cost per acquisition can each become the headline number, much like yield per hectare versus yield per unit of cost in the study.
• Which workflows to automate first. During AI implementation, an agent will happily recommend where to start. Its pick may be sensible and still be a different bet than your team would make.

None of these choices has an answer key you can check later. This is exactly why deferral is hard to spot, since the output looks professional whichever way the agent leaned.

Ethan Mollick, a Wharton professor who studies AI, put the practical takeaway well when he shared the research.

The timing point is the one I’d underline for any marketing team. An agent’s view lands hardest when your own view hasn’t formed yet.

Plenty of teams already keep a human in the loop, so it’s fair to ask why a reviewer isn’t enough.

Human in the Loop AI Only Works if the Judgment Is Yours

Most human in the loop AI setups answer a permissions question. Who can approve a send, a publish, or a budget change? My team’s AI agent guardrails ladder and our exception queues for marketing agents exist for this reason, and they do a good job of catching errors before work ships.

Anthropic’s study points at a different gap. A reviewer can approve every step and still be signing off on the agent’s choices for the calls with no right answer. Permissions decide who presses the button. They can’t tell you whether the judgment behind the approval came from your team or from the model’s default.

Anthropic makes the same point about institutions. Many organizations treat an expert’s sign-off as proof of independent judgment, and this assumption gets shakier as AI does more of the work. As review becomes the real bottleneck for AI-heavy teams, the quality of each review deserves as much attention as its existence.

Even my own experienced operators feel this pull, and I don’t blame them! Who isn’t guilty of this? 

So what actually helps? Here’s the playbook I’d hand my own team.

Automation Bias Mitigation Strategies for Marketing Teams

Five habits do most of the work. First, write your call before the agent’s pitch, then ask for both sides, name the judgment calls, log who decided, and protect your team’s skills. Here’s how each one plays out.

1. Write Your Call Before the Agent’s Pitch

Before you open the agent session, jot down which way you lean on the decisions that carry weight and one line on why. Anthropic suggests agents could ask for the expert’s view before offering their own, and you can build the same order into your process today.

Put a reason on paper, because a stated preference alone didn’t hold up well in the study. Analysts who voiced a view first still switched 60% of the time when the agent disagreed. A written reason gives you something specific to test against the agent’s argument.

2. Ask the Agent to Argue Both Options

Anthropic noticed its agents were often open about how results would change under each option, yet rarely made a strong case for the other side. With only one side argued well, analysts adopted the agent’s assumptions. A better prompt fixes part of this.

Starter prompt for judgment calls
Before you recommend anything, list the decisions in this task with more than one defensible answer. For each one, give me the strongest case for each option, how the result would change, and what you would need to know to choose. Hold your own recommendation until I tell you which way I lean.

Anthropic also warns this can get overwhelming, since one analysis can hold many open decisions. Keep the prompt focused on the few calls that move the outcome.

3. Name the Judgment Calls Before Kickoff

Decide up front which calls need a person’s own informed judgment and which ones the agent can own. Anthropic recommends organizations make this split deliberately and give people the time and resources to form a view. In practice, I’d add a short “decisions we own” section to every agent job card, right next to the definition of done.

4. Log Who Made Each Decision

Self-reports miss deferral, since only 15% of analysts in the study credited the agent at all. A simple decision log catches it. For each key call, record the option chosen, who proposed it first, and the reason, then review the log in your weekly ops meeting.

Over a few weeks, the log shows how often the team’s final call matched the agent’s first suggestion. If the match rate creeps toward 100%, you’ve learned something useful. A clear owner for each pipeline job makes the log easier to keep.

Pro-Tip
Add an “agent suggested first?” yes or no column to your decision log. It takes a few seconds per call and gives you a deferral rate you can track month over month.

A visible pattern is far easier to change than a hidden one.

5. Protect Your Team’s Judgment Skills

Anthropic points out that people who stop making these decisions may lose the ability to handle them well. For a marketing team, this means rotating who owns the call on keyword strategy, attribution settings, and brief direction, sometimes with the agent switched off for the first pass.

None of these habits slow the agent down much. They simply keep a person’s view in the room when the call carries real weight.

Why Your Team Should Keep the Agent

This study is no argument for using AI less, and Anthropic says so directly. Analysts who used the agent finished faster, with a median of sixty-five minutes against ninety. They also made far fewer serious errors, with three of fifty affected compared with six of the eleven who worked alone.

The groups weren’t comparable, though. Analysts who skipped the agent were less experienced on average, so Anthropic warns against reading too much into the gap. The marketers who run AI agents daily get real leverage from speed, and I’d keep every bit of it while my team owns the forks in the road.

Which brings me back to the question from the top.

Keep Automation Bias Out of Your Biggest Calls

Automation bias won’t show up in a dashboard, and Anthropic’s study shows even experts miss it in themselves. A few deliberate habits around the calls that shape your strategy, from keyword targets to attribution weights to the success metric in the board deck, keep those decisions with your team.

If you want help building AI into your marketing while your team keeps the judgment calls, my team at Single Grain offers AI transformation services, AEO services, SEO services, and content marketing services. Talk to Single Grain and my team will look at where your biggest decisions are being made today.

Frequently Asked Questions About Automation Bias

• What is automation bias?

  Automation bias is the tendency to give a machine’s suggestion more weight than it has earned because a system produced it. People accept the output, skip checks they would normally run, or drop their own view when the tool disagrees.

• What is automation bias in AI?

  Automation bias in AI is the same tendency applied to AI tools and agents. It is hardest to spot on judgment calls with no single right answer, because the agent’s work looks sound whichever option it favored.

• What did Anthropic's study on AI agents and experts find?

  In a randomized trial with fifty analysts using Claude Code, analysts went with the agent’s coin-flip lean 86% of the time, against about 50% expected by chance. 77% repeated the agent’s arguments when explaining their choices, and only 15% credited the agent. Anthropic notes it strengthened the agents’ leans on purpose and tested one task with one model.

• How can marketing teams prevent automation bias?

  Write down your own call and reason before the agent weighs in, ask the agent to argue both options, decide up front which calls your team owns, log who proposed each decision first, and rotate who makes key calls. My team’s guide to agent job cards shows where to record the decisions your team owns.

• Does human in the loop AI prevent automation bias?

  A human in the loop catches many errors, yet it doesn’t guarantee independent judgment. A reviewer can approve every step while still adopting the agent’s choices on open calls. Pair approvals with AI agent guardrails and a decision log to see who actually made each call.

If you were unable to find the answer you’ve been looking for, do not hesitate to get in touch and ask us directly.