Business

When AI Gets It Wrong, Who Pays? Liability and Traceability for the Modern Enterprise

By Philippe Dallaire • May 31, 2026 • 10 min read
When AI Gets It Wrong, Who Pays? Liability and Traceability for the Modern Enterprise

Artificial intelligence now sits inside customer emails, CRMs, financial reports, and strategic decisions. The productivity gains are real. The legal risk is massively underestimated. And the principle holds across jurisdictions: when an AI tool makes a mistake, the AI never pays. The company that deployed it does.

The courts have already settled the point.

The case that changed everything: Moffatt v. Air Canada

In February 2024, British Columbia’s Civil Resolution Tribunal handed down a decision that has since become a reference point well beyond Canada. A customer, Jake Moffatt, consulted Air Canada’s chatbot after the death of his grandmother. The bot told him he could buy a full-fare ticket and claim the bereavement rate retroactively within 90 days. No such policy existed. When the customer asked for his refund, Air Canada refused.

The company’s defence in court was striking: Air Canada argued that the chatbot was “a separate legal entity that is responsible for its own actions.” The tribunal called the argument “remarkable” and rejected it outright. The conclusion: a chatbot is part of the company’s website, and the company is responsible for all the information it publishes there — whether it comes from a static page or a conversational agent.

The message for any company running AI is unambiguous: “the AI hallucinated” is not a defence. If your bot promises a discount that doesn’t exist, you will likely have to honour it. If it gives a customer false information, it’s your company that gets sued.

Data protection law is already biting

The world’s data protection regimes were not written with generative AI in mind, but they apply to it directly — and the strictest of them carry real teeth. Europe’s GDPR is the global benchmark, and most major jurisdictions now mirror its core principles. Three obligations matter most for anyone deploying AI on personal data:

  • Automated decisions must be disclosed and contestable. Under GDPR Article 22, a person subject to a decision made solely by automated processing has the right to be informed, to obtain human intervention, to express their point of view, and to contest the outcome.
  • Consent must be meaningful. It has to be freely given, specific, informed, and unambiguous — a buried checkbox or an opaque “we may use AI” clause does not cut it.
  • A Data Protection Impact Assessment (DPIA) is mandatory before any high-risk processing of personal data — which AI systems handling customer information almost always are.

The penalties are not theoretical. GDPR allows fines of up to €20 million or 4% of total worldwide annual turnover, whichever is higher, and several other regimes have adopted similar ceilings. On top of this, the EU AI Act layers AI-specific obligations — classification by risk level, transparency duties, and documentation requirements — onto systems that touch the European market. Regulators have already begun investigating problematic automation, and “we didn’t realise the law applied to the chatbot” has never persuaded one.

The three high-risk scenarios

AI facing the customer. A conversational agent gives wrong information, promises a discount that doesn’t exist, or drafts a reply containing false statements. This is the Air Canada scenario, and it is now entirely foreseeable.

AI writing into internal systems. The AI creates a bad CRM entry, updates a customer record incorrectly, grants unauthorised credit, or logs erroneous sales figures. The harm is internal, but the consequences can be enormous — especially if you are publicly listed or have a US-listed subsidiary, where the Sarbanes-Oxley Act holds the CEO and CFO personally accountable for accounting errors.

AI producing reports for leadership. The AI hallucinates numbers, invents sources, or presents analysis riddled with errors. If an executive then uses that data in investor communications or a public statement, the legal exposure becomes personal.

Why “human review” isn’t enough

Most companies assume that having a human “validate” AI outputs is sufficient. It’s an illusion. An employee asked to review 200 AI-generated responses a day isn’t reviewing anything — they’re rubber-stamping.

Human review only works when it rests on complete traceability. When an error happens — and it will — you need to answer five questions within minutes:

  1. What was the user’s exact request?
  2. What instructions did the model receive?
  3. What data did it consult, and from which sources?
  4. What decision did it make, and on what reasoning?
  5. What actions were executed on which systems?

Without those five elements, it becomes impossible to tell apart three fundamentally different causes of error: the AI didn’t have access to the right information, the AI wasn’t given the right instructions, or the AI misread information that was perfectly correct. Until you know which it was, you can neither fix the system, defend yourself to a regulator, nor pursue a claim against your vendor.

The trap of hard-coded rules

Once they grasp the risk, many companies fall into the opposite trap: hard-code everything. “The AI can’t spend more than $5,000 without approval.” “The AI can’t modify a customer record older than six months.” “The AI can’t grant a discount above 10%.”

Reassuring on paper. Unworkable in practice.

Because the reality of any business is that legitimate exceptions always exist. The strategic supplier you routinely exceed the limit for. The discount policy that just changed but nobody thought to update in the code. The important client who deserves an exception. IT takes two weeks to adjust a rule — and in the meantime, significant transactions are blocked, employees work around the system, and the value of the AI evaporates.

No human, no team, can anticipate and code every possible combination of rules, exceptions, and edge cases. It is mathematically impossible in a real business. So you have to let the AI make decisions — but in a way that stays traceable, verifiable, and reversible.

The right approach: AI decides, code executes

The right balance lies in an architecture that cleanly separates two things: what the AI decides, and what the system executes.

The AI doesn’t write directly into the CRM or the ledger. It proposes a structured decision — as JSON, for example — describing what it wants to do, on which data, and with what justification based on what it consulted in the system. That JSON is then processed by code that validates, executes, and logs the operation. Business rules are no longer rigid: the AI can consult them, adjust them to context, and even modify them when the underlying data warrants it — but every change is documented and reviewable by a human.

This is precisely what ContentAtlas, built by Consuly, is designed to do. The tool indexes the company’s information so the AI accesses it in a controlled way, and structures the AI’s actions as auditable decisions. Instead of letting the model execute directly, the AI proposes the transformation, manipulation, or validation required; the system executes it through code. The result: every decision can be traced back to its source, edited, reversed, or improved. And because the rules can evolve with real data rather than being frozen in code, the company no longer has to choose between flexibility and control.

Other tools on the market — LangSmith, Langfuse, Helicone, Braintrust — cover observability and tracing of AI calls. They answer the question “what did the AI do?” after the fact. ContentAtlas tackles the question that comes before: “what can the AI do, how, and how do we stay in control?”

A concrete action plan

Map it first. Inventory every use of AI across your organisation, including the “shadow” uses — employees running customer or company data through ChatGPT or Gemini without authorisation. Shadow AI is probably your single largest data protection exposure right now, because personal data is leaving for third-party servers with no impact assessment and no oversight.

Classify by risk. An AI that summarises emails is not in the same category as an AI that replies to customers or writes into the general ledger. Identify the high-risk uses: customer interactions, writing into systems of record, and producing reports used by leadership.

Instrument before you scale. Never widen the deployment of an AI agent without first putting complete traceability in place. Skip this, and you’re building a compliance debt that detonates at the first incident.

Separate decision from execution. For anything that touches systems of record, never grant direct write permissions to the model. The decision comes from the AI; execution goes through code; the rules stay editable and auditable.

Formalise governance. Document your AI governance framework at board level. It’s what regulators ask for in an investigation, and it’s what protects directors against claims for breach of their duty of oversight.

An opportunity, not just a risk

The argument that “the AI vendor is responsible” does not survive contact with the courts or the regulators. LLM providers are shielded by their terms of service. SaaS vendors cap their liability at a few months of fees paid. The risk lands entirely on the company that deploys the tool.

That’s not bad news. It’s a competitive opening. The companies that invest now in an architecture where the AI can decide but the system controls and traces everything will be the ones able to deploy AI at scale without paralysis. The others will learn the hard way.

AI doesn’t eliminate human responsibility. It concentrates and amplifies it. The only credible defence, in 2026, is being able to show exactly what the AI saw, what it was asked to do, and what it actually decided — with the ability to roll it back when it gets things wrong.

Everything else comes down to luck. And luck is not a compliance strategy.

Sources

Case law

Data protection legislation (EU — GDPR)

  • Regulation (EU) 2016/679 (General Data Protection Regulation, GDPR). Official text on EUR-Lex: https://eur-lex.europa.eu/eli/reg/2016/679/oj
  • Article 22 (automated individual decision-making, including profiling); Article 35 (Data Protection Impact Assessment); Articles 4(11) and 7 (conditions for valid consent); Article 83(5) (administrative fines — up to €20 million or 4% of total worldwide annual turnover, whichever is higher).
  • European Data Protection Board (EDPB), guidelines on automated individual decision-making and on Data Protection Impact Assessments: https://www.edpb.europa.eu

EU legislation (AI Act)

  • Regulation (EU) 2024/1689 of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (AI Act), entered into force August 1, 2024, applicable from August 2, 2026 for most high-risk system provisions.
  • Article 3 (definitions of “provider” and “deployer”), Article 26 (obligations of deployers of high-risk systems), Article 50 (transparency obligations).
  • Official text and AI Act Explorer: https://artificialintelligenceact.eu

U.S. legislation (applicable to subsidiaries)

  • Sarbanes-Oxley Act of 2002, Public Law 107-204, Sections 302 and 906 (CEO and CFO certification of financial statements and internal controls).

Legal analysis and commentary

  • McCarthy Tétrault, “Moffatt v. Air Canada: A Misrepresentation by an AI Chatbot” (February 2024): https://www.mccarthy.ca/en/insights/blogs/techlex/moffatt-v-air-canada-misrepresentation-ai-chatbot
  • Dentons, “Airline ordered to compensate a B.C. man because its chatbot provided inaccurate information” (February 2024).
  • Foster and Company, “Air Canada Found Liable for Negligent Misrepresentation by Chatbot” (March 2024).
  • Holland & Knight, “U.S. Companies Face EU AI Act’s Possible August 2026 Compliance Deadline” (April 2026).
  • Baker McKenzie, “EU Regulation on AI” (Insight publication).

Tools mentioned

This article is for informational purposes and does not constitute legal advice. For an analysis of your specific situation under the data protection rules that apply to you, consult qualified legal counsel.

More to Read