Chloe M.Google reviewOrder arrived damaged and nobody has replied to my email in three days.

A modern AI customer service agent can read a messy question, find the answer in your help content or order system and reply in plain language. It is good at questions with one correct answer and bad at judgement, money and distress. In most markets you are responsible for what it says, and in the EU you must now tell people it is AI. Use it for the first group, keep people on the second and test it hard before anyone sees it.
For what to automate across your whole support operation, read how to automate customer support without trapping customers. This post is about the chatbot itself.
The chatbots most people hated were decision trees: a flowchart of buttons and keywords. Match a branch and you got an answer. Miss and you got the menu again. An AI agent built on a large language model works differently.
| Decision-tree bot | AI agent | |
|---|---|---|
| Understands | Buttons and set keywords | Free text, typos, several questions in one message |
| Answers from | Scripts someone wrote | Your help content, policies and connected systems |
| Can take actions | Only what was hard-coded | Whatever you connect, such as order lookup or address change |
| Fails by | Looping back to the menu | Giving a confident, fluent wrong answer |
The last row matters most. An old bot failed visibly. A new agent can fail invisibly, with an answer that sounds right and isn't.
A question suits an AI agent when all four of these hold:
Order status, opening hours, how to reset a password, what a plan includes, how to start a return and where to send a document all pass.
Some contacts fail the test on every point.
The principle that holds almost everywhere: what your bot tells a customer counts as what your business told them, and consumer law applies the same way whether a person or software answered. Specific duties to disclose AI vary by market. This is general information, it depends on your circumstances and it changes, so check your own market with a lawyer.
The case everyone cites is Moffatt v Air Canada, decided by British Columbia's Civil Resolution Tribunal in February 2024. Air Canada's chatbot told a grieving customer he could book a full fare and claim the bereavement rate within 90 days. The airline's real policy didn't allow that. Air Canada argued it could not be held liable for information from its representatives, including a chatbot. The tribunal found the airline "did not take reasonable care to ensure its chatbot was accurate" and awarded CAD 650.88 plus interest and fees. It is one small claims decision, but the reasoning travels.
| Market | Telling people it is AI | Accuracy and liability | Reaching a human |
|---|---|---|---|
| EU | AI Act Article 50: providers must design systems that interact with people so they are informed it is AI, unless obvious, "at the latest at the time of the first interaction". Applies from 2 August 2026; the Digital Omnibus did not delay it. | General consumer law. | GDPR Article 22: where solely automated decisions with legal or similarly significant effects are allowed, people get "at least the right to obtain human intervention". |
| UK | The CMA's guidance on AI agents says if knowing it is AI "might affect people's decisions then you should tell them". | Same guidance: "you are responsible for what an AI agent does in the same way you are responsible for what an employee does", even if a supplier built it. Fines can reach 10% of worldwide turnover. | Articles 22A to 22D, in force from 5 February 2026, require human intervention and a right to contest for significant decisions made solely by automation. |
| US | No federal chatbot law that DLA Piper identified. Maine, New Jersey, Utah (on request) and a narrow California rule apply. | The FTC said in Operation AI Comply (2024) there is "no AI exemption from the laws on the books". | No general rule. For financial firms, the CFPB warned in 2023 about "doom loops" with no "offramp" to a human, and said chatbots "must comply with all applicable federal consumer financial laws". |
| Australia | No AI-specific chatbot law we could confirm. | The ACCC says information from staff must be "accurate, truthful and based on reasonable grounds" and "it makes no difference whether a business intends to mislead". False claims about a consumer's rights or remedies are covered. | No general rule we could confirm. |
| India | No dedicated chatbot disclosure rule found. | General consumer law. | Under the Consumer Protection (E-Commerce) Rules 2020, e-commerce entities need a grievance officer who acknowledges complaints within 48 hours and resolves them within one month, per Lakshmikumaran and Sridharan. A bot can't be that officer's only face. |
| Canada | No federal rule we could confirm. | Moffatt v Air Canada, above. | No general rule we could confirm. |
The EU duty in Article 50(1) sits on the provider. If you bought the bot, ask your vendor in writing how disclosure is met. Whether a refund refusal is a "significant decision" under GDPR or UK law depends on the facts; keeping a person on refunds removes most of that question.
| Situation | Who owns it | Do first | Never |
|---|---|---|---|
| Order status, hours, password reset | AI agent | Answer from live data, cite the source | Guess when data is missing |
| What a plan includes, how returns work | AI agent | Answer from current policy text | Answer from an old help article |
| Delivery complaint, warranty question | Person, AI draft | Check the draft against policy | Let the bot decide eligibility |
| Refund, chargeback, payment error | Person | Acknowledge, confirm owner and timeline | Let the bot promise or refuse |
| Legal threat, regulator, safety, data breach | Senior person | Short holding reply, escalate | Let the bot argue or admit fault |
| Bereavement, illness, hardship, fear | Trained person | Slow down, drop the script | Route to self-service |
| Bot fails to understand twice | Person | Hand over with the full transcript | Send them back to the menu |
| Customer asks for a person | Person | Hand over at once | Ask them to try the bot first |
The CMA puts it simply: "Testing is a crucial part of training AI agents", and test results should be used to improve the agent "before you deploy it". A workable process:
The pass mark is yours to set. A suggested starting point, not a benchmark: no invented policies, every escalation case handed over, and a support lead happy to sign their name under the rest.
Three models dominate. All prices below are from each vendor's own page, as of November 2026, and change often.
| Model | Example | What the page says |
|---|---|---|
| Per resolution | Intercom Fin | $0.99 per resolution, defined as "no further help is requested after Fin's last answer". Minimum 50 outcomes a month with other helpdesks. |
| Per resolution, inside seat plans | Zendesk | AI agents included in every plan, billed per automated resolution. 5 to 10 included per agent per month depending on plan, then US$2.00 each pay as you go, less if committed. Seats from US$19 per agent per month billed annually. |
| Per session | Freshdesk | First 500 Freddy AI Agent sessions included once per account, then $49 per 100 sessions. Seats from $19 per agent per month billed annually. |
| Per conversation or per action | Salesforce Agentforce | $2 per conversation, or Flex Credits at $500 per 100,000, with a standard action using 20 credits. |
What to check before you sign:
We break these down further in Intercom Fin pricing explained and Zendesk pricing explained. If a vendor beats your own team on cost per correct answer in your test set, it is the better choice for that slice of work.
Containment versus real resolution. Containment is the share of conversations that never reached a person. Real resolution is the share where the problem was solved. Only the second matters, so check a sample of contained conversations by hand each month.
Reopen and repeat contact. How many customers come back about the same issue within 7 days, on any channel. If containment rises and repeat contact rises with it, the work has moved to the customer.
Handover rate and wait. How often people ask for a person, and how long they then wait.
CSAT, split by bot and human. Survey both. Compare like with like, by contact reason, because the bot gets the easy questions.
Wrong answers found in reviews and complaints. The target is zero.
It is working when real resolution rises, repeat contact stays flat or falls and the wait for a person holds.
This example is made up to show the method. The company, people and numbers are invented.
Kestrel Home sells kitchenware online in the UK, Germany and Singapore. Its support lead, Amara Okafor, tags three months of tickets and finds order status, returns and product care cover most of the volume.
Her 250-question test set catches the first vendor trial inventing a 60-day returns window for Singapore, where Kestrel's policy is 30 days. The cause is an old help article still live there. Amara deletes it, assigns content owners per market and reruns the set clean.
Kestrel launches for order status only, with an opening line saying it is an automated assistant and how to reach a person. Refunds, damaged goods and anything mentioning a bereavement or a lawyer go straight to the team.
After six weeks, her monthly sample shows some "resolved" chats where the customer then emailed. Repeat contact within 7 days tells her which. She fixes the order-lookup gap behind them before adding returns.
Relay is a support inbox, and it has no website chat widget or customer-facing bot. Incoming messages are classified by category, urgency and sentiment, including sarcasm, so a "great service, thanks" on a one-star review isn't auto-thanked. Refunds, payments, explicit requests for a person and low-confidence cases are routed to a person. Relay drafts a reply for each message, but drafts come from category rules or a general language model and are not yet grounded in your policy documents, so they need checking. A person presses send on every reply. Reopen rate is computed from ticket history. See how Relay works or contact us.
An AI agent is a chatbot built on a language model that reads free text and can be connected to your systems to take actions. Older chatbots followed fixed decision trees. The practical difference is how they fail: decision trees loop, AI agents can give fluent wrong answers.
In the EU, Article 50 of the AI Act requires it from 2 August 2026 unless it is obvious, and several US states have their own rules. The UK CMA says to tell people if it might affect their decisions. A plain opening line is the safest default everywhere.
In Moffatt v Air Canada the airline was held liable for its chatbot's wrong answer, and the UK CMA treats an AI agent like an employee. Plan on owning every answer, and honour mistakes a customer acted on where you reasonably can.
It can explain your refund process and collect details. Keep the decision with a person, because a bot that promises or refuses a refund can create obligations or misstate the customer's rights.
Restrict it to content you control, keep that content current and test it with questions it shouldn't be able to answer. The correct reply to those is a handover. Rerun your test set after every change to content, model or prompt.
GuidesIntroducing Relay: your online reputation in one placeWhat Relay is, the gap it was built for, what it does today, what it does not do yet, and who should choose something else.29 October 2026 · 6 min read
GuidesHow to do online reputation management, step by stepFind what people say about your business, answer it within the rules, fix what caused it. A plan, a triage table and replies you can copy.9 November 2026 · 11 min read
GuidesHow to automate customer support without trapping your customersWhat to automate, what to keep with a person, what the law says about AI disclosure, and how to tell whether automation is really working.9 November 2026 · 12 min readGet a personalised 30 minute demo on your own channels and policies. We show you Relay handling your real conversations, then set it up with you.
Brands we work with


