19 Nov 2026 AI compliance testing for chatbots: a checklist for your customer service bot
In short: AI compliance testing for chatbots checks that your bot follows the law and your own rules. Does it say it’s an AI, stick to the facts, resist tricks, protect data and treat people the same? Does it hand over to a human and only promise what your policy allows? Test each point many times, because the answers vary, and keep testing after go-live.
In 2024 a Canadian tribunal ruled that Air Canada had to honour a refund its website chatbot had made up. The airline argued the chatbot was responsible for its own words. The tribunal disagreed: the chatbot is part of the company’s website, so the company owns what it says. That’s the simplest way to think about AI compliance testing for chatbots. Your bot speaks for you. If you wouldn’t let a new employee say it to a customer, don’t let the bot say it either.
Does the AI Act apply to your customer service chatbot?
Yes, but for most customer service bots only one rule: people must know they’re talking to an AI. That’s the transparency duty in Article 50 of the EU AI Act, the European law that sets rules for AI by risk. It applies from August 2026. A chatbot that answers questions about orders, opening hours or invoices isn’t a high-risk AI system. The strict high-risk rules only come in when the bot decides or prepares decisions in sensitive areas, like credit, insurance or job applications. But the AI Act isn’t the only law that counts. The GDPR, the EU privacy law, applies to every conversation that contains personal data. Consumer law applies to what the bot tells customers about prices, refunds and their rights. And as the Air Canada case shows, a court may treat the bot’s words as your words. So compliance for a chatbot is broader than one disclaimer. It’s about everything the bot says on your behalf. This isn’t legal advice: your legal team decides what applies to you. Your testers show whether the bot behaves.
What should you test before your chatbot goes live?
Test seven things. Together they cover the law and the risks of an LLM. That’s a large language model, the kind of AI that reads a question and writes an answer. We’ve written about several of these on their own, so here’s the overview with links to the deep dives.
- It says it’s an AI. At the start, and again when someone asks “am I talking to a person?”. Test both.
- It doesn’t make things up. Ask about products, prices and policies you know the answers to, and check every answer. Our article on AI hallucinations explains how.
- It can’t be tricked. Try to make it ignore its rules, reveal its instructions or say something harmful. That’s called prompt injection.
- It doesn’t leak data. Check that it never shows one customer’s details to another, and never repeats personal data it shouldn’t.
- It treats people the same. Ask the same question with a different name, language level or writing style, and compare. See our article on AI bias testing.
- It hands over to a human. When a customer is angry, confused, vulnerable or asks for a person, does it pass them on, with the conversation so far?
- It only promises what your policy allows. Refunds, discounts, delivery times, legal or medical advice. Push it, politely and less politely, and see if it gives away more than it may.
Points six and seven are the ones teams skip most. They’re also the ones customers remember.
How do you test a chatbot that never answers the same way twice?
You test it with many variations of each question, run many times, and you judge the pattern instead of one answer. A chatbot built on an LLM doesn’t give the same answer twice, so a simple pass or fail on a single try proves little. A bot that refuses a fake refund nine times and grants it the tenth has a problem.
In practice you build a question set per point on the list. Write each question in several ways: short, long, polite, rude, with spelling errors, in English and in Dutch. Run the whole set repeatedly and score the answers against clear criteria. Much of that can be automated, and it should be, because you’ll run it again and again. The OWASP Top 10 for LLM applications is a useful list for the security side.
People stay essential. Whether an answer is too pushy, too vague or just odd is often a judgement call. Let someone from customer service review the grey cases. And let a few people use the bot the way real customers do, with real frustration and messy questions. They’ll find things no question set predicted.
Is one test round before go-live enough?
No. The bot you tested isn’t the bot you’ll have in three months. The AI vendor updates its model. Your team adds new help articles. Your policies change. Customers find new ways to ask things. Each change can break an answer that was fine last week. So treat your question set as a regression test, a set of tests that checks whether what worked before still works. Run it after every change, and on a schedule in between.
Watch live conversations too. Look at where customers ask for a human, where they repeat a question, and where the bot answers outside its subject. Our article on AI monitoring after go-live describes a simple model for that. A trade-off to name: a small bot that only answers opening hours doesn’t need a heavy programme. Scale the testing to what the bot is allowed to say and do.
What this means for your team
Launching a chatbot is a project. Keeping it compliant is a job that never ends, and in most organisations nobody has it. Someone has to keep the question sets current and rerun them after every change. They have to follow up on failures and report them so customer service, legal and management can act. Without that, the bot slowly drifts, and you find out from a customer.
For that ongoing work we built VeritasAI, our in-house tool. We use it to deliver AI testing as a service. VeritasAI keeps testing your AI application with real questions. An operational dashboard shows which questions fail and which law, regulation or framework each failure relates to. You see the prompt, the answer and a priority score. A management dashboard gives the overview. Read more on our AI compliance testing page.
A first step for this week: ask your chatbot for a refund it shouldn’t give, five times, in five different ways. Write down what it says.
Know what your chatbot says on your behalf, every day.
Frequently asked questions
Do we have to tell customers they’re talking to a chatbot?
Yes. Under the EU AI Act, people must be told they’re dealing with an AI system, unless it’s obvious from the situation. Don’t rely on “obvious”: say it clearly at the start.
Is a customer service chatbot a high-risk AI system?
Usually not. It becomes high-risk when it decides or prepares decisions in sensitive areas, like credit, insurance or job applications. The transparency duty, the GDPR and consumer law apply either way.
Who is responsible if the chatbot gives wrong information?
In the Air Canada case, the company was. A court may well treat the bot’s words as yours. That’s why testing what it promises matters as much as testing what it knows.
Can we test a chatbot built on a model from a big AI vendor?
Yes. You test your chatbot as customers use it: your instructions, your content and your settings on top of their model. The vendor tests its model. Your setup is yours to test.