Journal

Our chatbot gave customers wrong answers. How do we fix it?

Answer only from your own documents with citations, have agents approve drafts, and route edge cases to people. That is AI customer service done properly.

nour k.growth marketing5 min read

Do it properly by changing what the AI is allowed to do. It should answer only from your own approved documents, show which document each answer came from, draft replies that an agent approves before sending, and route refunds, complaints and edge cases to people. Most wrong answers come from messy knowledge and missing rules, not from the model.

#Key takeaways

  • Chatbots give wrong answers when the documentation is scattered, outdated or contradicts itself.
  • The dangerous wrong answers are invented policies: refunds, exceptions and promises your company never made.
  • Retrieval from your own documents, with a citation on every answer, makes errors visible and fixable.
  • Draft-and-approve keeps a person between the AI and the customer while trust is built.
  • Your company is responsible for what its chatbot says, so the rules have to be yours.

#Why did our chatbot give customers wrong answers?

Usually for three reasons. The knowledge was messy: answers sat in old pdfs, shared drives and the heads of senior agents, and some contradicted each other. Edge cases were never written down, so the bot guessed. And nothing stopped it from sounding confident when it had no source at all.

An invented policy has real consequences. Skift, february 2024 reported that a Canadian tribunal ordered Air Canada to honour a bereavement refund its chatbot had described wrongly, and rejected the argument that the chatbot was responsible for its own words. The lesson for any support lead: what the bot says, your company said.

#What does AI customer service done properly look like?

  • one cleaned knowledge source: the answers your team stands behind, with an owner who keeps them current.
  • retrieval with citations: the AI finds the relevant passage in your documents and every answer shows where it came from.
  • no source, no answer: when nothing in your documents covers the question, the case goes to a person.
  • draft-and-approve: the AI drafts, an agent checks the draft and the citation, then sends.
  • hard limits: refunds, compensation and policy exceptions are never decided by the AI.

Stennir builds this into the help desk your team already uses, such as Zendesk or HubSpot, so agents do not switch tools. Every draft, edit and approval is logged, which shows where the knowledge is thin and which answers need fixing.

#Can AI reduce call centre handle time without new wrong answers?

Yes, if it helps agents rather than replacing them in front of customers. The time goes into searching for the right answer, reading the history and writing the reply. An AI that summarizes the case, pulls the cited passage and drafts a response lets the agent spend their time checking instead of digging.

Stennir has written about each piece: ticket triage that routes and tags, drafting replies agents edit and send and a support knowledge base agents stop digging through.

We are

Stennir builds AI support workflows that answer from your own documents with citations, keep an agent approving replies, and run inside your existing help desk.

We aren't

Stennir is not another chatbot vendor, and it does not put an AI in front of customers with permission to decide refunds or policy.

#What are the options for AI customer support in Arabic, beyond a chatbot vendor?

There are roughly three. A packaged chatbot is fast to switch on but answers from whatever it is given and is hard to control. Agent assist keeps people in front of customers and uses AI for search, summaries and drafts. A customer-facing agent with strict handover rules suits narrow, repeated questions. Be wary of relabelled products: Gartner, june 2025 estimated that only about 130 of thousands of agentic AI vendors are real.

For Arabic, test each option on your own past tickets, including Gulf dialect and mixed Arabic and English, before deciding.

#What should a support lead do next?

Collect twenty tickets where the old chatbot got it wrong and find the source document each answer should have used. If the source is missing or contradictory, that is the first fix. Then talk to Stennir about AI implementation and see AI for customer support. You can take the free readiness assessment or book a 30-minute call.

#Common questions

Do we need to clean all our documentation first?

No. Start with the documents behind the most common ticket types, name an owner for them and fix contradictions as you find them. The citation on every draft shows which documents agents rely on most, so the clean-up follows real use instead of becoming a separate project that never ends.

Does it work in Arabic?

Yes. The AI can search documents written in English and answer in Arabic, or the reverse, and still cite the source. Stennir tests on your own past tickets, including Gulf dialect and messages that mix both languages, and agents approve every draft until the results justify loosening that.

What stays with us?

The help desk, the knowledge source, the logs of every draft and approval, and the off switch. Your team owns the rules about what the AI may and may not answer. Stennir builds and monitors the workflow, but the accounts and data are in your name.

What if the AI still gets something wrong?

An agent sees the draft and its citation before anything is sent, so most errors stop there. Each correction is logged, which points to the document that needs fixing or the rule that needs adding. For refunds and policy exceptions the AI never decides; it routes the case to a person.

Tell us which job takes the most time.

30 minutes. No preparation needed. We will tell you honestly whether AI can help, and where to start.