---
title: "Why AI answers sound right and still get the facts wrong"
canonical: https://replypen.com/blog/why-ai-answers-are-fluent-but-wrong
description: "Why AI-written customer email can be fluent and still wrong, the mistakes that show up most, how grounding reduces them, and which facts to check before you press send."
author: "ReplyPen team"
published: 2026-09-30
operator: ProBackup BV (Hasselt, Belgium)
---

# Why AI answers sound right and still get the facts wrong

AI writing tools produce the most likely next words, not facts looked up in your systems. When the answer is not in what the tool was given to read, it still writes a fluent sentence, often a plausible policy, price or date that sounds like yours. Fluency is no sign of accuracy. Ground the draft in your own sources, and check the facts that commit you before you send.

## Why does AI sound confident even when it is wrong?

A language model writes by predicting what text comes next, based on patterns in a very large amount of writing. Confident, polite, well-structured email is a common pattern, so that is what comes out, whether or not the content is right.

The US standards body NIST describes this risk as "confabulation": "the production of confidently stated but erroneous or false content". It adds that confabulations "are a natural result of the way generative models are designed", because the models "generate outputs that approximate the statistical distribution of their training data" ([NIST AI 600-1, July 2024](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf)).

For a business, the practical point is simple. The model does not know your refund window, your prices or this customer's order unless the tool gives it that information to read. Without it, the model fills the gap with what is typical, and typical is not the same as yours.

## Which mistakes show up most in customer email?

| Mistake | Example | Why it happens |
|---|---|---|
| Invented policy | "You can return it within 30 days" when your window is 14 | The typical policy fills a gap |
| Out-of-date fact | Last year's delivery time, copied from an old reply | The source was old, and nothing said so |
| Wrong customer details | Quoting a plan or order the customer does not have | The tool mixed up records or guessed |
| Over-promising | "We'll send a replacement today" when that needs approval | Helpful-sounding text is the likely text |
| Answering a different question | A reply about delivery to someone asking about returns | The email was long or mixed several topics |
| Invented steps | A menu path or button that does not exist in your product | Plausible instructions from similar products |

The last row matters most for software companies. Instructions for "Settings, Billing, Change plan" read just as well when your product puts it somewhere else.

## Does it matter who wrote a wrong answer?

To the customer, and possibly to a court, no. In February 2024, a British Columbia tribunal held Air Canada responsible after its website chatbot told a customer he could claim a bereavement fare discount after travelling, which the airline's own policy page said was not possible. The tribunal rejected the argument that the chatbot was responsible for its own statements ([Moffatt v. Air Canada, 2024 BCCRT 149](https://www.canlii.org/en/bc/bccrt/doc/2024/2024bccrt149/2024bccrt149.html); summary by [McCarthy Tétrault](https://www.mccarthy.ca/en/insights/blogs/techlex/moffatt-v-air-canada-misrepresentation-ai-chatbot), both checked 30 September 2026).

That case involved a chatbot answering on its own, but the principle carries over to email: whatever is sent from your address is your answer.

## How does grounding reduce wrong answers?

Grounding means the tool reads your own sources, such as help articles, website pages, past replies and account data, before writing, and bases the reply on them. The model then has your actual refund window in front of it instead of guessing a typical one.

Grounding reduces the problem; it does not remove it. The tool can still pick the wrong article, read an outdated one, or combine two sources badly. So the second half of the fix is being able to see which sources a reply used. If a draft about refunds lists your refund article, checking takes seconds. If it lists nothing, you have to verify every fact from memory, which is the work the tool was meant to save.

## Which facts should you always check before sending?

Read the draft for style once, then read it again for these alone:

- **Numbers**: prices, amounts, percentages, quantities
- **Dates and durations**: deadlines, delivery times, refund windows, "within 2 business days"
- **Promises**: anything you or a colleague now has to do
- **Customer-specific details**: order numbers, plan names, what they bought
- **Instructions**: steps, menu names, links
- **Exceptions**: "unless", "except", "only if", which models tend to drop

If a fact in the draft is not in your sources and you do not know it to be true, take it out or check it. Leaving it in because it sounds right is how wrong answers get sent.

## How can you spot a guessed answer?

Some signs that a draft is filling a gap rather than answering from your sources:

- It is very specific about something your help pages never mention
- The policy sounds generic, like something any company might say
- It answers confidently when the question was unusual
- It is longer than your usual reply, with extra reassurance
- The tool cannot show where the answer came from

## What should happen when the tool does not know?

The right behaviour is the same as for a new colleague: say what is known, and do not make up the rest. A good draft for a question your sources do not cover reads more like a holding reply: "I'm checking this with our warehouse and will get back to you by tomorrow afternoon."

When you fill the gap yourself, add the answer to your help center or facts page, so the next draft has something to read.

## How does ReplyPen deal with this?

ReplyPen writes each Draft from your help pages, website and past replies, and places it in your own Gmail or Outlook thread. Under every Draft, a [Check note](https://replypen.com/product/inbox#what-is-in-a-check-note) lists what it looked at before writing: which help articles, which website pages, which past emails. If those sources do not cover the question, you can see that before you send. When you correct a Draft, ReplyPen notes the change in its Knowledge log and uses it for similar emails. Today every Draft is sent by you.

For the full picture of how these tools work, see [What is an AI email reply tool and how does one work?](https://replypen.com/blog/what-is-an-ai-email-reply-tool.md). For when a draft could safely be sent without a person, see [Draft or auto-send: when should AI send customer email itself?](https://replypen.com/blog/draft-vs-auto-send.md).