---
title: "The \"sent unchanged\" rate: an honest measure of AI reply quality"
canonical: https://replypen.com/blog/sent-unchanged-metric
description: "How to measure whether AI-drafted customer replies are good enough: count the drafts you send without edits, read why you edit the rest, and know when the number misleads."
author: "ReplyPen team"
published: 2026-09-30
operator: ProBackup BV (Hasselt, Belgium)
---

# The "sent unchanged" rate: an honest measure of AI reply quality

The sent unchanged rate is the share of AI-written drafts you send to customers without editing a word. It is the most honest single measure of whether drafts are good enough, because you count it yourself, from your own replies. Track it weekly, per kind of email, and pair it with the reasons behind every edit: a changed fact matters far more than a changed greeting.

## What exactly counts as "sent unchanged"?

A draft counts as sent unchanged when the reply the customer received is the draft as written. Decide up front how strict you are, and stay consistent:

| Change | Count as unchanged? | Why |
|---|---|---|
| None | Yes | The clear case |
| Your signature added automatically | Yes | Not a judgement about the draft |
| Whitespace or a line break | Yes | Formatting, not content |
| A word swapped for taste | No, but log it as "wording" | Shows tone drift if it repeats |
| A sentence removed or added | No | The draft said too much or too little |
| A fact changed | No, and log it as "fact" | The draft was wrong where it matters |

Everything that is not unchanged is "edited". The edits are where the useful information is.

## How do you measure it without special software?

With a spreadsheet and ten minutes a week.

1. **Take a sample**: every draft from one day a week, or the first 20 drafts of the week.
2. **Compare each draft with what you sent.** Most mail clients keep the sent version; the draft is in your tool's history or in the thread.
3. **Record one row per email**: date, kind of email (order question, refund, how-to, complaint), unchanged or edited, and for edits the main reason.
4. **Use a short list of reasons**: wrong fact, missing fact, wrong question answered, tone, length, promise removed.
5. **Once a week**, compute the unchanged share overall and per kind of email, and count the reasons.

Per kind of email is the important part. An overall rate hides the pattern: order-status drafts might go out unchanged almost every time while complaint drafts are always rewritten.

## How do you read the number?

The rate by itself says how much work the drafts save. The reasons say whether they are safe.

- **High unchanged, few fact edits.** The drafts do the job for this kind of email. This is where you could, later, consider letting the tool send on its own.
- **High unchanged, occasional fact edits.** Good, but the fact edits are the risk. Find the missing or outdated source behind each one.
- **Low unchanged, mostly tone and length edits.** The facts are fine and the style is off. Correct the tone settings or give feedback; this usually improves over a few weeks.
- **Low unchanged, many fact edits.** The tool lacks the sources for this kind of email. Add help articles or pages, or keep this kind of email away from the tool.

Watch the trend more than the level. A rate that climbs week after week means corrections are sticking. A flat rate while you keep making the same edits means they are not.

## When does the sent unchanged rate mislead?

It is honest, but it can still mislead. Four traps:

- **Rubber-stamping.** If people send drafts without reading them, the rate goes up and means nothing. Spot-check a few unchanged replies each week.
- **Easy mix.** A week full of simple questions produces a high rate. Compare per kind of email, not overall.
- **Edits for taste.** One teammate rewrites everything in their own style, another sends as is. The reasons tell you more than the count.
- **Small numbers.** Ten drafts a week is a noisy sample; a single odd email swings the rate. Look at several weeks together.

## Why is this better than a resolution rate?

A vendor's resolution rate usually counts conversations where the customer did not come back, under a definition the vendor chooses. The sent unchanged rate counts replies you judged good enough to send as written, under a definition you choose. One measures the customer's silence; the other measures your own judgement on each email. [Why AI resolution-rate claims mislead small businesses](https://replypen.com/blog/why-resolution-rate-claims-mislead.md) goes through the difference in detail.

## What target should you set?

Set it per kind of email, from your own starting point rather than an outside benchmark. A workable approach:

1. **Measure four weeks** before deciding anything.
2. **Pick the kinds of email** with the highest rate and no fact edits. Those are your candidates for less checking.
3. **For the rest, set a fix list, not a target**: which sources to add or update so the fact edits stop.
4. **Decide in advance** what rate, held for how long and with zero fact edits, would make you comfortable letting a kind of email go out with less review.

The last step matters if the tool can send on its own. [Draft or auto-send: when should AI send customer email itself?](https://replypen.com/blog/draft-vs-auto-send.md) covers that decision.

## How does ReplyPen report it?

ReplyPen's Reports show, week by week, how many Drafts were created, how many you sent unchanged and how many you edited, the share of incoming email that got a Draft, and an estimate of the time saved. Because every Draft sits in your own Gmail or Outlook thread and you send it yourself, the count comes from what you actually did. Each Draft's [Check note](https://replypen.com/product/inbox#what-is-in-a-check-note) shows the sources behind it, which helps when you look into why a Draft needed an edit.

For the full checking routine, read [How to check an AI-written answer before you send it](https://replypen.com/blog/check-ai-written-answer-before-sending.md).