Tickets closed, handle time and raw CSAT survive in reports because they always look fine. Here is the replacement set, how each number is gamed, and the weekly page a support lead can defend.
Also available in Dutch: Klantenservice meten zonder schijnstatistieken
Most support dashboards are full of numbers that go up when things go well, go up when things go badly, and cannot tell you which of the two is happening. They survive because nobody has to defend them: a metric that always looks acceptable never generates an uncomfortable meeting. The cost is that the queue can be deteriorating for a month before any chart notices.
None of these numbers is fake. Each one measures something real and then gets used to answer a question it cannot answer.
| Metric | What it actually measures | How it goes wrong |
|---|---|---|
| Tickets closed | Throughput of the closing action | Closing is an action an agent takes, not an outcome a customer experiences. It rises when the team closes aggressively, splits threads, or when a bad week produced more tickets to close. |
| Average handle time | How long agents spend per conversation | It is a cost input treated as a quality signal. Pushing it down rewards short answers and rushed handoffs, and the tickets that come back as a result land in next week's volume, where nobody attributes them. |
| First-contact resolution, self-reported | Whether an agent ticked a box | The person who ticks the box is the person being evaluated by it. Unless it is derived from whether the customer came back, it measures optimism. |
| Chats handled by AI | How many conversations the widget participated in | Participation is not help. This is the number that produces impressive automation percentages while the queue does not shrink. |
| Raw CSAT | The opinion of the small minority who answered | At a low response rate the score is dominated by the delighted and the furious. It is not wrong, it is just about a different population than the one you serve. |
Deflection rate deserves its own mention, because it is the one most often reported as a quality number when it is a capacity number. It counts contacts that did not arrive, and a customer who gave up counts the same as a customer who was helped — the difference is worked through in deflection rate vs resolution rate.
Three corrections turn first-response time from decoration into a signal. Report the median and the 90th percentile rather than the mean, because the mean is set by a handful of conversations that sat over a weekend. Measure business hours only, or you are grading yourself on the clock rather than on the service. And measure it on conversations that reached a human, because mixing in instant AI replies produces an average that is technically true and operationally meaningless.
The 90th percentile is the one to watch week over week. A median that holds steady while the 90th percentile climbs is the classic signature of a queue where the routine work is fine and the hard work is quietly rotting.
A single satisfaction score across a mixed queue tells you almost nothing you can act on. Split by how the conversation ended — resolved by AI, or resolved by a person — and the same responses start answering real questions: whether automation is holding quality, and which topics should not be automated at all.
That split has to be built into the collection path rather than reconstructed afterwards. In CustomerEagle a rating is a whole integer from 1 to 5, one per conversation, and it is stored together with whether the conversation was AI-resolved, so the breakdown is available without joining anything by hand. There are two places a customer can rate — the card in the widget and the link in the resolved-conversation email — and both run through the same code path on purpose, so the two channels cannot drift into measuring different things.
One rating per conversation is a deliberate limit: a second submission is rejected rather than averaged in, so a follow-up cannot quietly overwrite the first impression. A low rating also does something beyond reporting. On a conversation that was counted as AI-resolved, a rating of 2 or lower reverses that resolution — it comes out of usage and is credited back if it was already invoiced, which is the same settling-window mechanic described on how we measure AI resolutions. A satisfaction score that can cost the vendor money is a satisfaction score the vendor has an interest in collecting honestly.
Cost per resolved contact is the number executives ask for and the one most easily improved in the wrong direction. It falls when handle time is squeezed, when agents close faster, and when customers give up — none of which you want. It should fall because contact rate fell: fewer people needed to ask, because the article existed, the tracking page was clear, or the product stopped generating the question.
So report it as a pair. Cost per resolved contact next to contact rate, and a movement in the first that is not explained by the second is a movement to investigate rather than celebrate. If part of your cost is billed per resolution, the definition behind that billing is part of the metric — the shapes vendors use are compared in what is an AI resolution.
Seven numbers, one page, each with a note on what it would look like if it were being gamed. If a number cannot be defended in that last column, it does not belong on the page.
| Number | The question it answers | What gaming it looks like |
|---|---|---|
| Conversations, and contact rate per order or account | Is demand growing faster than the business? | Merging or splitting threads to move the count without moving the workload. |
| Resolution rate, zero human replies | How much is genuinely being handled without a person? | Counting conversations where an agent sent one message, or counting silence as success. |
| Reopen rate within a stated window | Did the resolutions hold? | Shortening the window, or opening a new conversation instead of reopening the old one. |
| First response: median and 90th percentile, business hours | How long does a person actually wait? | An auto-acknowledgement counted as the first response. |
| Backlog older than 24 and 72 hours, as counts | Is anything rotting at the bottom? | Closing old conversations to clear the count, which reappears as reopens. |
| CSAT split by AI-resolved and human-resolved, with response rates | Is quality holding where automation grew? | Sending the survey only after conversations that went well. |
| Top handoff reasons, grouped | What should be written down this week? | A catch-all 'other' category large enough to hide the real answer. |
The last row is the one that turns a report into work. Grouped handoff reasons are a content backlog in priority order, and the top reason on Monday is usually an article that can be written on Tuesday.
A metric is only as good as the event behind it, and most reporting arguments are really arguments about events. Before a number goes on the page it needs three things: a defined event with a written trigger, a named owner who is allowed to change the definition, and an export you can check against real transcripts.
None of this requires a data team. It requires deciding which seven numbers you will defend, writing down what each one means, and being willing to publish a number that goes the wrong way. Where the reporting for the AI side of this lives in the product is on the features overview, and the definitions our own numbers use are on how we measure AI resolutions.
Vanity metrics are numbers that reliably look acceptable regardless of what customers experienced: tickets closed, average handle time, self-reported first-contact resolution, chats the AI participated in, and raw CSAT collected at a low response rate. Each measures something real, but none of them distinguishes a customer who was helped from one who gave up.
Resolution rate on a strict definition, reopen rate within a stated window, contact rate per order or per active account, backlog age as a distribution rather than an average, response time as a median and 90th percentile on business hours, and CSAT split by whether the conversation was resolved by AI or by a person.
Yes, and it is the only split that reliably produces action. A blended score hides whether automation is holding quality or eroding it, while the split tells you which topics are safe to automate and which should route to a person. Report the response rate for each segment alongside the score, because the two segments rarely have the same one.
As a median and a 90th percentile rather than a mean, over business hours only, and calculated on conversations that reached a human rather than on all conversations. Instant AI replies mixed into the same average produce a figure that is technically correct and tells you nothing about how long a person waited for a person.
There is no portable benchmark, because the score depends on your channel mix, when the survey is sent and how many customers answer it. The comparison that means something is against your own baseline: measure it now with the response rate recorded alongside it, split by AI-resolved and human-resolved, and track the direction rather than the level.
Write down, for each number on the report, what it would look like if someone were optimising for the number instead of the outcome — closing threads early, splitting conversations, sending surveys only after good interactions. If you cannot describe that failure mode, you do not yet understand the metric well enough to act on it.
2026-08-10 · 10 min read
A proactive message is one the customer did not ask for. Which signals are worth acting on, what a message has to contain to be support rather than noise, and how to measure win-back without fooling yourself.
2026-08-09 · 10 min read
Customers forgive a bot that does not know. They do not forgive one that will not let go. The triggers worth wiring, what has to travel with the conversation, and how to tell a good handoff from a bad one.
2026-08-09 · 9 min read
The biggest driver of your AI resolution rate is not the model — it is your content. A practical guide: one question per article, front-loaded answers, real values instead of screenshots, and how to test with your own tickets.
See how honestly-measured AI resolutions cut your support load — start on the Free plan, no credit card, no sales call to get started.