SeedSweep

Containment rate vs resolution rate

Containment rate is the share of chatbot conversations that ended without reaching a person. Resolution rate is the share the vendor counts as resolved, under a rule the vendor writes. A conversation can be contained without being resolved, for example when the customer gives up and calls instead.

What each rate counts

Containment rate is the number of conversations that never went to a person, divided by all conversations the chatbot handled. It counts the conversations your staff did not take over, even when the customer then called or emailed them. It tells you nothing about whether the answers were right.

Resolution rate is the number of conversations counted as resolved, divided by the conversations the vendor includes in its total. The hard part is the word "resolved". Many customers leave without saying whether their problem was solved, so the vendor needs a rule for those conversations, and that rule decides the number.

You will also see automation rate. It usually means the share of work the AI finished without a person. Fin, an AI support agent made by the company that was called Intercom until May 2026, says in its glossary that automation rate is "calculated by multiplying Involvement Rate by Resolution Rate". Involvement rate is the share of all conversations Fin takes part in, so an automation rate built this way depends on how resolution is counted.

How a vendor decides a conversation was resolved

Fin counts two kinds of resolution. Its help center says a resolution is counted when, after Fin's last answer, the customer "either confirms the answer was satisfactory (confirmed resolution), or exits the conversation without requesting further assistance (assumed resolution)". Fin charges per outcome, and a customer who leaves without asking for more help "is counted as an Assumed Resolution and will be charged". If the customer later returns to the same conversation for more help, "that resolution will be deducted and not charged", and "if Fin detects frustration and escalates to a human agent, that escalation is not billed".

Assuming a resolution is a fair guess when someone asked how to reset a password and left. It is a poor guess when a chatbot quoted last year's refund policy and the customer left to dispute the charge with their bank. Under a rule like this, both count as resolved.

Why the gap matters

If the resolution rate in front of you comes from the vendor's dashboard, it follows the vendor's rule. When assumed resolutions make up a large share, much of the number measures how often customers stopped replying rather than how often they got a correct answer.

The same question comes up when buying a company. In January 2025 Presto Automation settled SEC charges over what it told investors about its drive-thru voice AI between 2021 and 2023, without admitting or denying the findings. The SEC's order says Presto's reported automation rates counted orders completed without restaurant staff involvement "but not without any human involvement". Those are two different counts. If you pay a price based on one count, check that it is the count you meant.

How to check the number yourself

You can run this check with your own team if the person grading did not choose the vendor.

  1. Write down the definitions in use. Find the vendor's rule for resolved, assumed resolved and escalated in its documentation or your contract.
  2. Export conversations from the last 90 days. Choose them from every topic and outcome, including ones counted as resolved, ones counted as assumed resolved and ones that went to a person.
  3. Grade each answer against the help center article or policy that was in effect when the conversation happened, not today's version. Mark it correct, wrong, based on an outdated article or impossible to tell.
  4. Check what happened next. Look for a repeat contact by any channel within a week, a refund request or a chargeback.
  5. Have a second person grade part of the sample without seeing the first grades, and compare. If the two often disagree, agree on clearer grading rules before you trust any rate.

You end with three numbers side by side. They are the containment rate, the vendor's resolution rate and the share of conversations where the answer was correct and the customer did not come back with the same problem.

An example

A support chatbot solved 61% of 400 conversations, not the 78% its vendor reported

Reportedcounted as resolved by the vendor

Measuredproblem solved, by our evaluation

Differencecounted as resolved but not solved

17 points
From an evaluation we ran for one of our clients, shown in full in the sample report.

We ran this evaluation for the client before a contract renewal. Most failures came from one outdated pricing page, and the vendor counted 68 of the 156 failed conversations as resolved.

What to ask your vendor

  • How is a resolution counted, and are assumed resolutions included in the rate you report?
  • Are assumed resolutions billed?
  • Can we export the conversations behind the rate, with the label each one received?
  • Does a conversation still count as handled by AI if a person approved or edited the answer?
  • Which version of each help center article did the chatbot use on a given date?

Before you renew, switch or buy

If a renewal, a vendor switch or a purchase depends on one of these rates, tell us the decision and when you need to make it. We reply within 24 hours.

Contact us