AI-Resolved vs AI-Touched Tickets: How to Measure Real Customer Support Automation
An AI-resolved ticket is completed by AI without human intervention and remains resolved during the business's chosen confirmation window. An AI-touched ticket only shows that AI participated, perhaps by classifying the issue, drafting a reply, collecting details or sending the first answer.
The distinction matters because AI involvement can save time without removing the human work. If a person still reviews, corrects or completes the case, it should not be reported as an AI-only resolution.
A recurring concern in merchant discussions is that one generous label can make automation look more effective than it is. The answer is not to ignore AI assistance. It is to report assistance and autonomous resolution as different outcomes.
What is the difference between AI-resolved and AI-touched tickets?
AI-resolved describes the final outcome. AI-touched describes participation somewhere in the workflow.
| Ticket label | What happened | Human work required? |
|---|---|---|
| AI-resolved | AI supplied the answer or completed the task, with no related human work during the confirmation window | No |
| AI-touched | AI classified, summarised, drafted, replied or gathered information | Possibly |
| AI-assisted | AI reduced part of the agent's work, but a person completed the issue | Yes |
| Escalated | AI recognised a boundary and transferred the issue | Yes |
Consider a return request. If AI explains the policy and a staff member approves the exception, the ticket was AI-touched and human-resolved. If AI only drafts the agent's response, it was still useful, but it did not autonomously resolve the case.
This outcome-based approach complements the calculation of cost per durable support resolution. The financial guide asks what each completed issue costs. This article asks which completion label the issue deserves.
Use five outcome states instead of one automation label
A useful reporting system gives every customer issue one final state. These five categories keep assistance visible without inflating AI-only resolution.
AI-only durable resolution
AI completed the customer's request without a person joining, approving, correcting or finishing the task. The same issue did not reappear during the confirmation window.
Examples include providing a verified order status or answering a product question from current store data. This is the only category that belongs in the AI-only resolution numerator.
AI-assisted human resolution
AI completed a meaningful part of the work, but a person delivered or authorised the outcome. It might identify the intent, collect an order number, summarise the exchange or prepare a draft.
Assistance can reduce handling time. Report it separately so the saving can be measured without claiming that the person was removed from the case.
Human handoff or escalation
AI transferred the issue because the customer requested a person, the information was missing or the case required judgement. A timely handoff can be the correct service outcome even though it lowers the AI-only resolution rate.
The AI-first support boundaries guide explains why good automation should not make human access difficult.
Reopened or human-corrected outcome
The system initially marked the issue resolved, but the customer returned or an agent corrected the answer. Reclassify it instead of leaving the original automated label untouched.
If the customer comes back with a different issue, that is a new case. The store needs an intent and customer or order identifier to tell the difference.
Abandoned or unknown outcome
The conversation ended without proof that the customer received what they needed. Do not automatically count silence as resolution.
An abandoned conversation is not always a failure, but the evidence is incomplete. Keeping it separate makes the report honest.
Six rules for measuring real customer support automation
The purpose of these rules is consistency. They should be agreed before the monthly report is built, not adjusted afterwards to improve the result.
1. Count customer issues, not AI messages
One customer issue may contain ten messages. It may also move from web chat to email. Count the underlying need, such as the status of order 1842, rather than the number of replies the AI produced.
This prevents a long conversation from appearing more valuable than a concise one.
2. Require evidence that the outcome was completed
Define the signal that marks each intent as complete. A correct information response may be enough for a delivery-policy question. A cancellation request is not complete until the authorised action occurs.
Useful evidence includes a successful data lookup, a completed approved action, explicit customer confirmation or no same-issue return during a documented window. The required evidence should match the task.
3. Separate information answers from actions
AI may correctly explain how a refund works without issuing the refund. It may collect a new address without being authorised to change the order.
Report an information answer as resolved only when information was the customer's actual need. If the customer requested an action, the action must be completed through an approved workflow or by a person.
4. Apply a documented confirmation window
A resolution can fail after the conversation closes. Choose a period that gives repeat contacts, unsuccessful actions and corrections time to appear.
There is no universal window for every ecommerce intent. Apply the same rule to comparable cases and state it in the report.
5. Reclassify repeat contacts and corrections
When the same customer returns about the same order and issue, update the original status. The report should reflect the final operational outcome, not the most flattering point in the conversation.
For a broader cross-channel method, see how to measure whether support work stayed deflected.
6. Report AI assistance separately
Track the number of AI-assisted human resolutions and the agent minutes used. This shows whether classification, drafting or context collection saved time.
Do not add assisted cases to the AI-only resolution numerator. A clear report can show both autonomous completion and productive assistance without confusing them.
A worked ecommerce ticket-classification example
Suppose an ecommerce store reviews 1,000 eligible issues. Its platform initially labels 620 as resolved by AI.
A ticket-level audit finds:
- 430 stayed resolved without human work;
- 90 required an agent to complete an action;
- 45 were handed to a person later;
- 25 needed a human correction; and
- 30 ended without enough evidence to confirm an outcome.
The defensible AI-only resolution count is 430, not 620. The other 190 cases still contain useful information: 90 show assistance, 45 show handoff, 25 reveal quality failures and 30 require a clearer outcome signal.
The purpose of the audit is not to make the result look smaller. It is to reveal which work the AI completed, shortened, transferred or left uncertain.
What should an AI-resolution report contain?
Build a ticket-level ledger before presenting a percentage. At minimum, record:
| Field | Why it matters |
|---|---|
| Issue ID | Stops one conversation being counted twice |
| Customer or order ID | Helps match repeats across channels |
| Intent | Defines what completion means |
| AI involvement | Shows classification, drafting, answering or action |
| Human involvement | Separates assistance from autonomy |
| Completion evidence | Supports the final status |
| Repeat or correction | Reveals delayed human work |
| Final outcome | Supplies the reporting category |
Report the share of eligible issues in every state. A single automation percentage hides the distribution and makes changes difficult to diagnose.
Questions to ask an AI customer support vendor
Ask for the definition behind the dashboard before comparing rates or charges:
- What event marks a conversation as resolved?
- Does inactivity or auto-close count as success?
- Is the denominator every conversation, every AI-handled conversation or only selected intents?
- What happens when a customer returns about the same issue?
- How are multi-intent conversations treated?
- Does a human correction change the original status?
- Is billing triggered by the vendor's resolution label?
- Can the merchant export conversation-level outcomes for an independent audit?
These questions are more useful than asking for the highest published rate. Two products can display the same percentage while counting different events.
How AeroChat can support outcome-level reporting
AeroChat is an AI agent platform that helps ecommerce brands run customer service on autopilot.
Its AI-powered conversation insights show conversations handled, AI resolution rate, time saved, top topics and escalated topics. AeroChat defines AI resolution rate as the share of conversations fully resolved by AI without human intervention.
Those metrics provide useful reporting inputs. Escalated topics can identify where the AI reaches its boundaries, while time-saved estimates can help a merchant investigate assisted value.
When a conversation needs a person, AeroChat's human handover with conversation context keeps the transfer visible and gives the agent the previous exchange. The case should still be reported as escalated or human-resolved rather than AI-only.
Merchants should also check repeats, corrections and any contacts outside connected channels. AeroChat supplies the platform-level signals; the business defines its durable outcome and reconciles the final report.
Report automation that changed the work
AI-touched tickets show adoption. AI-resolved tickets show autonomous outcomes. AI-assisted tickets show time-saving potential. All three can matter, but they answer different questions.
Start by sampling 50 conversations currently labelled AI-resolved. Place each into one of the five outcome states and record the evidence. That small audit will show whether the headline percentage represents completed support or simply broad AI involvement.
