An AI classification proof of concept can quickly lead to a tempting rule: confidence ≥ 0.90 → AUTO. The number is clear, and it looks easy to put into a workflow. Let the system handle anything above 0.90; send the rest to a person.
I reached a similar point while building a JEV email classification PoC in n8n. For both L1 and L2 decisions, the workflow could record the winner, runner-up, probability or score, and margin. Once those numbers were visible, it felt as though choosing a threshold was the last step toward a production policy.
The workflow was missing a step. It measured uncertainty signals, but it had no validated policy for using those signals to separate AUTO from REVIEW. Even when the margin was narrow, the winner still became the final classification.
Model confidence is an observable signal, not business authority.
The practical question is how much automated action an application should allow on the strength of that signal.
What Does 0.90 Actually Mean?
Suppose the model classifies an email as ON_SITE_TECHNICAL_SUPPORT and reports confidence of 0.90. The dangerous reading is, “There is a 90% chance this is correct.”
Confidence, probability, score, and margin can mean different things depending on the model and method. With LLMs, uncertainty can arise from ambiguous input, variation in reasoning, and stochastic decoding. The 2025 KDD paper Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey describes why LLM uncertainty is difficult to reduce to a single probability.
For an enterprise workflow, the immediate question is more concrete: among past cases reported at confidence 0.90 or higher, how many were actually correct?
If about 90 of 100 such cases in a validation set were correct, the reported values in that range may be reasonably aligned with observed correctness. That is a useful starting point for understanding calibration. If only 70 were correct, the high-looking number would be a poor basis for granting AUTO authority.
Before selecting a threshold, measure how the signal relates to actual outcomes.
The JEV PoC Had Signals, but No Authority Policy
In my JEV email classification workflow, L1 recorded the selected probability, runner-up probability, margin, and confidence. L2 recorded the winner score, runner-up score, and margin. Keeping those values made the PoC more useful than a system that stored only the final category. It gave me a way to investigate questions such as:
- Which category pairs repeatedly have narrow margins?
- Does a particular L1 domain produce errors even at high confidence?
- Does the runner-up repeatedly fall near a boundary such as Spare Parts versus Repair?
- Does confidence change as an email thread gets longer?
- What signal patterns appear in cases people later correct?
The PoC stopped at measurement. It had uncertainty signals, but it did not yet have an operational policy validated against those signals.
A model can provide a prediction, score or probability, runner-up, margin, and supporting evidence. The application must decide whether the resulting operational state is AUTO, VERIFY, REVIEW, or REJECT. Those are different responsibilities. A winner does not acquire permission to trigger a business action simply because the model can calculate one.
Overall Accuracy Cannot Set the AUTO Policy
A second tempting shortcut is to ask whether overall accuracy is high enough. Would 95% accuracy justify automation? In an enterprise workflow, the answer depends on which decisions are wrong and what those decisions cause downstream.
Misrouting an internal information request might mean someone moves it to another queue. A wrong classification that leads to an RMA, an on-site engineer assignment, customer communication, or cost approval has a different consequence. Both count as one error in an accuracy figure.
The policy question is therefore: how much work can be automated, and which kinds of error can the business tolerate within that automated work?
- Coverage is the share of requests handled without human intervention.
- Risk is the danger of a wrong decision within that automated share.
Narrowing the AUTO range generally lowers coverage and increases human involvement. Widening it by lowering a threshold may reduce operating effort, but it may also increase the risk of a wrong automated decision. A threshold should follow validation data, signal calibration, analysis by category and confusion pair, the consequences of each error, and an explicit risk and coverage decision. Only then can the application assign AUTO, VERIFY, or REVIEW.
The Same 0.90 Need Not Grant the Same Authority
Consider two classifications that both report confidence of 0.90. One is an internal information request that can be reassigned if routed incorrectly. The other is On-site Support, where a downstream action could commit an engineer’s time or other resources. The model reports the same confidence, but the business consequences differ.
With sufficient calibration evidence, an application might allow a wider AUTO range for a low-impact category. It might require VERIFY at the same confidence for a high-impact category, or separate human approval before a critical side effect. These are policy choices grounded in observed performance and business impact, not different interpretations invented for the number itself.
In that sense, a confidence threshold belongs closer to policy configuration than to a model parameter. Model version alone is not enough to track it. Question contract version, validation dataset version, and policy version also matter. Even with the same model, a taxonomy change or a different validation distribution should trigger another look at the policy.
REVIEW Is a Normal Outcome
A high REVIEW rate can make an automation system feel unfinished. Yet sending an uncertain request to a person can be the correct production decision. In statistical machine learning, selective prediction and the reject option describe the same basic choice: accept predictions in a sufficiently reliable region and abstain elsewhere.
An enterprise workflow can express that choice in operational terms:
- AUTO: the current policy permits an automated decision.
- VERIFY: the application presents the AI suggestion for a brief human check.
- REVIEW: human judgment is the primary path.
- REJECT / FAIL-CLOSED: the current contract cannot handle the request.
REVIEW must also stay separate from OTHER. In the JEV PoC, OTHER was a category in the business taxonomy. REVIEW would mean that the current model decision has not been given operational authority. If OTHER becomes a bucket for uncertainty, the taxonomy itself deteriorates. A separate REVIEW state lets the workflow manage uncertainty without changing what its categories mean.
Human Review Is Part of the Measurement Loop
Human-in-the-loop is often described as a final check because AI can make mistakes. In production, an approved or corrected human decision has another role: it becomes outcome data for the next evaluation and calibration cycle.
AI prediction → current AUTO/VERIFY/REVIEW policy → human-approved outcome → evaluation dataset → calibration and policy regression → next policy version.
The prediction and the approved outcome need separate records. Treating the prediction as ground truth would make calibration impossible.
There is also a human decision problem. A well-calibrated AI confidence value does not guarantee that a person will use it well. Human-Aligned Calibration for AI-Assisted Decision Making examines why aligning confidence with correctness alone may not tell a human decision-maker when to rely on AI.
In an enterprise interface, a prominent “confidence = 97%” can read like a near-certain recommendation rather than a model signal. Interface design has to consider whether to show the confidence value, an alternative candidate, the supporting evidence, or a known boundary case. It may also need to capture why someone approved or corrected a suggestion. Calibration does not end with a backend metric; it affects how people rely on the system.
How I Would Prepare This PoC for Production
If I were taking the email classification PoC toward production now, I would build the evidence base before choosing a threshold.
- Record the prediction and its signals. Alongside the final category, keep the runner-up, score or confidence, margin, boundary signals, and model and contract versions where available.
- Store the human-approved outcome separately. The AI prediction cannot serve as its own ground truth; approval or correction needs its own outcome record.
- Evaluate slices, not just overall accuracy. Examine categories, confusion pairs, confidence and margin ranges, thread length, boundary cases, and business consequences. An overall 95% could be driven by one easy category.
- Derive AUTO policy from validation results. Look at the risk observed in each signal range and whether the affected business process can accept it. Then choose the AUTO range.
- Recheck production drift. New products, customer types, taxonomy changes, and prompt or question changes can make earlier calibration less relevant. A threshold is an operating policy that needs revalidation.
Confidence Needs an Authority Contract
A category and confidence score can make a one-number automation rule look complete. In a real enterprise workflow, the path is longer: Model Signal → Validation Evidence → Business Risk Policy → Execution Authority.
The model can report what it selected and provide a score or margin. The application and its operating policy decide what actions those numbers can authorize. AUTO, VERIFY, and REVIEW are authority states assigned from validated evidence and business consequences; they are not confidence labels produced by the model.
Confidence is not authority.
The aim is to measure and control what the system may do, including when the model sounds sure.
References
- Xiaoou Liu, Tiejin Chen, Longchao Da, Chacha Chen, Zhen-Yu Lin, and Hua Wei. Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey. KDD 2025.
- Manuel Gomez Rodriguez. Human-Aligned Calibration for AI-Assisted Decision Making. 2023.