What Building an Email Classifier with JEV Taught Me About Decision Boundaries

When I first looked at JEV, I thought of a Salesforce Email-to-Case problem I had worked on: read a customer email and assign the right Level 1 and Level 2 category. The task called for a choice within a defined business taxonomy, not a generated response. A model built for classification seemed like a natural fit.

Connecting an n8n workflow to sample emails in Google Sheets and calling the JEV API was quick. The slower part began when a classification looked wrong. I kept returning to the questions I had written and the boundaries between categories.

Eventually, the central question changed. Instead of asking what an email was about, I asked: What does the sender expect us to do now? That distinction shaped the rest of the proof of concept.

The Taxonomy Became a Decision Contract

I started with five Level 1 business domains:

• ENGINEERING_SUPPORT
• SALES_SUPPORT
• POST_SALES_SUPPORT
• TRAINING_PROGRAM
• OTHER_QUESTIONS

Naming those domains was easy. Defining when to choose each one, and when to rule it out, took more work. The Level 1 request eventually included this decision rule:

decision_rule:
  'Classify the action that the sender currently expects the recipient to perform, not merely products, symptoms, technical words, historical events, or possible solutions mentioned in the email.'

A customer email can mention a product, a fault, a part replaced months ago, and the customer’s theory about the cause. The word seal does not make the request Spare Parts. The word replacement does not necessarily make it Repair.

Each category therefore needed include and exclude criteria alongside its definition. I also made the boundary around OTHER_QUESTIONS explicit. OTHER is a real business category, not a place to send whatever the model finds uncertain.

That made the specification feel less like a classifier prompt and more like a small decision contract: a reviewable statement of which business request each category represents.

In an Email Thread, the Verb Matters More Than the Noun

Post-sales requests exposed the sharpest boundaries. The same equipment and fault could lead to four different categories:

• “What is this part number, and is it in stock?” points toward Spare Parts.
• “Can you tell us what caused this alarm?” points toward Remote Technical Guidance.
• “Please send an engineer to the site” points toward On-site Technical Support.
• “Please repair this unit or process an RMA” points toward Repair Request.

The equipment and symptoms may be identical. The action the customer wants from the recipient changes the classification.

Threads make this harder. The newest message might say only “Any update?” On its own, that sentence offers little to classify. If it follows an unresolved request for an engineer visit, though, the active request remains on-site support. The workflow followed four rules:

• Give priority to the newest message written by the sender.
• If it is a follow-up, reminder, scheduling note, or escalation, recover the earlier unresolved request.
• Do not let a completed past action override the current request.
• Use a specific category when even a short follow-up clearly refers to an unresolved request in the thread.

I have found a similar issue when interviewing teams about enterprise taxonomies. Asking “What does this category mean?” often produces an abstract definition. A more useful question is: What must the next team actually do when a request lands here? That operational boundary gives the classifier something concrete to decide.

Narrow the Choice Before Asking the Next Question

I did not ask JEV to pick from nearly 20 final categories at once. The workflow first chose one of the five Level 1 domains, then evaluated Level 2 questions specific to that domain. A deterministic resolver produced the final category.

Email Thread
  ↓
L1 — 5-way business domain
  ↓
Domain-specific L2
  ↓
Deterministic Resolver
  ↓
Final Classification

If Level 1 selected POST_SALES_SUPPORT, Level 2 could assess Spare Parts, Remote Technical Guidance, On-site Technical Support, Commissioning, Repair/RMA, and Maintenance requests. Each JEV noul was a small question with separate true and false criteria:

function q(task, trueCriteria, falseCriteria, extra) {
  const instructions = { task: task };

  return {
    type: 'noul',
    instructions: instructions,
    criteria: {
      true: trueCriteria,
      false: falseCriteria
    }
  };
}

That structure made the boundaries inspectable. To revisit Spare Parts versus Repair, I could read those two questions. To clarify On-site versus Remote support, I could revise their criteria. A business analyst, operator, or developer could review the same rules. Knowledge that had been implicit in the taxonomy became an application asset.

Hierarchy has a cost. If Level 1 chooses the wrong domain, the correct Level 2 category disappears from the candidate set. An email incorrectly sent to ENGINEERING_SUPPORT cannot reach POST_SALES_SUPPORT__REPAIR_REQUEST, however well the Level 2 questions work. Improving this design requires attention to low-margin Level 1 choices, cross-domain confusion pairs, and known boundary cases, as well as better Level 2 criteria.

The Useful Work Was Writing the Questions

At first, I cared most about whether JEV returned the right answer. By the time the workflow reached v3, I was more interested in how its questions were specified.

A general LLM classifier can be prompted to read an email, choose one of 20 categories, and explain why. For this experiment, I broke that request into smaller decisions: Is spare-parts handling itself the primary request? Is the sender asking for remote technical guidance? Is an engineer visit the requested action? Is repair or RMA the explicit outcome?

Each question had its own true and false boundary. To me, this was closer to decision specification engineering than prompt engineering. Copying an existing taxonomy into a prompt would have preserved its hidden ambiguities. Writing the boundaries forced those ambiguities into language that people could review and agree on.

Models and APIs can change. The application still needs to own the policy that says which situations count as which business requests.

Stop Calling the Model When the Choice Is Settled

One small branch in the workflow illustrates that ownership. When Level 1 returns TRAINING_PROGRAM, there is no further Level 2 choice to make. The workflow takes a direct path instead of calling JEV again:

if (domain === 'TRAINING_PROGRAM') {
  return {
    json: {
      ...input,
      l2_execution: 'DIRECT',
      direct_l2_action: 'TRAINING_PROGRAM',
      direct_classification:
        'TRAINING_PROGRAM__TRAINING_PROGRAM'
    }
  };
}

The saved call reduces latency, cost, and another possible failure point. More significantly, the reason for the result is explicit. Once a rule settles a choice, code can carry it through. In this workflow, that is a concrete version of a principle I have returned to on Peter’s Stack: let AI handle uncertainty and deterministic workflows handle certainty.

Let the Model Score; Let the Application Decide

The workflow did not pass a Level 1 response straight to Level 2. It checked that the answer type was choice and the selected domain was allowed. It retained the selected and runner-up probabilities, then calculated their margin.

At Level 2, JavaScript read the noul scores, built a candidate list, and sorted it deterministically. The action name breaks a score tie:

candidates.sort(function(a, b) {
  if (b.score !== a.score) {
    return b.score - a.score;
  }
  return a.action.localeCompare(b.action);
});

const winner = candidates[0];
const runnerUp = candidates[1];

A separate categoryMap then resolves the winner to a canonical business code. The governing rule is simple:

The model produces scores. The application turns those scores into a business decision.

An unexpected answer type or disallowed domain raises an error. An L2 execution route other than JEV or DIRECT goes to a separate fail-closed path. These checks matter when a result enters an enterprise workflow: the application should not have to guess what an approximately valid model response meant. Code owns the allowed business states.

A Recorded Margin Is Not Yet a Review Policy

The PoC records useful signals. At Level 1, it retains selected probability, runner-up probability, confidence, and margin. At Level 2, it retains winner score, runner-up score, and margin. It also stores question version, model version, token usage, and run status.

I initially felt that this provided a good view of uncertainty. One step was still missing: the workflow measured uncertainty without using it as an operational policy. In v3.0.1, even a narrow margin still produces a final classification for the winner.

uncertainty measurement
        ≠
uncertainty policy

The production step is not to choose a threshold by intuition. It is to accumulate human-approved outcomes and examine how often a classification is correct given its Level 1 and Level 2 margins, category pair, and thread pattern. A low margin might be risky for one cross-domain pair while a smaller margin remains stable for another category.

Only after that calibration should a deterministic policy define when a result goes to AUTO, VERIFY, or REVIEW. A rule such as “margin below 0.2 means REVIEW” would claim a boundary this PoC has not validated.

OTHER and REVIEW Answer Different Questions

OTHER belongs to the business taxonomy. REVIEW is an operational state in which the current model decision is not automatically accepted.

Sending every uncertain email to OTHER would be easy to implement, but it would mix ordinary general questions, requests outside the taxonomy, and requests the model simply could not classify confidently. That would make OTHER harder to understand and contaminate the evidence needed to improve the taxonomy.

Keeping REVIEW separate lets the application ask two independent questions: Does this request belong to OTHER as a business category? Can this model result be accepted automatically? The distinction holds whether the classifier uses JEV or a general LLM.

Could a General LLM Use the Same Design?

Yes. This experiment does not establish that JEV is superior to a general LLM. If I implemented the same taxonomy with an LLM, I would preserve the explicit boundaries developed here:

Email Thread
  ↓
Active Request Extraction
  ↓
Constrained L1 Classification
  ↓
Domain-specific L2
  ↓
Boundary Verifier when needed
  ↓
Deterministic Policy
  ↓
AUTO / REVIEW

A general LLM is more flexible with long context, unexpected wording, evidence explanations, and new edge cases. But a self-generated value such as confidence: 0.93 should not be treated as the same kind of signal as JEV’s probability. For an LLM implementation, I would examine system-level evidence:

• Did the output satisfy the structured schema?
• Is there direct evidence for the selected category?
• What is the alternative candidate?
• Is this a known boundary case?
• Do the classifier and verifier agree?
• How did comparable decisions fare against past human-approved outcomes?

The model can change while the decision harness remains.

The Pattern I Would Carry into Enterprise AX

This PoC clarified a set of practical design choices for AI classification:

1. Rewrite the business taxonomy around requested actions and explicit include/exclude boundaries instead of copying category names into a prompt.
2. Separate the current request from historical context. In a long thread, the unresolved action matters more than the most frequently mentioned topic.
3. Version the decision contract, not just the category names or model. The workflow records question_version: hierarchical-v3.0 so a result can be tied to its decision criteria.
4. Treat probability and margin as signals, not service guarantees.
5. Put assignment, case updates, and customer notifications behind deterministic policy rather than letting the classifier directly change business state.
6. Use human corrections to revisit category boundaries, confusion pairs, and eventual AUTO/REVIEW thresholds.
7. Remove AI calls where a rule has already settled the choice.

JEV looks especially suited to a finite, relatively stable category set that can be divided into constrained choices or yes/no questions. It also fits repeated classification work that does not need generated prose, provided the organization can maintain explicit decision boundaries and the application owns the final side effects.

A general LLM may fit more naturally when the taxonomy changes continually, long multistep reasoning or external knowledge retrieval is central, or the output must be descriptive. The choice depends on the shape of the problem and the authority contract around the model.

Classification Is a Runtime Boundary

I began by trying a new model. I ended up revisiting a familiar architectural boundary. An agent may choose a tool without owning permission to execute it. Likewise, a classifier may propose a category without owning the decision to change a business record.

JEV or an LLM can provide bounded judgment. The application must own taxonomy, validation, retry and review policy, and authority over business changes. The lasting work in this PoC is the decision contract: breaking a business judgment into small questions and defining when their answers can become an operational decision.

The next experiment is to compare JEV and a general LLM against the same canonical taxonomy and validation set, inside the same decision harness. Accuracy alone would be too narrow. Boundary cases, margin or verifier signals, human corrections, latency, cost, and operational complexity all belong in the comparison.

The key design question is which judgments to give AI, and where those judgments become business decisions.