I classified the same 17 emails in two completely different ways.
One used TypeSafe AI’s JEV, a decision model built around predefined questions, finite answers, probabilities, and scores. The other used GPT-6 Luna, reading the full email thread and returning a structured interpretation of the current request together with the basis for its classification.
At first, the result looked almost anticlimactic.
Both engines produced the same final classification for all 17 cases.
My first reaction was obvious: if they reach the same answer, why not simply choose the faster or cheaper one? But when I looked again at the workflow, the opposite became more interesting. Matching answers did not make the two systems look more alike. They made it easier to see where each system requires engineering complexity in order to reach that answer.
This is not a benchmark declaring a winner between JEV and GPT-6 Luna. It is a record of the Decision Layer design differences I encountered while implementing the same Salesforce Email-to-Case classification problem in two different ways.
They agreed 17/17, but the numbers needed a second reading
I reused the same 17 emails from the earlier Email-to-Case PoC.
JEV used jev-1.13.0. The general-purpose LLM path used gpt-6-luna with medium reasoning. Both were evaluated against the same business taxonomy.
| Observed metric | JEV 1.13.0 | GPT-6 Luna Medium |
|---|---|---|
| Final classification agreement between engines | 17/17 | 17/17 |
| Match against existing expected labels | 15/17 | 15/17 |
| Input tokens | 94,996 | 49,590 |
| Output tokens | 4,060 | 4,825 |
| Estimated API cost for 17 cases using the public prices checked at the time | about \$0.00399 | about \$0.00737 |
| Comparable decision window observed in this workload | about 1.68s | about 5.89s |
The cost calculation used the public prices rechecked on October 1, 2026. TypeSafe AI listed Jev input at \$0.042 per million tokens with no separate output charge. OpenAI’s GPT-6 Luna Standard short-context pricing was \$0.10 per million input tokens and \$0.50 per million output tokens. So GPT-6 Luna used far fewer input tokens in these 17 cases, yet its estimated API cost was about 1.85 times JEV’s. The calculation followed TypeSafe AI’s Jev introduction and the OpenAI GPT-6 Luna model documentation.
Latency needs more caution. The two experiments did not use perfectly identical instrumentation. I cannot turn these measurements into a general claim that JEV is 3.5 times faster. The defensible statement is narrower: in this n8n workload, the comparable decision windows observed were about 1.68 seconds and 5.89 seconds.
But the number that became more interesting to me was not 17/17. It was 15/17.
If both engines missed the same two cases, were the models really wrong?
Against the existing expected labels, both JEV and GPT-6 Luna matched 15 of the 17 cases. S005 and S013 were counted as failures.
At first, that looked simple.
15 / 17.
Then I reread the two email threads, and the meaning of that score became less stable.
S005 — It had already moved beyond ‘online technical support’
The existing label for S005 was POST_SALES_SUPPORT__TECHNICAL_SUPPORT_ONLINE.
But the thread had already gone through repeated troubleshooting over a long period. The manufacturer’s technical team had been involved. There had been an onsite visit. Recommended actions had been performed and had not resolved the issue. An earlier message explicitly asked for an indoor unit replacement, and the latest message was no longer asking someone to visit and diagnose the problem again. It was an escalation asking for an appropriate resolution now.
JEV classified it as REPAIR_RMA at 0.91, with ON_SITE_TECHNICAL_SUPPORT second at 0.29. GPT-6 Luna also placed REPAIR_REQUEST first.
Two independent engines reached the same conclusion.
Should that automatically count as two model errors because the existing label says otherwise?
When I reread the thread, it looked at least as plausible that the human label had not fully captured the accumulated state of the conversation.
S013 — Read the action being requested, not just the noun ‘repair’
S013 was a boundary case in the opposite direction.
The existing label was REPAIR_REQUEST, but the customer’s request was very specific: they wanted the manufacturer to send a technician to respond to the fault, and they were willing to pay for the visit.
JEV scored ON_SITE_TECHNICAL_SUPPORT at 0.97 and REPAIR_RMA at 0.87, leaving a margin of only 0.10. GPT-6 Luna also chose TECHNICAL_SUPPORT_ON_SITE_VISIT as primary and REPAIR_REQUEST as secondary.
In everyday language, calling this a repair request is reasonable. But under the taxonomy boundary we had defined in the earlier PoC, the action currently requested from the recipient is a technician visit. Repair may be an outcome after the visit.
The small JEV margin was useful here too. The model did not pretend that the boundary was wide. It left a signal that On-site Support and Repair were close alternatives.
This was where the evaluation problem became more interesting than the model problem.
A disagreement between model prediction and a human label is not sufficient evidence of model error. The ground truth itself has to be validated against the business decision policy.
In an enterprise project, a validation set is less like a timeless answer key and more like an artifact that compresses business policy. Change the taxonomy boundary and some labels may need to change with it. A label does not become canonical truth simply because a human assigned it once.
With JEV, I spent more time designing a Question Graph than a prompt
In the earlier JEV implementation, the API integration was not where most of the work went.
The hard part was designing the questions.
At L1, the workflow narrowed an email into a broad domain such as Engineering, Sales, Post Sales, Training, or Other. At L2, it asked small yes/no decisions independently for categories inside the selected domain.
Simplified, the path looked like this:
Email Thread → L1 finite choice → Domain-specific L2 questions → probability / score → deterministic resolver → business category
The important asset was not a single ‘good prompt’.
For Spare Parts, I had to define what kind of request should evaluate to true. For Repair, what belonged inside the category and what did not? For On-site Support, when should it be selected, and when should it explicitly not be selected? The business boundary had to be decomposed into small decision contracts.
With JEV, complexity accumulated here:
- how small each question should be
- how to define the true/false boundary for each category
- how to divide L1 and L2
- what signal to preserve when scores are close
- how code should handle ties or unexpected responses
- where the deterministic resolver should take ownership of the final category
In the end, the more important question was not where to put intelligence, but which questions the intelligence should be allowed to answer.
That structure felt quite enterprise-friendly. A Business Analyst and a developer could read the same question contract and review the category boundary together. When a classification looked wrong, the first question did not have to be ‘why did the model do this?’ We could open the relevant decision contract instead.
The trade-off was equally clear.
If the hierarchy narrows incorrectly, the correct L2 option can disappear from the candidate set entirely. If the taxonomy changes frequently, the question graph changes with it. And if every long, exceptional context is forced into a set of tiny questions, the decomposition itself can become the complexity.
With GPT-6 Luna, the complexity moved somewhere else
For the general LLM version, I did not translate the JEV request into one enormous prompt.
Instead, I asked one structured inference to return several things together:
primary classification / secondary classification / decision strength / thread resolution / boundary code / current request / decision basis
If questions were the application asset in JEV, the LLM path made the evidence contract that reveals how the context was interpreted more important.
For S013, for example, simply returning ON_SITE was less useful than returning a bundle such as:
- primary: On-site Visit
- secondary: Repair Request
- boundary:
ON_SITE_VS_REPAIR - current request: technician visit
- basis: the current action requested is the visit itself, not the possible repair outcome
This approach was strong when the thread was long or several intentions overlapped. It was easier to connect a follow-up that made little sense in isolation to the history that gave it meaning.
But receiving structured output did not make the production classifier complete.
The application still had to own:
- the allowed taxonomy
- schema validation
- rejection of invalid categories
- primary/secondary consistency
- policy for each boundary
- whether a case becomes REVIEW
- authority to mutate the actual Salesforce fields
In other words, where Prompt Engineering ended, Evidence Contract and deterministic validation engineering began.
The same answer does not make the same system
Once I placed the two implementations side by side, the original question — which model is better? — became much less useful.
In these 17 cases, the outcomes matched.
But the control surfaces used to produce those outcomes were different.
| JEV | General LLM | |
|---|---|---|
| Core design asset | Question Contract | Context + Evidence Contract |
| Output | finite typed decision / probability | structured semantic interpretation |
| Natural strength | narrow, repeated bounded judgment | long context, exceptions, multiple intents |
| Main risk | poorly designed questions or hierarchy | semantic drift and policy leakage |
| What remains in the application | resolver / threshold / authority | validation / policy / authority |
So before asking ‘JEV or LLM?’, I would look at the workload first.
If categories are finite and stable, the same judgment repeats at scale, and the decision can be decomposed into small bounded questions, a specialized decision model such as JEV is a natural fit.
If the system must interpret long context, unusual thread transitions, and ambiguous multiple intents that are difficult to express through taxonomy rules alone, a general LLM can be much more convenient.
The two approaches looked less like complete substitutes and more like systems that pay complexity in different places.
Fewer tokens did not automatically mean lower cost
One of the more interesting parts of the comparison was the relationship between tokens and cost.
GPT-6 Luna used almost half as many input tokens as JEV.
Yet at the public prices used for the comparison, its estimated API cost was higher.
There is nothing contradictory about that. Token volume and token price are separate variables.
JEV repeated input across L1 and several L2 questions, so its total input-token count was larger. At the time, however, TypeSafe listed Jev input at \$0.042 per million tokens with no separate output charge.
GPT-6 Luna packed more judgment into one richer structured inference, reducing input tokens, but its Standard pricing was \$0.10 per million input tokens and \$0.50 per million output tokens.
For these 17 cases, that produced an estimated cost of about \$0.00399 for JEV and \$0.00737 for GPT-6 Luna.
I would not extrapolate that into ‘JEV is always half the price.’ Economics can change with prompt length, category count, caching, batch or flex usage, reasoning settings, and the shape of the question decomposition.
The more useful lesson was this:
The cost of a decision workload cannot be understood from the model price card alone. You have to inspect the shape of the entire decision graph.
In production, I probably would not choose only one
Seventeen cases are not enough to choose a production architecture.
But if I redesigned the system from this experience, I would probably examine a hybrid path before sending every email through one model.
Email → bounded / repeatable decision → known and calibrated case → deterministic routing → ambiguous / exceptional case → richer semantic interpretation → verify / review
If most traffic falls into stable routine categories, a low-cost bounded decision engine can handle that path. Known confusion pairs, low-margin cases, or complex thread transitions can escalate to a general LLM or human review.
The important point is not that JEV specifically must become the first-stage model.
The structural principle is broader.
Do not price narrow, inexpensive judgment and broad, expensive interpretation as though they were the same kind of work.
Sending every case to the strongest model may simplify implementation, but for repeated workloads it can expand not only cost and latency but also the failure surface.
Forcing every exception into a finite taxonomy is not a good answer either.
The application needs to know which cases are routine and which cases are escalations.
Then OpenAI announced the Decisions API
While I was writing up this experiment, OpenAI announced the Decisions API at DevDay 2026.
The official OpenAI DevDay 2026 recap described the Decisions API as a way to focus Luna’s intelligence on a user-defined set of specific questions with finite predefined answers. Developers can provide text or image context and use the result for tasks such as content classification, request routing, or selecting an agent’s next action.
As of the September 29, 2026 announcement, it was in limited preview, with OpenAI saying broader release was planned within days.
That description immediately reminded me of what I had found useful about JEV.
The problem shape was similar at an abstract level:
open-ended generation → bounded questions → finite answers → application decision
There is no basis for claiming that OpenAI copied JEV, and the two products do not need to share the same internal architecture or goals.
What interests me is the abstraction rather than the product comparison.
Even as general-purpose LLMs improve, there is value in treating repeated, constrained judgments — classification, routing, next-action selection — as a separate engineering surface.
What I want from the Decisions API is not an accuracy claim
I cannot say from this experiment whether the Decisions API will be more accurate, faster, or cheaper than JEV.
I did not rerun these same 17 cases against the Decisions API. There is no reason to invent benchmark results that were not available from the preview.
What interests me more is the API abstraction.
When we build bounded classification with a general LLM, developers typically assemble prompts, allowed values, structured-output schemas, validation, retry behavior, boundary representation, and decision policy themselves.
Much of that work is plumbing required to make a model that is good at generating language behave like a decision component.
If the Decisions API makes finite questions and finite answers a first-class contract, the interface for expressing bounded judgment can become clearer.
That matters in enterprise workloads because the desired request is not always ‘write me a good answer.’
Sometimes the actual requirement is much narrower:
Read this context, answer only the small set of questions I allow, and stay inside the answers I allow.
Email routing, document triage, eligibility checks, approval routing, and agent next-action selection all fit that pattern well.
A Decision API does not make Business Policy disappear
One point still needs to remain explicit.
A decision-focused API does not remove the application’s deterministic policy.
If anything, a better decision interface can make the boundary easier to draw.
A model or Decisions API may own judgments such as which candidate best fits the current context, the answer to a finite question, semantic choice among options, and perhaps a useful uncertainty signal.
But the application still needs to own different responsibilities:
- the canonical taxonomy
- category definitions and policy version
- which signals qualify for AUTO versus REVIEW
- which categories require human approval
- which Salesforce fields may actually be changed
- retry and idempotency for the same request
- audit and human correction
- side-effect authority
That boundary kept appearing in both my JEV implementation and the general LLM version.
A stronger model did not make it disappear.
What lasted longer than the model was the Decision Surface
I started with a simple question: would JEV be a better fit for classification than an LLM?
After running the same 17 emails through both approaches, the question changed.
They agreed 17/17.
But JEV required question design, while the general LLM required context and evidence-schema design. Their cost and latency profiles were different. And when I reread the two cases where both engines disagreed with the existing labels, I was reminded that the ground truth itself, not only the model, can be an object of validation.
At almost the same moment, OpenAI announced a Decisions API that treats finite questions and finite answers as a separate surface.
So the question I now find more durable is not the product name.
What Decision Surface should contain the intelligence for this workflow?
The model can change.
It might be JEV. It might be a general LLM. It might eventually be a managed decision primitive such as the Decisions API.
But the work of making the permitted questions, answers, boundaries, escalations, and final authority explicit remains.
In my experience, that is often a more durable enterprise asset than the model-selection table.