ESL Engineering Software LabAI SDLCENGINEERING THE DECISION
ESL FIELD NOTES / 01
JEV · COST-AWARE AUTOMATION

The example behind the comment 01 / The stakes

99% sure.
Two different decisions.

Same probability.
Different consequences.
A probability is not permission.

01 / LOW-COST SERVICE CREDIT
$0.20

EXPECTED ERROR LOSS

Probability correct
99%
Loss if wrong
$20
Human review
$5
AUTOMATION CANDIDATE ↗
02 / HIGH-VALUE REIMBURSEMENT
$200

EXPECTED ERROR LOSS

Probability correct
99%
Loss if wrong
$20,000
Human review
$5
REVIEW FIRST ↗

Illustrative cases, not measured Jev performance. Assumes validated probabilities, negligible review error,
and mandatory policy checks already passed. All amounts in USD.

← Swipe to navigate →

02 / Change the stakes

The probability stays.
The decision moves.

Move the controls. Watch a cheap model decision become an expensive business mistake.

Expected error loss
= P(wrong) × loss if wrong
EXPECTED ERROR LOSS
$200.00
REVIEW FIRST

Educational model, not an authorization system. Assumes review resolves the uncertainty with negligible error. Equal costs route to review.

03 / Before the math earns your trust

Jev makes
judgment cheaper.
It does not make
mistakes cheaper.

Decision theory, applied to AI. A low API bill says nothing about the cost of a wrong action.

  • 01
    Validate probabilities on your workload.
    Check observed outcomes against predictions, including important subgroups and changing inputs.
  • 02
    Price the whole mistake.
    Include recovery, delay, customer harm and reviewer error, not only the face value of a transaction.
  • 03
    Keep hard policy outside the model.
    Authorization, legal requirements and mandatory reviews cannot be bought away by a favorable expected value.
This simplified comparison assumes negligible review error. In practice compare automatic-action loss with review cost plus residual expected loss after review.

04 / Code controls

Turn probability
into policy.

One small function. No agent framework. No payment execution. Just an explicit, testable routing recommendation.

PYTHON / NO DEPENDENCIESILLUSTRATIVE POLICY

Inputs: a validated probability and finite, nonnegative cost estimates. A policy veto or invalid probability routes to review. Equal expected costs also route to review.

“Automation candidate” is not authorization. Existing access controls and execution safeguards still apply.
decision_policy.py
def route(p_correct, loss_if_wrong, review_cost, *, policy_ok):
    if not policy_ok or not 0 <= p_correct <= 1:
        return "REVIEW"

    expected_loss = (1 - p_correct) * loss_if_wrong
    return (
        "AUTO_CANDIDATE"
        if expected_loss < review_cost
        else "REVIEW"
    )

print(route(0.99, 20, 5, policy_ok=True))
print(route(0.99, 20_000, 5, policy_ok=True))

# Equal costs? Review. Policy veto? Review.
assert route(0.75, 20, 5, policy_ok=True) == "REVIEW"
assert route(1.0, 20, 5, policy_ok=False) == "REVIEW"
$20 case     → AUTO_CANDIDATE
$20,000 case → REVIEW

05 / The subtle Jev trap

confidence
is not P(correct).

TypeSafe describes confidence as a statistic derived from the shape of the answer's probability distribution. Do not use it interchangeably with the probability of the selected outcome.

Type-safe.Not necessarily truth-safe.

A perfectly valid enum can still select the wrong action.

Inside an existing Jev Choice response
answer = response.answers["department"]

# Probability of the selected classification:
p_selected = answer.probabilities[answer.choice]

# Not an interchangeable substitute:
# p_selected = answer.confidence
  • !
    A classification is not an entire action.
    The probability of “billing” is not the probability that issuing a refund is safe.
  • !
    Calibration still needs evidence.
    Validate against labeled workflow examples before assigning economic meaning to the number.

06 / Take the idea with you

Same model.
Same probability.
Different decision.

The benchmark worth asking for: how much can we automate within a fixed error-loss budget?

Count review and recovery.
Not just milliseconds per call.

Optimize cost per correct outcome, not cost per call.

Download the infographic ↓

An illustrative comparison, not a Jev benchmark.

Two cases with 99 percent probability correct: 20 dollars loss if wrong gives 20 cents expected error loss; 20,000 dollars gives 200 dollars. Review costs 5 dollars in both cases.

07 / ESL AI SDLC

Make the judgment fast.
Make the policy explicit.

Bring this question to your next AI workflow review: what does a wrong decision cost, and what evidence justifies automation?

Daniel Liezrowicedaniel.l@eswlab.com ↗+972 9 8855803https://eswlab.com/ ↗

Engineering Software Lab · Ha-Nagar 24 A, Hod Hasharon, Israel

Read the benchmark caveats. TypeSafe says its headline 193.6× speed and 444.6× cost gains are at the high end of expected real-world gains. Its launch workflow evaluations use frontier-model predictions as reference answers, not independently established ground truth.
SOURCES / CHECKED 22 SEPTEMBER 2026https://craftbettersoftware.com/p/jev-engineering-101https://typesafe.ai/blog/introducing-system-one-models-and-jevhttps://docs.typesafe.ai/confidence

Independent ESL educational commentary. No endorsement by TypeSafe implied. The calculator runs locally in your browser; it does not call an AI service or submit your inputs.