# Composition patterns

Questions in one request never see each other's answers. They run independently against the same state; parallelism improves wall time, not the token bill or shared request budget. Maximize independent questions that add decision or action value, subject to their question-token cost and the 64,000-token request budget. Compose in code.

## Pattern index

| Pattern | When to use | Sketch | Dissolve difficulty unlocked |
|---|---|---|---|
| Speculative fan-out | Multiple branches share state | all heads in one call; code ignores unused | multi-branch skills with shared evidence |
| Second request | First answer decides what data to fetch | call 1 picks, call 2 judges the picked data | two-phase skills (detect then act) |
| Confidence-gated routing | Action cost varies by stake | floor, act, confirm, hand-off paths | skills with escalation logic |
| Composite scoring | Multiple quality dimensions | one Score per dimension, weights in code | multi-rubric review skills |
| Intent routing | Classify then route | Choice + complexity Score, both gated | router/dispatcher skills |
| Taxonomy walk | Hierarchical classification | one Choice per tree level, follow in code | category-heavy triage skills |
| Choice + Nouls | Relative pick needs absolute check | Choice picks, per-option Nouls confirm | skills that must reject all options |
| Multi-Noul decomposition | Compound goal check | one Noul per clause, combine in code | phased skills with compound gates |
| Cascade + verification | Wide scan, narrow confirm | cheap pass proposes, second call verifies | review/audit skills |
| Deterministic pre-filter | Obvious ends + ambiguous middle | programs decide extremes, Jev judges rest | skills with regex/grep pre-phases |
| History injection | Sequential actions, loop risk | recent actions as do-not-repeat in state | interactive/browser skills |
| Label over index | DOM or list selection | pick by meaning, not position | UI-driven skills |
| Classify before acting | Unknown initial state | feasibility Noul on first observation | skills that gate on preconditions |

## Speculative fan-out

Ask every question the code might need in one request, including questions that matter only on some branches. Questions run in parallel, but each question's instructions and criteria still add billed tokens and consume shared request budget. Batch independent heads that can change a decision or action; omit low-value or noise heads. Do not batch a head that requires another answer to select evidence, construct candidates, or define its criterion. Measure the cost, wall time, and answer agreement of batching against serial sends on the target workload before relying on it.

```python
answers = call(state, {"category": choice, "bug_severity": score, "refund_requested": noul})
if answers["category"]["choice"] == "bug_report" and is_actionable_severity(answers["bug_severity"]): escalate()
elif answers["category"]["choice"] == "billing" and passes_refund_policy(answers["refund_requested"]): billing(refund=True)
```

For example, a browser program can send operation-type Choices beside readiness Nouls, then execute only the answers its deterministic policy needs.

## Second request only when the first answer decides the data

Make a second call only when code cannot build it without the first answer: the answer picks what to fetch, what the state is made of, or which options the next Choice offers. Examples include ranking candidates before re-judging a shortlist against full text, classifying blocks produced by a first-pass merge, and hierarchical classification where each answer selects the next options. If the second call's questions could have run against the original state, put them in the first call. The code—not an implied dependency between heads—must build and label the new state.

## Confidence-gated routing

The answer says what; confidence says whether to act. Three paths: act, confirm or flag, hand off.

```python
if intent["confidence"] < HANDOFF_CONFIDENCE: hand_off()      # floor
elif intent["choice"] == "check_balance": act()                # low stakes
elif intent["choice"] == "approve_transfer":
    act() if intent["confidence"] > HIGH_STAKES_CONFIDENCE else confirm()
```

Select thresholds on labeled data and the cost of a wrong call; one system may need several. Gate a Score's confidence too, not only the Choice's. Keep thresholds as named policy constants and report their calibration.

Treat low confidence on a non-terminal browser action as a reason to hold the pick, re-observe, and count it toward the stall guard. Set the cutoff from labeled data and action costs. Terminal claims skip this gate only when an independent verification gate controls them.

## Composite scoring

One Score per dimension, normalized by `len(criteria) - 1`, combined with weights in code.

```python
def normalized(answers, qid): return answers[qid]["score"] / (len(QUESTIONS[qid]["criteria"]) - 1)
priority = 0.6 * normalized(a, "severity") + 0.3 * normalized(a, "frustration") + 0.1 * normalized(a, "report_quality")
```

Change a weight when priorities shift; do not rewrite a question to change policy. Different role profiles are different weight vectors over the same answers.

## Intent routing

A Choice for intent beside a complexity Score can route each intent to deterministic code, a specialist LLM, or a person; send low-confidence classifications and high-complexity edge cases to a person. A wide pass may shortlist candidates for a detailed rerank with `not_for` and per-candidate fit Nouls. Log the full distribution: a near-tie between the winner and runner-up is a close call that the pick alone hides.

## Taxonomy walk

One Choice per tree level; walk in code. Each option's criteria value is its subtree, so Jev sees what lives under a branch before committing. Trim large subtrees to direct children and a sample of leaves. When probabilities are close, follow several branches (beam) and let a later level or a per-leaf Noul settle it. Two calls for a two-level tree, not one call per leaf.

## Choice picks, Nouls decide whether to pick

A Choice is relative: something always wins. Add one Noul per shortlisted option (absolute: "does this genuinely fit?") in the same call. Act on the Choice only when its winner's Noul also passes.

## Multi-Noul decomposition

Split a compound goal into one Noul per clause. Programs split, Jev judges each, programs combine. Compare decomposed and compound forms on labeled cases; keep the decomposition only when it improves the action or quality measure. When an answer looks wrong, the question may be too big.

## Cascade plus verification

One wide request per unit proposes; code thresholds pick the survivors; a second request verifies them and carries only evidence the first lacked. Only confirmed findings reach the report. Add deterministic post-checks beside Jev: does the file exist, does the test pass, does the build succeed. A model's claim of completion is not evidence of completion.

Severity must spread. If findings collapse into a few indistinguishable levels, a ranking is unusable. Define Score levels as distinct situations and, when the evidence requires it, use a verification-stage severity judgment to produce a ranking someone can act on.

## Bounded residual review

Use a stronger reviewer only after code and Jev have handled the obvious units, and only for a declared referral band. This is an exception to the normal rule against LLM review, not a second general-purpose judge. The runner gives the reviewer the immutable, source-bound evidence bundle Jev saw (plus explicitly declared new evidence if one was fetched), pins a fixed answer schema, and keeps the reviewer from drafting explanations or changing the evidence.

Before shipping, compare Jev-only and Jev-plus-review on a held-out split. Report: referral rate, accuracy/recall and false-positive lift on both all units and referred units, incremental cost per input KB and per corrected unit, p50/p95 added wall time, accumulated reviewer call time, timeouts, and declined referrals. Cap per-call spend, output, retries, and deadline in code. Keep the stage only when the marginal lift is worth those limits; otherwise improve Jev evidence or criteria instead.

The decision definition stays model-neutral: code creates evidence bundles and source IDs, the runner selects the model, and downstream policy consumes the same typed answer. Persist each attempt and final decision so a later model swap or audit can replay the exact workload.

## Deterministic pre-filters and per-class thresholds

Programs decide the obvious ends; Jev judges the middle. In compaction, deterministic result classes may always drop or keep when representative evidence supports those rules, and Jev judges the rest. Per-class policy thresholds can differ. A `referenced` Noul ("was this result cited later?") can boost keep probability in code. Missing answers default to keep.

## History injection and stall detection

Put a bounded action history in state as a plain list (`actions_already_taken`). Put the rule ("do not repeat an action listed in `actions_already_taken`") in the instructions: state holds facts, instructions hold the judgment. This keeps Jev from choosing the same action again. Fingerprint each observed state (hash of role:label:value); an unchanged fingerprint after an action means no effect, and repeated fingerprints are a cycle. Both end the loop in code with policy-defined limits. When an action had no effect and confidence was low, retry with the runner-up from `probabilities` rather than the same pick.

## Label over index

Jev picks by meaning. `link:Dashboard` is stable across re-renders; `element_3` breaks when the DOM shifts. Expand each `<select>` option as its own Choice entry (`select_option:Region:US-East`) and map back in code.

## Classify before the first action

A `goal_feasibility` Noul on the first observation catches error pages, login walls, wrong domains, and empty states before the loop spends steps. Informational Nouls (`has_real_content`, `interactive_elements_work`) run in the same verify call and let the program abort early.

## Decision and generation are different tiers

When a Jev decision needs generated text (commit message, field value), Jev decides what; the cheapest adequate model writes the text. Reserve large models for diagnosis and synthesis.

## Inventing a new pattern

Start from the failure: what did Jev get wrong, or what could it not see?

1. **Name the failure.** Wrong answer, low confidence, missed case, loop, or latency.
2. **Change one of four levers.** State shape (what evidence Jev sees), question decomposition (how many questions and of what type), call sequencing (one call vs. chained calls), or policy (thresholds and weights in code).
3. **Measure against labeled cases.** Run the new pattern on the same labeled set. Compare accuracy, confidence distribution, and false-positive rate.
4. **Keep only with evidence.** A pattern that reads well but does not improve a measured decision, cost, or quality outcome against its baseline is not a pattern. Record the before/after evidence alongside the labeled cases.

# Composition positions

A Jev judgment occupies one of 11 positions relative to a function F.

| # | Position | Form | Governing rule | Script |
|---|----------|------|----------------|--------|
| 1 | Operand | `F(Jev(...))` — probability feeds F | Version question defs with the consumer; it is a belief, not a natural quantity | none |
| 2 | Post-judge | `F(x) -> Jev judges result` | Only the post-judge sees what the call printed; a pre-gate cannot | `jev-browser-verify.py` |
| 3 | Gate | `if Jev(x): apply F` | A gate is a filter, not authorization; validate operation+target in code; every error path fails open | none in this repo yet |
| 4 | Selector | Jev picks which F runs | Dispatch stays in code; per-option consequences are your cost model; confidence-gate the selection | `jev-route.py` |
| 5 | Comparator | Semantic sort key inside rank/sort | Comparability needs a shared rubric; measure recall separately from rerank quality | none in this repo yet |
| 6 | Prior | Jev seeds a deterministic refinement | It is a heuristic prior, not a posterior; refine with real observations | none |
| 7 | State-estimator | Jev estimates probs; deterministic policy acts | The model never commands; interventions are enumerated in code | none in this repo yet |
| 8 | Metric | Jev as judge inside an optimizer loop | Optimizer metrics must be repeatable; verify judge variance on your data | `jev-harness.py` |
| 9 | Verifier | Jev judges spec-conformance | Verdicts are evidence, not enforcement; the checker enumerates requirements in code | `jev-browser-verify.py` |
| 10 | Discretizer | Unstructured state -> typed values code requires | Jev selects from candidates you produce; it never generates | `jev-browser-decide.py` |
| 11 | Bounds/budget | Jev decides how far to continue | Termination conditions stay conservative and code-owned | `jev-compact.py` |

## Logical operators

- **Negation**: `p(not X) = 1 - p(X)`. Ask the question in the form you branch on; double negatives cost accuracy.
- **AND / OR over parallel Nouls**: combine in code with an explicit labeled policy. Never multiply — same-state answers are not independent.
- **FOR-ALL / EXISTS**: batch one Noul per item in one request, then aggregate in code (`min` = AND, `max` = EXISTS) with an explicit escalation rule.
- **Chains**: decompose into gate -> act -> post-judge. Never encode multi-hop logic in one question.

## Walk the positions (dissolving a skill phase)

1. Pick the construct (operator, algorithm, or workflow step) that the skill phase wraps.
2. Walk positions 1-11. For each: is there a judgment-shaped hole? Most are trivially no.
3. For each lit cell, apply the economics inversion: was this step previously infeasible because a judgment cost seconds and cents? If yes, it is a newly-feasible candidate.
4. Name the governing rule from the table and enforce it in code. No nameable rule means the cell is a metaphor.
5. Falsify: labeled cases, split A/B, judge-variance check. A candidate that survives becomes a catalog row.

# Decision card

Complete one card per gate or judgment before writing code. Each field
constrains the design; an empty field is a gap, not an option.

## Fields

1. **Desired behavior.** State the decision, the unit it applies to, and what
   happens when the system acts correctly. Name a non-Jev baseline (majority
   class, strongest single signal, existing rule) and its measured accuracy.
2. **Judgments.** List every Jev question, its primitive, and what each returned
   number means for this domain. Define "high," "low," and "ambiguous" with
   concrete examples.
3. **Evidence source.** Name the data the state draws from, its refresh rate,
   and any coverage gaps (missing labels, skewed slices, stale sources). State
   which gaps the design tolerates and which would invalidate it.
4. **Deterministic policy.** Write the pure function `policy(answers) -> action`
   with named thresholds. State which code path owns each action and who is
   accountable for the outcome.
5. **Batchable vs dependent steps.** Mark each question as batchable (runs in
   the same request, no data dependency) or dependent (needs a prior answer
   before it can be asked). Dependent steps add a second request; justify each.
6. **Failure and abstention.** Define behavior when Jev is unavailable, when
   confidence is below the gate, and when the response is malformed. State
   whether the gate fails open, fails to warn, or fails to block.
7. **Falsifying experiment.** Describe the smallest labeled test that could
   reject this design: the split, the metric, and the threshold below which
   the gate is not worth shipping.
8. **Versions.** Pin the Jev model id, the rubric version (date or hash), and
   the policy version. State what triggers re-evaluation: a model upgrade, a
   rubric edit, a data distribution shift, or a policy change.
9. **Evaluation integrity.** Record fixture and rubric hashes, human/provisional
   label provenance, and an untouched group-disjoint heldout. State
   that unavailable or invalid answers are `unknown` with separate failure
   receipts, not quality scores. Reserve the heldout for one post-selection run.
10. **Action safety.** For any action-changing selector, define shadow-mode
    evidence required before enforcement. Name the source-evidence ledger and
    constraints such as permissions that confidence cannot override.

## Usage

Fill the card in the ADR, design doc, or code comment before the first Jev
call ships. Review the card when any version field changes. A gate that ships
without a card has no defined failure behavior and no falsifying experiment
-- it cannot be evaluated or retired.

# Dissolving a skill into a Jev program

## Method

1. List every phase of the skill.
2. Classify each: program (deterministic), Jev (judgment), LLM (generation), orchestration.
3. Judgment phases become Noul/Choice/Score questions with hard-case criteria.
4. Deterministic phases stay as code.
5. Any generation phase keeps an LLM that receives Jev's decisions as `prior_results`.
6. Write a pure policy function with named thresholds. This is the dissolved skill's contract.
7. Prove agreement on the skill's labeled cases before deleting the SKILL.md.

## Worked example: joy-check

**Why this skill.** joy-check has four phases. Three are fully deterministic or judgment. Only the `--fix` rewrite in Phase 3 requires generation. Without `--fix`, the entire skill dissolves into a program plus Jev.

### Phase table

| Phase | Current tier | Dissolved tier | Reason |
|---|---|---|---|
| 0: Detect mode | Program | Program | file path matching, flag parsing |
| 1: Pre-filter | Program | Program | regex/grep scan |
| 2: Analyze | LLM | Jev | score each item against rubric dimensions |
| 3: Report (scoring) | LLM | Program + Jev | average scores (program), classify overall (Jev) |
| 3: Report (fix mode) | LLM | LLM | rewriting flagged items requires generation |

### Jev question set

Phase 2 currently uses an LLM to evaluate each paragraph or instruction against a rubric. The rubric has named dimensions. Each dimension becomes a Jev question.

**Writing mode** (per paragraph):

| Question ID | Type | Instruction |
|---|---|---|
| `curiosity` | Noul | Does the paragraph frame the experience through curiosity or exploration? |
| `grievance` | Noul | Does the paragraph frame the experience through accusation, blame, or resentment? |
| `defensive` | Noul | Does the paragraph contain defensive disclaimers or reluctant generosity? |
| `overall_joy` | Score | How does this paragraph sit on the joy-grievance spectrum? |

**Instruction mode** (per instruction):

| Question ID | Type | Instruction |
|---|---|---|
| `positive_frame` | Noul | Does the instruction tell the reader what to do (not what to avoid)? |
| `prohibition` | Noul | Is the instruction framed as a prohibition (NEVER, do NOT, must NOT, FORBIDDEN)? |
| `subordinate_neg` | Noul | Is a negative clause subordinate to a positive instruction (e.g., "use X, not Y")? |

### Policy function

```python
# Named thresholds — the dissolved skill's contract
GRIEVANCE_T, CURIOSITY_FLOOR = 0.6, 0.3
PROHIBITION_T, SUBORDINATE_PASS = 0.7, 0.6
OVERALL_PASS, STRICT_FLOOR = 60, 60

def score_item_writing(a: dict) -> dict:
    """Score one paragraph in writing mode. Pure function."""
    g, c, d = a["grievance"]["noul"], a["curiosity"]["noul"], a["defensive"]["noul"]
    normalized = round((1 - a["overall_joy"]["score"] / 2) * 100)
    flagged = g > GRIEVANCE_T or c < CURIOSITY_FLOOR or d > GRIEVANCE_T
    label = "GRIEVANCE" if g > GRIEVANCE_T else ("CAUTION" if flagged else "JOY")
    return {"score": normalized, "label": label, "flagged": flagged}

def score_item_instruction(a: dict) -> dict:
    """Score one instruction in instruction mode. Pure function."""
    pr, pos, sub = a["prohibition"]["noul"], a["positive_frame"]["noul"], a["subordinate_neg"]["noul"]
    if pr > PROHIBITION_T and sub > SUBORDINATE_PASS:
        return {"score": 85, "label": "PASS", "flagged": False}
    if pr > PROHIBITION_T:
        return {"score": 20, "label": "NEGATIVE", "flagged": True}
    s = round(pos * 100)
    return {"score": s, "label": "PASS" if s >= 60 else "CAUTION", "flagged": s < 60}

def policy(items: list[dict], strict: bool = False) -> dict:
    """Overall pass/fail. Pure function, no Jev calls."""
    scores = [r["score"] for r in items]
    avg = round(sum(scores) / len(scores)) if scores else 0
    ok = avg >= OVERALL_PASS and not any(r["label"] == "GRIEVANCE" for r in items)
    if strict:
        ok = ok and all(r["score"] >= STRICT_FLOOR for r in items)
    return {"score": avg, "passed": ok, "items": items}
```

### What stays in the LLM

Only `--fix` mode: rewriting a flagged paragraph to shift its framing while preserving substance. The LLM receives `prior_results` (which items were flagged, why, at what score) and rewrites only those items. It does not re-judge.

### Proving agreement

Run the dissolved program on joy-check's existing labeled cases (rubric examples in `references/writing-rubric.md` and `references/instruction-rubric.md`). Compare each item's score and label against the rubric's expected classification. The dissolved version ships only when agreement matches or exceeds the LLM-based version on those cases.

# Improve and calibrate

Find the failing question before changing anything. Collect labeled examples, run them, and compare each question's answers and probabilities against the labels. Rewording on intuition trades one error for another.

## Symptom table

| Symptom | Likely cause | Fix |
|---|---|---|
| Wrong answers with high confidence | literal reading of the instruction | state the exact condition; put the boundary case in criteria |
| Low confidence on a Choice | options overlap, or none fits | add `what`, `not_for`, `examples`; add `other` |
| Low confidence on a Score | levels overlap, two dimensions, or thin state | rewrite levels as distinct situations; split; add the missing field |
| Scores cluster in the middle | levels are degrees or numerals | one concrete situation per level; remove numerals |
| Top-of-scale cases look alike | the extreme has no level | add a level for it |
| Noul hovers near 0.5 | vague condition | define it; add `true`/`false` criteria with examples |
| Accuracy falls as inputs grow | irrelevant state | filter in code; send only needed fields; judge per unit |
| Errors on counts, sums, dates, nearness | Jev doing arithmetic | move it to code; per-item Nouls; Choice per date part |
| Errors on nested or negated questions | indirection | ask directly; name the path; two literal questions combined in code |
| Answer follows text inside the state | state steering | tighten criteria; adversarial and self-reference tests; confidence gate |
| Rewording one question trades errors | one question weighs several properties | split into atomic questions |
| Each answer right, decision wrong | policy | change weights or thresholds in code; leave questions alone |
| Slow or costly | sequential calls | merge into one request; keep a second only when it depends on the first |
| Findings unverifiable | whole-input judgment | judge per file or element; anchor to file:line |
| Severity all one band | levels not situations | rewrite levels; add a second-opinion Score in verification |
| `max_tokens_exceeded` mid-pipeline | a later stage skipped fitting | cap items per call at every stage; count `calls_failed` per stage |
| `calls_failed` nonzero, output looks fine | defaults masquerading as answers | surface failed calls; never default silently on a safety path |

## Revision rules

- Change one or two questions per revision. Probabilities shift in ways that are hard to predict; leave questions that discriminate well alone.
- Judge on labeled data. Higher confidence alone does not show a better question; two wordings of one scale behave differently on your data.
- Keep the answer space stable once code depends on it. Adding or removing a level or option changes what every earlier answer meant and invalidates stored calibration.
- General rules in instructions and criteria; specific names and values only in `examples`.
- Do not loosen criteria to remove a false positive. Sharpen the boundary by adding the misjudged case to `not_for` or the `false` side.
- Pin the model version when thresholds are tuned on measured behavior. Rerun the labeled set and reread the jaggedness page when the model changes.
- Treat state as a design lever only when it supplies evidence needed by the decision. More context is not a fix. Change one state, instruction, criterion, or policy lever at a time so a result remains attributable.
- Do not use a generic probability label, bundled cues, or a list of loosely related tells as a decision surface. Ask a narrow, observable question with `not_for` or restraint cases that protect valid exceptions, quoted input, domain-specific values, and already-supported conclusions.

## Labeled-example loop

1. Build a set with known labels, including adversarial and self-describing inputs and the cases that were misjudged in production.
2. Run every question; store answers with full `probabilities`.
3. For each miss, read the distribution: a near-tie means overlap; a confident miss means literal reading or a missing condition.
4. Revise one or two questions. Rerun the whole set, not just the miss.
5. Keep the set and the results under version control or in the assessment store so the next model version can be compared.

## Labels, holdouts, and failures

Hash frozen fixtures and the rubric that produced their labels. Record label provenance. Human-confirmed labels may evaluate or promote an action; agent or provisional labels are useful for finding cases but remain diagnostic until confirmed.

Keep one untouched, group-disjoint heldout split out of selection and tuning. Use it once for the final report. If work resumes after seeing it, create a new independent heldout before making another claim. This prevents repeated inspection from turning a test result into another dev signal.

Treat an unavailable service, malformed response, missing answer, or failed validation as `unknown`, not a negative label or passing score. Store a failure receipt (request identity, stage, error, timestamps, retry outcome) separately from quality metrics. Never let missingness improve an accuracy, pass, or action-rate metric.

For any action-changing selector, run the policy in shadow mode until human-confirmed, disjoint-heldout results show that the action helps. Confidence cannot authorize an action or override source evidence, permissions, or deterministic policy. Evidence quality is evaluated against a supplied source-evidence ledger; plausibility and stated intent do not substitute for that evidence.

`scripts/jev-compact-evidence.py` shows the measurement side: read the engine's own records, not the tool's claims.

## Reading `score`

Jev's `score` is the probability-weighted mean of 0-based level numbers. Adding an offset or rounding is a bucketing step: say so in the code, and read `probabilities` when the decision hinges on which level won. Label levels with situations, and keep the 0-based numbering out of user-facing text.

# Integration lifecycle

## Three tiers

| Tier | Owner | Work |
|---|---|---|
| 1 | programs | regex, counting, parsing, tests, builds, fingerprints, budgets |
| 2 | Jev | classification, scoring, triage, gates: pick one, score this, yes or no |
| 3 | LLM | code, prose, diagnosis, synthesis |

Apply the lowest tier that can do the job. Jev runs after programs, not instead of them: a program extracts or computes the available evidence, then Jev judges the residual. The LLM receives tier 1 and 2 findings as input and does not re-judge them, except in the bounded, benchmarked residual-review stage. Jev pricing and latency are workload and model dependent; measure them from actual runs rather than relying on a universal promise. An LLM call commonly costs more and can rationalize a wrong answer.

## Hook points

Start every program as an on-demand command. A hook multiplies cost by its firing rate: every firing bills the state plus every question's text. Measure the event rate in the target environment before promotion. Promote a program to a hook when all four hold:

1. Code decides the obvious cases first; Jev receives the residual.
2. A labeled set shows the answers are right.
3. The finding rate shows the answers carry information: when answers are overwhelmingly constant on representative data, measure whether a deterministic rule can replace the call.
4. Something acts on the answer, and a log shows how often the answer changed the action.

Promote one hook at a time and read `scripts/jev-cost-report.py` after a day of use.

| Event | Firing rate | Fits when |
|---|---|---|
| UserPromptSubmit | once per request | the answer changes how the request is handled (routing) |
| session.compact (plugin) | at a context threshold | Jev replaces messages directly |
| PreToolUse, PostToolUse | every tool call | code has already narrowed the input to a rare, undecided case |
| SubagentStop, Stop | every agent or reply | a later step reads the stored finding and acts on it |

Registered events live in `.claude/settings.json`. Hooks exit 0 on their own errors and print a warning rather than fail the tool.

## Reader, storage, action

Every integration needs all three:

1. Reader: a hook or script that runs Jev on the evidence and extracts structured findings.
2. Storage: findings persist where something reads them (state file, injection, return value). Findings only in stderr are invisible.
3. Action: something changes behavior (retry, block, stop iterating, inject).

A SubagentStop hook that writes to a state file nobody reads has no action. When the action cannot happen at the same hook point, bridge it: a later hook reads the state file and injects at a point that supports injection. A PreToolUse secret scan has all three in one step.

## Thread prior assessments

Each tool's result flows into the next call's state as bounded, labeled evidence: the completion validator sees that scope-creep flagged three files; the quality loop sees that the slop scan found hedges. Extract actionable fields (which dimensions fired, at what confidence, the aggregate), not the full prior JSON.

## Jev assesses, policy decides

A pure function `policy(state, assessment) -> action` reads probabilities and returns block, warn, allow, retry, or escalate. It never calls Jev. Threshold constants live in the policy, so a threshold change is a unit test without a mocked API. Keep the call and the policy in different functions, ideally different modules.

## Failure modes

| Check type | When Jev is unavailable or the response is invalid |
|---|---|
| advisory (commit readiness, scope creep, output triage) | fail open: skip silently |
| safety (secret scan, breaking change) | fail to warn: inject "Jev safety check unavailable, manual review recommended" |
| blocking (secret scan at severity >= 4) | fail to warn, never fail to block: an unavailable Jev must not stop work |

Validate every response with `validate_jev_response` between the call and the policy.

Validation failure, a missing answer, or an unavailable service is `unknown`, not `no`, pass, or a zero score. Persist a failure receipt separately from assessment metrics: stage, request/evidence identity, error, timestamps, and retry outcome. The policy must branch explicitly on `unknown`; it must never silently default it into a quality or action result.

Availability check: `typesafe_available()` requires both `TYPESAFE_API_KEY` and the enabled plugin (`JEV_KEY_ONLY=1` for standalone CLI use). Never log the key or the Authorization header.

## Persist for calibration

Append each assessment's key fields (tool, evidence-bundle ID/version and source IDs, question values and version, full distributions, verdict, model/version, attempt/retry fields, deadline/cap, wall latency, model-call duration, usage, timestamp) to a bounded, rotating store: a session-scoped JSONL pruned after 24 hours, or `learning.db`. Persist the request and response references where policy permits. This enables false-positive measurement, threshold tuning on real data, drift detection when the model changes, replay, and A/B comparison of question versions. Log full Choice distributions, not just the pick.

For a high-throughput runner, keep two clocks: wall time for user-visible completion and accumulated model call time, the sum of each attempt's duration. Report throughput against a named clock (`rows/wall-second`, `KB/call-second`); concurrency can make wall time low while accumulated call time remains high. Record keep, refer, decline, retry, timeout, and final-action rates beside cost, so a low average price cannot hide an expensive referral tail.

Claims are not evidence; the engine's record is. A plugin log line says what the plugin believes it did; the transcript's `compact_boundary` row (`preTokens`, `postTokens`, `durationMs`) says what happened. Record both and show them side by side (`scripts/jev-compact-evidence.py`).

## Compaction specifics

- Compaction is deletion, not generation: Jev judges each tool call (referenced later? input still constraining? reproducible by re-running?), stale items are deleted verbatim, nothing is rewritten.
- Threshold-gate; do not fire every turn. The Jev call may be inexpensive relative to compaction, and editing a cached prefix can make the next request a cold write. Select a context threshold from measured workload cost and behavior.
- Use two stages when measurement supports it: Jev prunes while there is slack; the built-in summarizer runs when Jev alone cannot get under the line. Measure retained tokens and downstream quality for both stages on the target workload.
- A self-triggered compaction (`event.trigger == "plugin"`) that finds nothing to prune returns skip; on `auto` or `manual`, let core fall back.
- The function-hook plugin owns compaction for the main session.

## Deployment checklist

- [ ] Reader, storage, action all present; the action is observable.
- [ ] Policy is a pure function with named thresholds.
- [ ] Failure mode matches the check type.
- [ ] Response validated before the policy runs.
- [ ] Assessments persisted with distributions and usage.
- [ ] Live API call in the test run, not only mocked responses.
- [ ] Enforcement tested on its failure path, not assumed from registration.
- [ ] Unknown answers and failure receipts are excluded from quality/pass metrics and handled by an explicit policy branch.

# Primitives: API shape and answer semantics

Endpoint: `POST https://api.typesafe.ai/v1/systemone`, bearer auth. Python SDK: `client.system_one(state=..., questions=...)`. Stdlib client: `scripts/jev_router_common.py` (`call_jev`, `validated_call_jev`, `validate_jev_response`, `extract_usage`, `bound_text`).

## Request

```json
{"model": "jev-latest",
 "state": {"request": "...", "diff": "...", "prior": {"scope_flags": 3}},
 "questions": {
   "in_scope":  {"type": "noul",   "instructions": "...", "criteria": {"true": "...", "false": "..."}},
   "category":  {"type": "choice", "instructions": "...", "criteria": {"a": "...", "b": "...", "other": "..."}},
   "severity":  {"type": "score",  "instructions": "...", "criteria": ["level 0", "level 1", "level 2"]}}}
```

- `state`: string or JSON. Every question sees the same state. Structure it so questions can point into it by path.
- Question ID: your key. It is not sent to the model; write the full question in `instructions`.
- `model`: the shared client calls `jev-latest`. Pin the version when a calibration set or threshold depends on measured behavior. When the model changes, rerun the labeled set and reread the jaggedness page.
- Budget: A request holds 64,000 tokens: the state plus every question. The state plus the longest single question must stay under 32,000. Check the [models page](https://docs.typesafe.ai/models.md) for current limits.
- Response carries `usage.input_tokens` and `usage.output_tokens`; `extract_usage` passes them through.

## Structured fields

`instructions`, Choice option values, Score level entries, and Noul `criteria.true`/`criteria.false` each accept a string, an object, an array, or `null`. Use an object when a question has labeled parts or needs supporting data. Pass a schema, taxonomy, or row as JSON; never serialize it into a string.

Instruction object keys the docs use:

| Key | Meaning |
|---|---|
| `question` | the question text |
| `focus` | what to weigh, what to ignore |
| `inspect` | the state path to judge |
| `note` | a scoping remark ("judge the number of changes, not their size") |
| `compare` | list of state paths to compare |
| `field` | `{name, type, unit, description}` shared by several questions about one field |

```json
"instructions": {"question": "Does `message` ask the recipient to disclose a credential?",
                 "inspect": "message",
                 "focus": "A request to send the credential itself, not to reset it."}
```

Choice option object: `{"what": ..., "not_for": ..., "examples": [...]}`. Option value `null` is allowed: a bare candidate list for extraction, where code generated the candidates and Jev picks.

Score level object: `{"summary": ..., "signals": [...]}` or `{"what": ..., "examples": [...]}`. Use the same keys on every level so the model compares like with like.

Noul criteria: optional. Add `true` and `false` sides with `what` and `examples` when the boundary is subtle. Put the neighboring case on the side it belongs to.

Schema strictness: keys outside the documented set (`options`, `min`, `max`, nested `what` at the Choice top level) return HTTP 422 or are ignored. A wrong schema can fall back to defaults silently, and a mocked test passes anyway. Validate against the live API.

## Answers

| Type | Fields | Read it as |
|---|---|---|
| Noul | `noul` | probability of yes. Near 1 yes, near 0 no, 0.5 unsure. No `confidence`; distance from 0.5 is the signal. |
| Choice | `choice`, `probabilities`, `confidence` | `choice` is the argmax. `probabilities` covers every option. `confidence` is how peaked the distribution is. |
| Score | `score`, `legend`, `probabilities`, `confidence` | `score` is the probability-weighted mean of level numbers. `legend` maps level number to description. |

Score example: levels 0, 1, 2 with probabilities 0.0, 0.7, 0.3 give `score` 1.3 and `confidence` 0.54. Level numbers start at 0 and follow array order. The API keys `probabilities` and `legend` by string ("0"); the Python SDK keys them by integer.

Reading `score`:
- It is a position, not a picked level. A 1.0 can be all weight on level 1 or a 0/2 split. Read `probabilities` when the distinction changes the action.
- Threshold it, rank by it, or round it to the nearest level for one outcome. Say in code that rounding is a bucketing step.
- Never interpolate a quantity from it. Levels are weakly calibrated as numbers. "Score 1.3 means 30% of users lack a workaround" is false.
- Scale offsets: adding 1.0 to map a 0-based mean onto labels "1..5" is a bucketing step. Say so in the code, and call the value a mean.

Reading `confidence`:
- A statistic of the distribution, computed for you. Compute your own from `probabilities` when a different measure suits the decision (entropy, top-2 margin).
- It describes the answer, not its correctness. Confidence 1.0 on a wrong answer happens under literal reading.
- Low Choice confidence: options overlap or none fits. Low Score confidence: levels overlap, the question measures two things, or the state says too little.

## Independence and invariants

Every answer is constrained to the options you supplied; code never parses prose. Every answer is independent: adding or removing a question does not change the others. Jev is consistent: similar inputs give similar outputs, so a labeled set stays meaningful across runs.

Structural identities you might expect are not guaranteed:
- A Noul and a yes/no Choice on the same question can return different numbers. Do not carry a threshold from one primitive to the other.
- Do not assume `P(noul)` and `1 - P(not noul)` sum to 1; test the primitive and wording you will deploy.
- A Choice is relative (which option wins); a Noul is absolute (it can be low for every option). Use both on a shortlist: the Choice picks, per-option Nouls decide whether to pick at all.

Validate before acting: `validate_jev_response` checks Noul in [0,1], Choice is a string, Score is numeric, probabilities in [0,1] summing to about 1, confidence in [0,1]. Advisory callers log and continue; safety callers treat an invalid response as unavailable.

# Question design

A good Jev question is one a knowledgeable person answers in a second given the right context. If the question needs steps, split it and compose in code.

## Instructions

| Rule | Weak | Strong |
|---|---|---|
| State the exact condition | "Is this candidate strong in Python?" | "Does the resume state the candidate used Python at work?" |
| One property per question | "Is the PR small and safe?" | `is_small` Noul and `is_safe` Noul |
| Name the state path | "Does the message ask for a refund?" | "Does `ticket.messages[0].text` ask for a refund?" |
| No double negatives | "Is it not the case that no tests were added?" | "Were tests added for the edited code?" |
| No property of a property | "Is the author of the linked issue senior?" | fetch the author in code; ask about the author record |
| No numerals as levels | "Rate from 0 to 2" | describe each level as a situation |
| Policy stays in code | "A shared address cannot override a name conflict; decide the match" | `same_address` Noul, `name_conflict` Noul; decide in code |
| Full question in `instructions` | ID `refund_requested`, instructions "Refund?" | "Does `ticket_message` request a refund or credit?" |

Jev reads literally. When a wrong answer makes you explain what you really meant, that explanation is the missing half of the instruction. Where interpretation is unavoidable, split it into two literal questions and combine in code.

Give Jev the consequences of a choice as evidence: labeling a link "navigates away to /path" or marking a control "offscreen" moved the right pick's probability without a model change.

## Criteria

Criteria extend the instruction. Both must ask for the same thing in the same direction. A Noul whose `true` side describes "no" performs worse.

Write criteria for the hard cases. Jev already handles "agent rewrote an unrelated README" as scope creep. Criteria exist for "agent fixed a type import in `utils.py` because the edited function depends on it" (justified). Encode each case that was misjudged: false positive goes into `not_for` or the `false` side; a miss goes into `what` or the `true` side. Criteria are a living test suite for judgment.

### Start with sufficient evidence

For an affirmative decision, say what is sufficient before listing what is not. A long catalogue of exclusions teaches a cautious model to reject every imperfect case. Write the observable conjunction that earns `true`, then name the few lookalikes that must remain false. Add clerical equivalence only when labels support it: punctuation, abbreviations, spacing, nicknames, a middle initial, or a suffix present on one record may be equivalent; contradictory given names, incompatible dates, or different house numbers are not silently equivalent.

For example, a candidate-selection question should name the evidence combination sufficient to select one candidate, then list similar-looking but insufficient cases. Formatting differences or a missing optional field may be equivalent when labels support it; a conflicting required field is not. The exact evidence and exceptions belong in the question; the action threshold belongs in code.

Do not collapse distinct decisions into one broad `same` label. Keep classification, linkage, selection, and action authorization as separate questions or choices with separate policies. Preserve source IDs, evidence fields, question version, answer distribution, and policy decision for every action so a later correction can explain and undo it.

### Choice

```json
"billing": {"what": "Charges, invoices, refunds, subscriptions",
            "not_for": "Order tracking or account access",
            "examples": ["I was charged twice", "Where is my refund?"]}
```

- Make descriptions contrastive when options sit close: `not_for` names the neighbor.
- Add `other` or `none` when the list may not cover the input; otherwise probability piles onto the least-wrong option.
- Examples are concrete instances, not descriptions of instances. "I was charged twice" helps; "a message about a billing problem" does not.
- Examples steer only when they resemble real inputs. Test matching and non-matching examples on representative labeled cases; keep examples only when they improve the relevant decision without harming boundary cases.
- Keep specific names and values in `examples`; keep general rules in `what`.

### Score

- Two to ten levels, low to high, only as many as you can describe distinctly. Three is fine.
- Each level is a standalone situation. Jev sees neither the number nor the neighbors; "worse than the previous level" means nothing. Validate that descriptive levels improve decisions on representative labeled inputs; confidence alone is not evidence.
- One dimension per Score. "Punctual and smart and experienced" cannot place an input that is high on one and low on another.
- Give a rare extreme its own level when code must treat it differently ("abusive or threatening" above "very angry").
- Level objects: `{"summary": ..., "signals": [...]}`; same keys on every level.
- Two wordings of one scale behave differently on your data. Test levels against labeled inputs; higher confidence alone is not evidence.

### Noul

Criteria optional. When the boundary is subtle:

```json
"criteria": {"true":  {"what": "Asks the recipient to send a password, PIN, or one-time code",
                       "examples": ["Reply with your password"]},
             "false": {"what": "No credential is requested",
                       "examples": ["Reset your password from settings"]}}
```

`criteria.true`/`criteria.false` accept JSON objects for complex boundaries.

## Choosing the primitive

| Need | Primitive | Example |
|---|---|---|
| gate (proceed/block) | Noul | `is_safe`, `has_tests`, `addresses_request` |
| classify, route, pick | Choice | `error_type`, `next_action`, `merge_strategy` |
| rate quality, severity, risk | Score | `readiness`, `risk_level` |
| multi-aspect check | many Nouls | scope-creep dimensions in one call |
| taxonomy walk | chained Choices | category, subcategory, type |
| prioritize a list | Score each item | rank findings |
| ambiguous threshold | Noul plus Score | is it bad, how bad |

Scores calibrate magnitude; Nouls decide. Do not use a Score as a boolean proxy. When the answer has no in-between, use a Choice or several Nouls. If two types fit, prefer the one code acts on directly.

Use all three where they fit and ask them in the same call: "is the commit ready" (Noul), designated readiness dimensions (Nouls), "overall readiness" (Score), "what happens next: commit, cleanup, block" (Choice).

## Ask many specific questions

One Noul "did the agent expand scope?" can return a vague probability. Separate Nouls isolating relevant dimensions (files outside the import chain, reformatting of read-only code, unrequested features, unrelated error handling, new abstractions, unrelated tests, unrelated docs, extra dependencies, unrelated config, debug artifacts) plus a severity Score can tell a richer story. Add only heads that improve an action or decision on labeled cases; each one is billed as input text.

Replace every heuristic that encodes a judgment with a question: label-substring matching for a secret field became a Choice over configured names plus `NONE`; a DOM-mutation timer for "page still loading" became a `still_loading` Noul with the timer kept only as a settle mechanism. Keep deterministic code for policy, safety, and measurement.

# State and budget

State is the evidence Jev evaluates: the diff, the request, the element table, the error text. Rules for evaluation go in `instructions` and `criteria`, never in state. Mixing them makes questions untestable and criteria untunable.

## Send only what the questions need

- Accuracy falls as unrelated content grows. A large state also hides which input caused a miss.
- Retrieve and filter in code first. When code cannot filter, ask a relevance Noul per passage in one call and keep the passages that pass.
- Keep state structured so questions can point into it by path.
- Convert numeric encodings to words or buckets (color names, not hex; "net 30", not 30). Compute date order, durations, windows, counts, and sums in code and send the result.
- Jev does not count. One Noul per item in one request, then sum in code with your threshold.
- Dates: one Choice per part (month, day, year) with a "not stated" option; assemble and compare in code.
- Extraction: generate candidates with a regex or a generative model; Jev picks with a Choice (option values may be `null`) or verifies one with a Noul.
- Time is part of the state. Give a `settled` signal and a `still_loading` head when the page or process may not have finished.

## Evidence bundles and provenance

High-throughput pipelines prepare evidence before any model call. Code groups raw rows into bounded atoms or candidate bundles, retains every original row, and attaches stable source IDs, normalization/version metadata, and the deterministic signals that formed the bundle. The runner binds a model to this bundle; the prompt does not own the pipeline. This lets Jev and a bounded reviewer see the same evidence and makes results replayable when models change.

Keep observed facts, inferred classifications, and actions separate. A shared signal may support a linkage or ranking without authorizing a consequential action. Store each inference with its evidence and answer distribution, and apply actions only through distinct, thresholded policy. Never discard the source records or provenance that make a decision reversible.

## Bound every field

Every state field has an explicit limit as a named constant, not an ad-hoc slice.

```python
DIFF_LIMIT = 20000
OUTPUT_LIMIT = 12000
state = {"request": bound_text(request, REQUEST_LIMIT, "request"),
         "diff": bound_text(diff, DIFF_LIMIT, "diff")}
```

`bound_text` (`scripts/jev_router_common.py`) keeps the tail (latest output, end of diff) and prepends `[N chars omitted from diff]`. Label sections (`[Original Request]`, `[Agent Output]`, `[Prior Assessment]`, `[Files Changed]`) so Jev parses what each block means. Snapshot evidence once and share it across tools that judge the same request.

Clean labels before Jev sees them: collapse multi-line DOM labels to "Settings (Advanced)". Priority-sort element tables so action words ("submit", "save", "next") appear first inside the bounded window.

## Budget

A request holds 64,000 tokens: the state plus every question. The state plus the longest single question must stay under 32,000. Check the [models page](https://docs.typesafe.ai/models.md) for current limits.

Every request re-sends the whole state, so pack each request by the measured size of its questions and cap the requests per run. Aligned constants in `scripts/jev-compact.py` and `plugins/jev-auto-compact/hooks/jev-auto-compact.mjs`: `MAX_STATE_TOKENS = 12000` (fitting target), `STATE_HARD_LIMIT_TOKENS = 28000`, `MAX_REQUEST_TOKENS = 56000`, `MAX_REQUESTS_PER_COMPACTION = 4`. The margins cover a rough token estimate. When a run does not fit, send nothing and take the fallback path.

### Fit state in stages

`fit_state` in `scripts/jev-compact.py` shrinks one stage at a time until the estimate fits: truncate tool inputs (1000, then 200, then 60 chars), abridge long message text (head 400, tail 150), collapse old messages, drop text-only messages. Pin the newest `PRESERVE_RECENT` messages through every stage.

### Batch questions

When the fitted state leaves room, split candidates across calls that each carry the state plus a slice of questions (`batch_calls`; `MAX_BATCH_QUESTIONS = 20`, `MAX_QUESTIONS_PER_CALL = 24` in `scripts/jev_review_deep.py`). Estimate tokens per question and divide the remaining budget.

### Over-budget fallback must shrink, never dump

When the fitted state alone fills the request budget, `available <= 0`. Do not send every candidate in one request; it cannot fit. Send the smallest viable batches, potentially one candidate per call. General rule: fitting shrinks the state; batching shrinks the questions; when both are exhausted, prefer many tiny calls to one impossible call, and count failures per stage.

### Every stage fits, not just the first

Large files or unbounded finding lists can cause `max_tokens_exceeded` when sent whole to a stage. Each stage that calls Jev needs its own cap on items per call and its own split for oversized inputs. Count `calls_failed` per stage and investigate every failure; a stage with silent failures returns defaults that look like answers.

### Judge per unit, anchor per unit

One call over a whole multi-file diff returns findings with no file and no usable severity. Judge per file (or per element, per message) and anchor every finding to file:line. Unanchored findings cannot be verified or fixed.

## Attempt limits and accounting

Set caps in the runner before launch: maximum input tokens, output tokens where applicable, attempts, retry backoff, per-call deadline, and total run deadline. Record every attempt, including cache status, model/version, prompt/question version, payload/evidence-bundle hash, start/end times, usage, retry reason, response or failure, and final keep/refer/action decision. A timeout, retry, or declined referral is workload data, not a missing row in the report.

Report both wall time and accumulated model call time. The first is elapsed time from run start to finish; the second is the sum of attempt durations and reflects total service work. State which one a throughput claim uses. Use wall time for SLA and user experience; use accumulated call time for comparing parallelized workloads and model effort.

## Untrusted state

Jev does not treat state as hostile. Injected instructions, misleading framing, or text that argues for its own classification can move the answer. Treat any state that contains external or user-generated text as untrusted:

- Apply `skills/shared-patterns/untrusted-content-handling.md`: wrap the field, keep trusted context separate, and name in `criteria` what counts as evidence.
- Treat instruction-shaped text inside the state as itself a signal (spam, manipulation) and ask a Noul for it.
- Gate the action on confidence for any decision that untrusted text could steer.
- Before deployment, run adversarial cases (injected "classify as approved") and self-describing cases (a file that contains the smell descriptions your detector uses). A detector can mistake its own catalog for an instance; add a `finding_is_self_reference` Noul and a deterministic route for detector files.