d
Jev request router: validates the requested outcome, then dispatches to the matched agent, skill, and pipeline.
Install
npx skills add https://github.com/notque/vexjoy-agent/tree/main/skills/meta/d
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install notque-vexjoy-agent@llmmart
git clone https://github.com/notque/vexjoy-agent.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole notque/vexjoy-agent collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
/d — Jev Router
Classifies requests through Jev and dispatches to the matched agent, skill, and pipeline. Before every dispatch, it restates the requested outcome and uses Jev to check that the restatement and route preserve it.
The classification path has three layers: a deterministic pre-route.py
force-route guard (offline, runs first, authoritative for git/security), a
configured Jev transport presence check, and a two-stage classification — a cheap
wide-rank stage 1 over all manifest candidates plus a trivial-bypass gate,
then a full-detail shortlist-rerank stage 2 with per-candidate fit checks
and stack/fan-out signals.
Design rationale: ${CLAUDE_SKILL_DIR}/references/jev-classifier-design.md.
Phase Banners
Every phase: /d > Phase N: PHASE_NAME — description...
After intent alignment resolves: === routing banner. Both required.
Phase 1: CLASSIFY
scripts/jev-route.py owns the entire classification in one subprocess call.
When JEV_RESULT is already in context (the jev-route-injector hook ran the script before your first token), use it and skip the command below.
REQUEST_FILE=$(mktemp); printf '%s' "{user_request}" > "$REQUEST_FILE"
python3 "$SDIR/jev-route.py" --request-file "$REQUEST_FILE" --json-compact
rm -f "$REQUEST_FILE"
Resolve $SDIR: ${HOME}/.claude/scripts, falling back through
.hermes/.factory/.codex/.reasonix, or the repo's scripts/
directory.
Hold the result as JEV_RESULT. Shape (stable — see design reference for
full schema):
available, jev_called, matched, fallback, fallback_reason,
agent, skill, pipeline, complexity, confidence, match_type,
reasoning, stack, signals, signal_scores, source, latency_ms,
usage, agents, gate_score, fits_scores, stage1_shortlist, and
intent_alignment (the hook-generated baseline alignment receipt).
latency_ms and usage are itemized dicts
({"stage1_ms","stage2_ms","total_ms"} and {"stage1","stage2"}). Read
.total_ms for a single latency figure.
Gate: fallback == true → Phase 1F. Every matched result,
including source == "jev-trivial-bypass", proceeds to Phase 2: ALIGN INTENT.
Phase 1T: TRIVIAL-BYPASS (source == "jev-trivial-bypass")
Stage 1's gate fired: gate_score below threshold, no agent/skill/pipeline
is needed. It remains a direct-handling path, but it must still pass through
Phase 2 so the user outcome is restated and Jev validates it. After an
aligned Phase 2 result, show Classification: Trivial and
Source: jev-trivial-bypass, then answer or do the one-line action directly.
Do not run Phases 3–5 or call build-dispatch.py. Stop there.
Phase 1F: UNAVAILABLE (fallback == true)
Jev could not classify this request. JEV_RESULT.source explains why:
unavailable— neither configured Jev transport is available./daccepts Vercel AI Gateway (AI_GATEWAY_API_KEY) or the direct Jev API (TYPESAFE_API_KEY), selected byJEV_TRANSPORT=auto|vercel|direct.invalid-pick— Jev's pick was not a valid manifest name.error— a Jev call timed out or failed.
Show:
===================================================================
/d: Jev unavailable — [JEV_RESULT.fallback_reason]
Use /do for manifest-based routing.
===================================================================
Fail open to /do's full routing flow and continue the request. Do not reject
the request merely because Vercel AI Gateway is unavailable.
Phase 2: ALIGN INTENT (required for every matched /d route)
MANDATORY STOP: For every matched /d invocation, write
PROPOSED_INTENT and run the validator on that exact text before any routing
banner, dispatch, answer, edit, or other action. JEV_RESULT.intent_alignment
is only the hook baseline and does not satisfy Phase 2. This requirement has no
exception for force routes, trivial routes, or an apparently aligned baseline.
Before selecting the work method, write PROPOSED_INTENT: a concise one- or
two-sentence restatement of what the user wants accomplished. State the
outcome, material surfaces or deliverables, and every explicit constraint.
Do not describe the selected agent, skill, or implementation mechanics as the
outcome. Preserve the user's words where precision matters.
Run the Jev validator even when the hook already supplied
JEV_RESULT.intent_alignment; that receipt validates a conservative baseline,
while this call validates the actual restatement that will enter the task spec.
Put the request, route JSON, and proposed intent in temporary files rather
than shell-splicing user text, then call:
python3 "$SDIR/jev-intent-align.py" \
--request-file "$REQUEST_FILE" \
--route-file "$ROUTE_FILE" \
--proposed-intent-file "$INTENT_FILE" \
--json-compact
The validator sends one bounded state and all independent questions together
through the selected Jev transport. It checks whether the outcome and constraints
are preserved, the route can cover the material scope, the restatement is too
narrow, it introduces unrequested work, and essential clarification is needed.
It returns aligned, clarification_needed, issues, and raw scores.
Show this before the routing banner:
Intent alignment (/d):
-> Restated outcome: [PROPOSED_INTENT]
-> Jev: [aligned|review|unavailable] [issues, if any]
Gate:
clarification_needed == true→ ask one concise question that names the essential ambiguity; do not dispatch until answered.alignment == alignedandsource == jev-trivial-bypass→ direct handling in Phase 1T; otherwise → Phase 3.alignment == reviewbecause scope is lost, work was added, or the route cannot cover the request → correctPROPOSED_INTENTor the route and run this validator once more. Carry unresolved issues intotask_spec.gaps; do not silently proceed as though Jev approved it.alignment == unavailableorerror→ state that validation was unavailable, preserve the verbatim request and proposed intent in the task spec, then continue under the normal/drouting result. Gateway outage must not become a false request rejection.
This runtime gate applies to every matched route, including force-routes and
trivial bypasses. A Phase 1 fallback cannot run this gate because no usable Jev
route exists; it fails open to /do as described in Phase 1F.
This is an instruction gate enforced by the /d contract, not a hook-enforced
technical boundary. The user remains the final backstop if an agent violates it.
Phase 3: DECIDE (fallback == false, after aligned intent)
JEV_RESULT.source is either pre-route-force (deterministic guard matched)
or jev (Jev classification, manifest-validated).
Apply directly:
agent/skill/pipeline: useJEV_RESULT's values as-is. Already validated against the live manifest membership sets inside the script.complexity: useJEV_RESULT.complexitywhen set. Whennull(always forpre-route-force), default tomedium, except a single one-line trivial fix →simple.- Confidence:
JEV_RESULT.confidence(high/medium/low).
Routing banner (first visible output after the required intent record):
===================================================================
ROUTING (/d): [brief summary]
===================================================================
Selected:
-> Agent: [JEV_RESULT.agent] - [JEV_RESULT.reasoning]
-> Skill: [JEV_RESULT.skill] - [JEV_RESULT.reasoning]
-> Pipeline: [JEV_RESULT.pipeline, if set]
-> Source: [JEV_RESULT.source] (confidence: [JEV_RESULT.confidence])
Invoking...
===================================================================
Gate: Agent+skill set, banner shown. Phase 4.
Phase 4: ENHANCE (stack signals)
JEV_RESULT.signals (booleans at 0.6 confidence threshold, computed by the
script) map to stack entries:
| Signal true | Stack |
|---|---|
tests_requested |
test-driven-development + verification-before-completion |
research_needed |
add research-coordinator-engineer to agents (fan-out) |
comprehensive_review |
parallel-code-review (drop if a real multi-file diff exists — right-size-review.py outranks it) |
local_only |
inject shared-patterns/local-only.md |
objective_loop_worthy |
objective-loop |
anti-rationalization-core always rides. When source is
pre-route-force and JEV_RESULT.stack is non-empty (e.g. go-patterns),
keep it.
Fan-out agents: union JEV_RESULT.agents (script-computed fan-out picks,
each passed its per-candidate fit check) into the research_needed agent
list, deduped. Dispatch fan-out agents as separate parallel Agent tool
calls alongside the primary build-dispatch.py dispatch.
Gate: Stack applied. Phase 5.
Phase 5: EXECUTE
Build the task spec with request_verbatim unchanged and intent exactly
PROPOSED_INTENT; include any unresolved alignment issue in gaps, then invoke
build-dispatch.py:
python3 "$SDIR/build-dispatch.py" --json '{
"agent": "<JEV_RESULT.agent>", "skill": "<JEV_RESULT.skill; omit when agent-only>",
"pipeline": "<JEV_RESULT.pipeline; omit when null>",
"complexity": "<from Phase 2>",
"model": "inherit",
"context_mode": "summary",
"provider": "<anthropic|openai|other>",
"manual_model_override": false,
"health": "-",
"fallback_reason": "<REQUIRED when agent=general-purpose; omit otherwise>",
"stack": ["s1","s2"],
"task_spec": {"request_verbatim": "<user message, unchanged>", "intent": "...",
"constraints": "<applicable rules, limits, and authorization>",
"decisions": "...",
"gaps": "...",
"acceptance": "<command> -> <expected>",
"files": "<owned paths; optional line ranges>", "ownership": "<worker scope>",
"operator_context": "..."},
"flags": {"worktree": false, "local_only": false, "thinking_override": null},
"token_remaining": 480000
}'
The builder validates each name against its index, then emits the dispatch action. For Complex or creation requests, apply creation detection, plan-file gating, quality-loop, workflow dispatch, fan-out, and auto-pipeline fallback.
Gate: Agent invoked, results delivered.
Error handling
Errors inside jev-route.py resolve to fallback: true, source: "error" —
Phase 1F reports the error and fails open to /do.
References
${CLAUDE_SKILL_DIR}/references/jev-classifier-design.md— request/response contract, fallback conditions, phase-by-phase design decisions${CLAUDE_SKILL_DIR}/SPEC.md,${CLAUDE_SKILL_DIR}/EVAL.md— maintenance contract and regression cases (load only when creating, evaluating, or redesigning this skill)scripts/jev-route.py,scripts/jev-intent-align.py,scripts/jev_transport.py,scripts/jev_vercel.py,scripts/jev_gateway/jev_vercel_gateway.mjs,scripts/pre-route.py,scripts/routing-manifest.py,scripts/build-dispatch.py- Jev hook:
hooks/jev-route-injector-userprompt.py(UserPromptSubmit) precomputesJEV_RESULT
Files (vexjoy-agent)
-
references
-
jev-classifier-design.md 19.9 KB
# Jev classifier design Deep reference for `/d` (`skills/meta/d/SKILL.md`). Load this when explaining, tuning, or extending the router; SKILL.md's phases are enough to run it. **v2 redesign, 2026-09-16.** This document describes the CURRENT two-stage design. v1 (single flat `Choice` call per dimension) is superseded; its numbers are cited only as the documented before/after baseline (`scripts/routing-ab-results/jev-router-v1-2026-09-16/VERDICT.md`). ## Presence contract `scripts/jev_router_common.py: typesafe_available() -> (bool, str)`. Unchanged by the v2 redesign. True only when both hold: 1. `TYPESAFE_API_KEY` is set and non-empty in the environment (value never read into logs, prints, or files — only its presence is checked). 2. `enabledPlugins["typesafe@typesafe-ai"] == true` in the merged settings: `~/.claude/settings.local.json` values win over `~/.claude/settings.json` for any key present in both; a missing file is `{}`, not an error. Either condition false -> `/d` never attempts a network call. This is a presence check, not a health check — a configured-but-down TypeSafe endpoint still attempts the call and falls back on the resulting timeout/error, not on this check. ## Why v1 was replaced v1 asked ONE flat `Choice` question per dimension (agent, skill, pipeline, complexity) over ALL manifest candidates at once, with descriptions truncated to ~100 characters each — and never sent `not_for` disambiguation text to Jev at all, at any point, for any dimension. Two independent problems, not one: 1. **Truncation cut real disambiguating content inside `description` itself** for any entry whose description ran past ~100 characters (several agent/skill descriptions do — see "Cookbook reuse and divergence" below for the measured effect). 2. **`not_for` was never in the payload, truncated or not.** Many entries carry an explicit `not_for` clause naming the exact sibling skill/agent a request is likely to be confused with (e.g. `reviewer-code`'s `not_for` says "business-logic correctness, ADR conformance... (use reviewer-domain)"). v1's single pass had no way to show Jev that text, so Jev had no signal telling it "not this one, that one" beyond the short descriptions themselves. Owner-mandated fix: a two-stage progressive-disclosure pattern (cheap wide rank, then full-detail shortlist rerank), NOT the cookbook's shape transplanted wholesale — see below for exactly what was kept and what changed, and why. ## Request/response shape (v2, two calls) Up to TWO `POST https://api.typesafe.ai/v1/systemone` calls per routing decision that reaches Jev at all: 0 on force-route or TypeSafe-unavailable, 1 on trivial-bypass (stage 1 only), 2 otherwise (stage 1 + stage 2). Never per-dimension, never more than 2 — this is a hard cost constraint, not a default that grows with manifest size. ### Stage 1 (wide rank + trivial-bypass gate) One call. Body: ```json { "state": "<user request verbatim>", "model": "jev-latest", "questions": { "agent": {"type": "choice", "instructions": "...", "criteria": {"<agent>": "<~100-char desc>", ..., "general-purpose": "..."}}, "skill": {"type": "choice", "instructions": "...", "criteria": {"<skill>": "<~100-char desc>", ...}}, "pipeline": {"type": "choice", "instructions": "...", "criteria": {"<pipeline>": "<~100-char desc>", ..., "none": "..."}}, "needs_skill": {"type": "noul", "instructions": "..."}, "needs_pipeline": {"type": "noul", "instructions": "..."}, "prose_suffices": {"type": "noul", "instructions": "..."} } } ``` `criteria` for `agent`/`skill`/`pipeline` are the SAME ~100-char truncated descriptions v1 built (`_truncate_desc`, `_build_criteria_maps` — reused unchanged; a cheap skim doesn't need full text, only stage 2 does). Response: `{model, answers: {qid: Answer}, usage}`. Each `choice` answer carries `{choice, probabilities, confidence}` — `jev-route.py` reads `probabilities` (not just the single top `choice`) to build shortlists, since the whole point of stage 1 is "who are the top ~3 contenders," not "who is the single best guess from a cheap pass." `jev-route.py` (`_parse_stage1`) computes, per single-select dimension: - **agent, skill**: top `STAGE1_SHORTLIST_N` (3) names by `probabilities`, restricted to live manifest membership. - **pipeline**: top 1 REAL (non-`"none"`) candidate by `probabilities`, computed unconditionally — even when `"none"` won stage 1's own `choice`. Rationale: with only ~29 pipeline candidates and a cheap truncated pass, "none" can win narrowly on a genuinely ambiguous case; forwarding the best real candidate anyway gives stage 2's full-detail rerank a real chance to confirm or reject it, rather than letting the cheap pass foreclose the pipeline slot before the precise pass ever runs. This is the same "don't let cheap-pass noise be final" reasoning behind giving agent/skill a 3-wide shortlist instead of trusting stage 1's single top pick outright — pipeline gets a 1-wide shortlist only because there are far fewer real candidates to protect against, not because the reasoning differs. `gate_score` = mean of the three oriented gate `Noul`s: `needs_skill`, `needs_pipeline`, and `1 - prose_suffices` (prose_suffices is inverted before averaging, so a HIGH gate_score always means "routing is needed," matching the other two nouls' orientation). Below `--gate-threshold` (default 0.30): **trivial-bypass** — `source: "jev-trivial-bypass"`, `complexity: "trivial"`, `agent`/`skill`/`pipeline` all `null`, `matched: true`, `fallback: false`. Stage 2's HTTP call is skipped entirely; this is the one case with only one round trip. This maps directly onto `/do`'s own Trivial classification (`skills/meta/do/SKILL.md` Phase 1: "Trivial: ONLY user-named file by path... never dispatches, handled directly") — `/d`'s SKILL.md Phase 1T handles it the same way: direct, no Phase 4. ### State and shortlist size `state` is the bare request string when no project facts exist. When the hook passes `--cwd`, `detect_project_context()` reads marker files (for example `pyproject.toml`, `package.json`, `go.mod`) and dependency manifests in that directory, and `state` becomes `{"request": ..., "project": {"languages": [...], "frameworks": [...], "datastores": [...]}}`. The project block holds names only, never paths or file contents. The agent instructions gain one sentence that tells Jev to use `project` when the request names no language. The stage-1 shortlist is 6 agents and 6 skills (`STAGE1_SHORTLIST_N`, `--shortlist`). Stage 2 cannot pick outside the shortlist, so its size caps accuracy. Measure coverage offline from the stored stage-1 probabilities in `jev_calls` before changing it. Score the router with `scripts/jev-eval.py --split dev` while tuning and `--split test` once at the end. `--workload-dir NAME=PATH` supplies the repository for corpus cases that record a workload. ### Stage 2 (shortlist rerank + fits + multi-select), only when gate clears One call, only reached when `gate_score >= gate_threshold`. Body (shape, candidate counts vary per request): ```json { "state": "<user request verbatim>", "model": "jev-latest", "questions": { "agent": {"type": "choice", "instructions": "...", "criteria": {"<top-6 agent shortlist>": "<full desc NOT: not_for>"}}, "agent_fit__<name>": {"type": "noul", "instructions": "..."}, "skill": {"type": "choice", "instructions": "...", "criteria": {"<top-6 skill shortlist>": "<full desc NOT: not_for>"}}, "skill_fit__<name>": {"type": "noul", "instructions": "..."}, "pipeline": {"type": "choice", "instructions": "...", "criteria": {"<top-1 pipeline>": "...", "none": "..."}}, "pipeline_fit__<name>": {"type": "noul", "instructions": "..."}, "tests_requested": {"type": "noul", "instructions": "..."}, "research_needed": {"type": "noul", "instructions": "..."}, "comprehensive_review": {"type": "noul", "instructions": "..."}, "local_only": {"type": "noul", "instructions": "..."}, "objective_loop_worthy": {"type": "noul", "instructions": "..."}, "fanout__<name>": {"type": "noul", "instructions": "..."} } } ``` Criteria text for every stage-2 candidate is `description + (" NOT: " + not_for if present)`, sourced straight from `routing-manifest.py`'s live `load_entries()` output — already untruncated, no filesystem reads of SKILL.md/agent.md files added. Question keys for per-candidate `Noul`s are sanitized names (`_sanitize_key`, `[^a-zA-Z0-9_]` -> `_`) prefixed by dimension (`agent_fit__`, `skill_fit__`, `pipeline_fit__`, `fanout__`), with an in-memory map back to the original manifest name for parsing. **Fits check** (`_parse_stage2`, `--fits-threshold`, default 0.30): for each single-select dimension, the stage-2 `Choice` picks one name from the shortlist; that SAME name's own per-candidate fit `Noul` is then checked against `fits_threshold`. Below threshold -> that dimension's pick is rejected to `null`, regardless of what the `Choice` answer said. This reads owner's "the BEST-shortlisted candidate's own fit-Noul" as "the candidate the stage-2 Choice actually selected" (not a separate max-over-shortlist search) — `Choice` produces the name, `Noul` is the fit veto on that specific name. `fits_scores` in the result reports this per-dimension value for eval visibility (`{"agent": x, "skill": y, "pipeline": z_or_null}`). **Fallback trigger**: `agent` rejected (invalid membership OR sub-threshold fit) OR `skill` rejected -> the WHOLE decision is `fallback: true` — this replaces v1's flat `agent_conf < confidence_floor` check. `pipeline` is NEVER gated into the fallback decision (same as v1: a `null` pipeline is often the correct answer, most requests are single-phase). **Multi-select**: the 5 stack signals are independent per-candidate `Noul`s over the WHOLE request (not per-manifest-candidate), unchanged in mechanism from v1 — just moved into stage 2's call (they need the same rich context stage 2 already has, and stage 1's trivial-bypass gate makes them moot when it fires, so stage 2 is the natural home). Fan-out candidates (`agents` field) are a NEW per-candidate `Noul` over agents ranked just below the primary shortlist (the three agents ranked just after it in stage 1), asking "should this agent ALSO run in parallel, on a distinct independent subtask" — see "Fan-out selection rule" below for the exact gating heuristic and why it exists. ## Complexity: dropped as a Jev question, derived in Python v1 asked Jev a fourth `Choice` question for `complexity`. v2 does not — the owner's exact stage-1/stage-2 question lists never include a complexity `Choice`, and asking one would violate the "never proliferating" cost constraint for no clear benefit once gate_score, the final picks, and the 5 stack signals already exist. `jev-route.py` derives `complexity` deterministically after stage 2 resolves: - `pipeline` set OR `agents` (fan-out) non-empty -> `"complex"` (matches `/do`'s own "2+ agents... or a genuine multi-phase pipeline" criterion) - else any stack signal true -> `"medium"` (matches `/do`'s "extra rigor" criterion) - else -> `"simple"` - trivial-bypass -> `"trivial"` (stage 1 only, no stage 2 needed to know this) This is a deliberate, documented trade: one fewer Jev question per decision, a small saved cost, in exchange for a simpler and fully auditable Python rule instead of a fifth judgment call riding in the same HTTP call. ## Fan-out selection rule `_select_fanout_candidates`: agents ranked 4-6 in stage 1's probability ranking (`FANOUT_RANK_START=3`, `FANOUT_MAX_CANDIDATES=3`), asked a per-candidate fan-out `Noul`, but ONLY when BOTH: 1. `gate_score >= FANOUT_GATE_SCORE_MIN` (0.6) — a heuristic proxy for "this is substantive/complex work," well above the 0.30 trivial-bypass floor, so borderline-trivial requests never pay for fan-out questions. 2. The top-ranked agent's stage-1 probability is `< FANOUT_DOMINANCE_PROB` (0.75) — a heuristic proxy for "more than one plausible domain owner exists." When one agent clearly dominates, fan-out questions would almost certainly all come back false, so they're skipped rather than spent. Both thresholds are cost-control heuristics, not precision claims — read `/do`'s own MULTI-AGENT RULE (`skills/meta/do/SKILL.md` Phase 1: "Parallel FIRST: 2+ failures / 3+ subtasks -> multiple Agent tools") before changing the fan-out `Noul`'s instructions text, so the question asked matches what `/do` actually means by fan-out-worthy. **Known gap, not wired**: `scripts/build-dispatch.py`'s `--json` schema has no dedicated fan-out/`agents` field — it dispatches exactly one agent per call. This is not actually a `/d`-specific gap: `/do`'s own real multi-agent mechanism is separate parallel `Agent` tool calls issued by the orchestrator alongside the primary `build-dispatch.py` dispatch, not a `build-dispatch.py` JSON field. `/d`'s Phase 4 reuses that same mechanism unmodified (`skills/meta/d/SKILL.md` Phase 3/4) — `JEV_RESULT.agents` is a list to read at that existing fan-out decision point, not a new contract to invent. ## Cookbook reuse and divergence TypeSafe's `skill_suggestion` cookbook (`https://docs.typesafe.ai/cookbooks/skill_suggestion.md`) is the source of the two-stage progressive-disclosure pattern adopted here. Its own published benchmark (182 skills, 488 requests, `claude-haiku-4-5`): suggestion-assisted selection cut wrong loads from 16.8% to 7.3% (2.3x fewer) and needless loads from 9.8% to 4.0% (2.4x fewer), against an oracle floor of 2.5%/1.2%. **This is the cookbook's own benchmark, on its own single-pick skill-selection task — not a `/d`-specific measurement.** `/d`'s own numbers (this session's `scripts/routing-ab-results/jev-router-v2-2026-09-16/VERDICT.md`) are the ones that actually matter for `/d`'s promotion decision; cite the cookbook figure only as the stated rationale for trying the pattern at all. **Reused as-is**: - The two-stage shape itself: cheap wide `Choice` rank over truncated descriptions, then a `Choice` + per-candidate `Noul` rerank over a shortlist with full descriptions. - The gate-then-fits threshold pattern: a cheap up-front gate (mean of oriented `Noul`s) deciding whether to do the expensive pass at all, then a per-candidate `Noul` "does this genuinely fit" veto on the expensive pass's own pick. `/d`'s `GATE_THRESHOLD`/`FITS_THRESHOLD` defaults (0.30 each) match the cookbook's own defaults. **Deliberately diverged, and why**: - **Multi-slot vs. single-pick.** The cookbook selects ONE skill suggestion. `/do`'s actual contract (`skills/meta/do/SKILL.md` COMBINATION DOCTRINE) is multi-slot: a primary agent plus an independent fan-out `agents` list, a primary skill plus an independent `stack` list, and one optional pipeline. `/d` runs the cookbook's single-pick shape three times over (once per single-select dimension: agent, skill, pipeline), not once. - **Multi-select via independent Noul, not Choice.** The 5 stack signals and the fan-out candidates are "zero or more apply independently," not "pick one." A `Choice`'s `probabilities` are a distribution that sums to 1 across its options — the wrong shape for "any subset can be true at once." Each multi-select dimension is instead an independent per-candidate `Noul`, exactly as v1 already did for the 5 stack signals (that part of v1 was already right; v2 keeps it and extends the same pattern to fan-out). - **Pipeline gets a 1-wide, not 3-wide, stage-2 shortlist.** The cookbook reranks a fixed top-3. `/d` only has ~29 pipeline candidates total (versus 182 skills in the cookbook's benchmark), and pipeline is the rarest non-null pick across the corpus — a 1-wide "best real candidate vs. none" shortlist was judged precise enough without paying for 2 more Choice criteria entries and 2 more fit-`Noul`s on every non-trivial request. The stage-2 fits-check still always runs for whichever real candidate stage 1 forwarded — never skipped, only narrower. - **Complexity dropped as a question entirely** (see above) — the cookbook has no complexity-equivalent question to diverge from; this is a `/d`-side simplification enabled by already having gate_score, the final picks, and the stack signals to derive it from. ## Fallback-to-`/do` behavior `fallback: true` on any of: `unavailable` (presence check failed), a sub-`fits_threshold` or invalid-membership agent/skill pick (`source: "low-confidence"`), or `error` (a stage-1 or stage-2 HTTP/parse/timeout failure). In every case `/d`'s SKILL.md Phase 1F instructs reading `skills/meta/do/SKILL.md` in full and running its Phase 1-4 unmodified — not a degraded in-between state. `jev-trivial-bypass` is NOT a fallback signal — it is a real terminal state handled by Phase 1T, distinct from Phase 1F. Measured on `scripts/routing-ab-corpus.json` v1.5 (269 cases, `scripts/routing-ab-results/jev-router-v2-2026-09-16/VERDICT.md`): fallback rate 26.0% (down from v1's 62.1% on the same corpus family, measured on corpus v1.4) — still common, not a rare edge case, but the fits-threshold redesign materially reduced how often it fires versus v1's flat confidence floor. ## Confident-wrong risk (read before trusting this router with real traffic) Unchanged conclusion from v1, restated because it still applies and the redesign does not remove it: the fits-threshold catches an uncertain Jev answer. It does NOT catch a systematically confident-but-wrong classifier — that failure mode broke the rejected `tiered-v2` manifest experiment (3 new safety-bucket misses, all confident or null, not low-confidence). The fits-threshold is a per-request rejection trigger, a better one than v1's flat confidence floor (it checks "does THIS specific candidate genuinely fit" rather than "was the classifier confident in general"), but it is still a **runtime** safety net, not a systematic-confident-wrong-classifier backstop. That job still belongs to the eval gate: `scripts/routing-ab-corpus.json`'s `SAFETY_BUCKETS` (`benchmark-force_route`, `false-positive-guard`, `paraphrase-git`, `paraphrase-security`) are checked individually, per bucket, before `/d` is trusted with real traffic — a confidently-wrong pattern fails that gate outright, at any confidence or fits-score level. `benchmark-force_route`/`paraphrase-git`/`paraphrase-security` are additionally protected at runtime by construction: they're supposed to be caught by `pre-route.py` before Jev is ever called (0 HTTP calls, 0 fits checks in play). `false-positive-guard` has no such runtime backstop, in `/d` or in `/do` — idiom traps (e.g. "fish out the bug") rely on semantic judgment catching them, whether that judgment is `/do`'s self-route or Jev's classification; neither router has a deterministic guard for this bucket, an existing property of semantic routing generally, not something `/d` introduced. The v2 redesign measured PASS on this gate (0 critical findings) on the same corpus that measured v1's PASS — the redesign did not introduce a new safety regression, but it also did not, and does not claim to, solve the underlying confident-wrong risk structurally. ## Hidden coupling (known limitation, unchanged by v2) `jev-route.py`'s `instructions`/criteria strings across both stages are a hand-written paraphrase of `/do`'s prose (Phase 1 Trivial table, Phase 2 SECTION-INTEGRITY/FORCE-ROUTE/SPECIFICITY/COMBINATION DOCTRINE/MULTI-AGENT RULE, Phase 3 signal table). Nothing automatically keeps these in sync with `/do`'s prose — if `/do`'s semantics change, `jev-route.py`'s instruction strings must be updated by hand in the same change. No drift-detection CI gate exists for this yet. ## Data egress Stage 1 sends the raw request text and truncated agent/skill/pipeline names+descriptions to `api.typesafe.ai`; stage 2 (when reached) sends the request text again plus the shortlisted candidates' full descriptions. Same kind of call as any other model API this toolkit already talks to. Force- routed and trivial-bypassed requests send less (trivial: stage 1 only) or nothing (force-route: Jev isn't called at all).
-
-
EVAL.md 5.1 KB
# EVAL: /d — Jev-first router Regression cases. Load only when evaluating or redesigning this skill. Full corpus run: `python3 scripts/jev-eval.py --out-dir <new-dir>` against `scripts/routing-ab-corpus.json` (269 cases); `--out-dir` is required and must be a fresh directory — never overwrite a completed run. Results land under `scripts/routing-ab-results/jev-router-v<N>-<date>/` per `docs/router-ab-runbook.md`. The manual set below is the fast sanity check; it does not replace the full corpus run. ## Idiom-guard sanity set (manual, run via `jev-route.py --request "..." --json-compact`) | # | Request | Expected | Mechanism under test | |---|---|---|---| | 1 | "push my changes" | pr-workflow, `match_type: force_route`, `jev_called: false` | pre-route guard fires before Jev | | 2 | "Push back on this architecture before we commit to it" | NOT pr-workflow | idiom guard (pushback/commit) | | 3 | "Commit these changes and push to origin" | pr-workflow, force-route | genuine git intent | | 4 | "Fish for compliments from the design team before shipping" | NOT shell-config | idiom guard (fish=search) | | 5 | "Configure my fish shell prompt to show git branch" | shell-config | genuine Fish shell intent | | 6 | "Make it public that we're hosting a charity stream next week" | NOT public-web-deploy | idiom guard (make public != deploy) | | 7 | "Compare static site generators for a docs site" | NOT public-web-deploy (force) | work-on-a-site guard | | 8 | "Deploy my landing page to Vercel" | public-web-deploy | genuine deploy intent + companion word | | 9 | "help me plan a birthday party" | some sensible agent/skill via real Jev call | end-to-end Jev path exercised | | 10 | (simulate) both Jev keys unset | `fallback: true`, `source: "unavailable"` | presence-check fallback | | 11 | (simulate) `--fits-threshold 0.99` on a multi-agent request | primary pick remains valid; optional fan-out is filtered | fan-out-only fits threshold | | 12 | "thanks" / "hi" / "say hello" / "what is 2+2" | `source: "jev-trivial-bypass"`, `agent`/`skill`/`pipeline` all `null`, `matched: true`, `fallback: false` | stage-1 gate_score below `--gate-threshold`; stage 2 never called | **Known result**: case 6 currently force-routes incorrectly to `public-web-deploy` — traced to a pre-existing bug in `pre-route.py` itself (the word "hosting" satisfies the "make it public" companion-word gate). `jev-route.py` behaves exactly as designed — it never overrides a force-route hit, by construction — the bug is upstream in `pre-route.py` and out of `/d`'s scope to fix; it affects `/do` identically, since `/do` calls the same `pre-route.py`. Case 6 therefore remains a known failing regression case; it must not be listed among the cases that resolve correctly. ## Intent-alignment checks - A proposed intent that drops material scope, adds unrequested work, or uses a route that visibly conflicts with the request must return `alignment: review`. - The instruction gate must use the receipt produced for the exact runtime `PROPOSED_INTENT`; the hook-time baseline receipt cannot satisfy it. This is not hook enforcement, so tests must not claim a technical boundary that the implementation does not provide. - Every matched route, including force-route and trivial-bypass, must attempt runtime alignment. A classification fallback delegates to `/do` and must not claim that intent validation succeeded. - Essential ambiguity must return `clarification_needed: true`; routine implementation choices must not. - A 429, 503, 529, or timeout from Vercel retries at most twice and persists a safe receipt; other gateway errors fail open. - `auto` prefers Vercel, explicit transport selection never switches, and `direct` sends the same state and questions to the Jev API. ## Known failure modes - Jev returns a name not in the live manifest (renamed/deleted skill) -> must null + fallback, not silently pass through. Covered by manifest- membership validation in `jev-route.py`. - Jev call exceeds timeout -> must fallback cleanly, not hang or crash `/d`. Covered by the Vercel bridge retry receipt + `--timeout` flag. - Missing Gateway bridge/HTTP-client dependency -> must not crash. Covered: `jev-route.py` uses the isolated Vercel bridge; the bridge package is installed by `install.sh`, and a missing bridge fails open to `/do`. - Confidently-wrong classification on `paraphrase-security`/ `false-positive-guard` — no runtime backstop exists (see `references/jev-classifier-design.md` "Confident-wrong risk"); caught only by the per-bucket `SAFETY_BUCKETS` gate on the full corpus run, not by any single-request check. ## Full-corpus evidence Do not cite a run unless its verdict exists in the repository and was produced from the current corpus, manifest, and implementation. The previously cited 2026-09-16 v1/v2 verdict paths are absent, so their historical percentages are not part of this maintenance contract. Generate a fresh output directory with the command above when comparative routing evidence is needed. Intent alignment is production functionality established by repeated use; the corpus remains a regression tool for route quality. -
SKILL.md 11.4 KB
--- name: d version: "1.1.0" description: "Jev request router: validates the requested outcome, then dispatches to the matched agent, skill, and pipeline." user-invocable: true argument-hint: "[request]" allowed-tools: - Read - Bash - Grep - Glob - Skill - Task routing: triggers: - "jev router" - "route with jev" - "use the d router" not_for: "General-purpose task execution — /d classifies and dispatches, it does not perform the work itself." category: meta-tooling --- # /d — Jev Router Classifies requests through Jev and dispatches to the matched agent, skill, and pipeline. Before every dispatch, it restates the requested outcome and uses Jev to check that the restatement and route preserve it. The classification path has three layers: a deterministic `pre-route.py` force-route guard (offline, runs first, authoritative for git/security), a configured Jev transport presence check, and a two-stage classification — a cheap wide-rank stage 1 over all manifest candidates plus a trivial-bypass gate, then a full-detail shortlist-rerank stage 2 with per-candidate fit checks and stack/fan-out signals. Design rationale: `${CLAUDE_SKILL_DIR}/references/jev-classifier-design.md`. ### Phase Banners Every phase: `/d > Phase N: PHASE_NAME — description...` After intent alignment resolves: `===` routing banner. Both required. --- ### Phase 1: CLASSIFY `scripts/jev-route.py` owns the entire classification in one subprocess call. When `JEV_RESULT` is already in context (the `jev-route-injector` hook ran the script before your first token), use it and skip the command below. ```bash REQUEST_FILE=$(mktemp); printf '%s' "{user_request}" > "$REQUEST_FILE" python3 "$SDIR/jev-route.py" --request-file "$REQUEST_FILE" --json-compact rm -f "$REQUEST_FILE" ``` Resolve `$SDIR`: `${HOME}/.claude/scripts`, falling back through `.hermes`/`.factory`/`.codex`/`.reasonix`, or the repo's `scripts/` directory. Hold the result as `JEV_RESULT`. Shape (stable — see design reference for full schema): `available`, `jev_called`, `matched`, `fallback`, `fallback_reason`, `agent`, `skill`, `pipeline`, `complexity`, `confidence`, `match_type`, `reasoning`, `stack`, `signals`, `signal_scores`, `source`, `latency_ms`, `usage`, `agents`, `gate_score`, `fits_scores`, `stage1_shortlist`, and `intent_alignment` (the hook-generated baseline alignment receipt). `latency_ms` and `usage` are itemized dicts (`{"stage1_ms","stage2_ms","total_ms"}` and `{"stage1","stage2"}`). Read `.total_ms` for a single latency figure. **Gate**: `fallback == true` → Phase 1F. Every matched result, including `source == "jev-trivial-bypass"`, proceeds to Phase 2: ALIGN INTENT. --- ### Phase 1T: TRIVIAL-BYPASS (source == "jev-trivial-bypass") Stage 1's gate fired: `gate_score` below threshold, no agent/skill/pipeline is needed. It remains a direct-handling path, but it must still pass through Phase 2 so the user outcome is restated and Jev validates it. After an aligned Phase 2 result, show `Classification: Trivial` and `Source: jev-trivial-bypass`, then answer or do the one-line action directly. Do not run Phases 3–5 or call `build-dispatch.py`. Stop there. --- ### Phase 1F: UNAVAILABLE (fallback == true) Jev could not classify this request. `JEV_RESULT.source` explains why: - `unavailable` — neither configured Jev transport is available. `/d` accepts Vercel AI Gateway (`AI_GATEWAY_API_KEY`) or the direct Jev API (`TYPESAFE_API_KEY`), selected by `JEV_TRANSPORT=auto|vercel|direct`. - `invalid-pick` — Jev's pick was not a valid manifest name. - `error` — a Jev call timed out or failed. Show: ``` =================================================================== /d: Jev unavailable — [JEV_RESULT.fallback_reason] Use /do for manifest-based routing. =================================================================== ``` Fail open to `/do`'s full routing flow and continue the request. Do not reject the request merely because Vercel AI Gateway is unavailable. --- ### Phase 2: ALIGN INTENT (required for every matched /d route) **MANDATORY STOP:** For every matched `/d` invocation, write `PROPOSED_INTENT` and run the validator on that exact text before any routing banner, dispatch, answer, edit, or other action. `JEV_RESULT.intent_alignment` is only the hook baseline and does not satisfy Phase 2. This requirement has no exception for force routes, trivial routes, or an apparently aligned baseline. Before selecting the work method, write `PROPOSED_INTENT`: a concise one- or two-sentence restatement of what the user wants accomplished. State the outcome, material surfaces or deliverables, and every explicit constraint. Do not describe the selected agent, skill, or implementation mechanics as the outcome. Preserve the user's words where precision matters. Run the Jev validator even when the hook already supplied `JEV_RESULT.intent_alignment`; that receipt validates a conservative baseline, while this call validates the actual restatement that will enter the task spec. Put the request, route JSON, and proposed intent in temporary files rather than shell-splicing user text, then call: ```bash python3 "$SDIR/jev-intent-align.py" \ --request-file "$REQUEST_FILE" \ --route-file "$ROUTE_FILE" \ --proposed-intent-file "$INTENT_FILE" \ --json-compact ``` The validator sends one bounded state and all independent questions together through the selected Jev transport. It checks whether the outcome and constraints are preserved, the route can cover the material scope, the restatement is too narrow, it introduces unrequested work, and essential clarification is needed. It returns `aligned`, `clarification_needed`, `issues`, and raw `scores`. Show this before the routing banner: ``` Intent alignment (/d): -> Restated outcome: [PROPOSED_INTENT] -> Jev: [aligned|review|unavailable] [issues, if any] ``` **Gate:** - `clarification_needed == true` → ask one concise question that names the essential ambiguity; do not dispatch until answered. - `alignment == aligned` and `source == jev-trivial-bypass` → direct handling in Phase 1T; otherwise → Phase 3. - `alignment == review` because scope is lost, work was added, or the route cannot cover the request → correct `PROPOSED_INTENT` or the route and run this validator once more. Carry unresolved issues into `task_spec.gaps`; do not silently proceed as though Jev approved it. - `alignment == unavailable` or `error` → state that validation was unavailable, preserve the verbatim request and proposed intent in the task spec, then continue under the normal `/d` routing result. Gateway outage must not become a false request rejection. This runtime gate applies to every matched route, including force-routes and trivial bypasses. A Phase 1 fallback cannot run this gate because no usable Jev route exists; it fails open to `/do` as described in Phase 1F. This is an instruction gate enforced by the `/d` contract, not a hook-enforced technical boundary. The user remains the final backstop if an agent violates it. --- ### Phase 3: DECIDE (fallback == false, after aligned intent) `JEV_RESULT.source` is either `pre-route-force` (deterministic guard matched) or `jev` (Jev classification, manifest-validated). Apply directly: - `agent` / `skill` / `pipeline`: use `JEV_RESULT`'s values as-is. Already validated against the live manifest membership sets inside the script. - `complexity`: use `JEV_RESULT.complexity` when set. When `null` (always for `pre-route-force`), default to `medium`, except a single one-line trivial fix → `simple`. - Confidence: `JEV_RESULT.confidence` (`high`/`medium`/`low`). **Routing banner** (first visible output after the required intent record): ``` =================================================================== ROUTING (/d): [brief summary] =================================================================== Selected: -> Agent: [JEV_RESULT.agent] - [JEV_RESULT.reasoning] -> Skill: [JEV_RESULT.skill] - [JEV_RESULT.reasoning] -> Pipeline: [JEV_RESULT.pipeline, if set] -> Source: [JEV_RESULT.source] (confidence: [JEV_RESULT.confidence]) Invoking... =================================================================== ``` **Gate**: Agent+skill set, banner shown. Phase 4. --- ### Phase 4: ENHANCE (stack signals) `JEV_RESULT.signals` (booleans at 0.6 confidence threshold, computed by the script) map to stack entries: | Signal true | Stack | |---|---| | `tests_requested` | `test-driven-development` + `verification-before-completion` | | `research_needed` | add `research-coordinator-engineer` to agents (fan-out) | | `comprehensive_review` | `parallel-code-review` (drop if a real multi-file diff exists — `right-size-review.py` outranks it) | | `local_only` | inject `shared-patterns/local-only.md` | | `objective_loop_worthy` | `objective-loop` | `anti-rationalization-core` always rides. When `source` is `pre-route-force` and `JEV_RESULT.stack` is non-empty (e.g. `go-patterns`), keep it. **Fan-out agents**: union `JEV_RESULT.agents` (script-computed fan-out picks, each passed its per-candidate fit check) into the `research_needed` agent list, deduped. Dispatch fan-out agents as separate parallel `Agent` tool calls alongside the primary `build-dispatch.py` dispatch. **Gate**: Stack applied. Phase 5. --- ### Phase 5: EXECUTE Build the task spec with `request_verbatim` unchanged and `intent` exactly `PROPOSED_INTENT`; include any unresolved alignment issue in `gaps`, then invoke `build-dispatch.py`: ```bash python3 "$SDIR/build-dispatch.py" --json '{ "agent": "<JEV_RESULT.agent>", "skill": "<JEV_RESULT.skill; omit when agent-only>", "pipeline": "<JEV_RESULT.pipeline; omit when null>", "complexity": "<from Phase 2>", "model": "inherit", "context_mode": "summary", "provider": "<anthropic|openai|other>", "manual_model_override": false, "health": "-", "fallback_reason": "<REQUIRED when agent=general-purpose; omit otherwise>", "stack": ["s1","s2"], "task_spec": {"request_verbatim": "<user message, unchanged>", "intent": "...", "constraints": "<applicable rules, limits, and authorization>", "decisions": "...", "gaps": "...", "acceptance": "<command> -> <expected>", "files": "<owned paths; optional line ranges>", "ownership": "<worker scope>", "operator_context": "..."}, "flags": {"worktree": false, "local_only": false, "thinking_override": null}, "token_remaining": 480000 }' ``` The builder validates each name against its index, then emits the dispatch action. For Complex or creation requests, apply creation detection, plan-file gating, quality-loop, workflow dispatch, fan-out, and auto-pipeline fallback. **Gate**: Agent invoked, results delivered. --- ## Error handling Errors inside `jev-route.py` resolve to `fallback: true, source: "error"` — Phase 1F reports the error and fails open to `/do`. ## References - `${CLAUDE_SKILL_DIR}/references/jev-classifier-design.md` — request/response contract, fallback conditions, phase-by-phase design decisions - `${CLAUDE_SKILL_DIR}/SPEC.md`, `${CLAUDE_SKILL_DIR}/EVAL.md` — maintenance contract and regression cases (load only when creating, evaluating, or redesigning this skill) - `scripts/jev-route.py`, `scripts/jev-intent-align.py`, `scripts/jev_transport.py`, `scripts/jev_vercel.py`, `scripts/jev_gateway/jev_vercel_gateway.mjs`, `scripts/pre-route.py`, `scripts/routing-manifest.py`, `scripts/build-dispatch.py` - Jev hook: `hooks/jev-route-injector-userprompt.py` (UserPromptSubmit) precomputes `JEV_RESULT` -
SPEC.md 7.4 KB
# SPEC: /d — Jev-first router Maintenance contract. Load only when creating, evaluating, or redesigning this skill — not during ordinary routing (SKILL.md's phases are the runtime contract). ## Purpose Alternate entry point to `/do` that replaces the in-context routing-manifest read with bounded external Jev classification calls, cutting per-dispatch router context cost when Jev is available. Intent preservation is production behavior: the router must return the requested apple, not expand it into an orchard. ## Scope - Classify agent/skill/pipeline/complexity/stack-signals for one user request, via `scripts/jev-route.py`. - Defer entirely to `/do`'s Phase 1-4 when Jev is unavailable, a Jev call errors, or Jev names an invalid/off-manifest pick. - Execute Phase 5 (Task Spec + `build-dispatch.py`) identically to `/do`. ## Intent alignment Every matched `/d` route receives a hook-time baseline and a required runtime intent restatement. One batched Jev judgment checks that the intended outcome and constraints survive routing; it reports scope loss, added work, route mismatch, and essential ambiguity without fabricating a task. The runtime receipt for the agent's actual `PROPOSED_INTENT` satisfies the instruction gate; the hook-time baseline receipt is evidence, not a substitute for it. This is not hook enforcement, so the user remains the final backstop if an agent violates the contract. This is a production requirement grounded in repeated daily use: preventing unrequested scope expansion is the feature's primary value. If classification falls back before a route exists, intent checking cannot run and `/d` delegates to `/do` rather than pretending it was validated. ## Non-goals - Not a replacement for `/do`: `/d` is a production Jev-backed entry point, while `/do` remains the manifest-based fallback and separate default route. - Not a new manifest format, INDEX schema, or `build-dispatch.py` contract change. - Not a new telemetry/marker schema — dispatches still emit `[do-route]` via the shared `build-dispatch.py`. - Does not reimplement `/do`'s Phase 1-3 semantic reasoning in Python; it replaces that reasoning with an external classifier plus a Python-side validation/floor layer, not a rules rewrite. ## Invariants 1. `scripts/pre-route.py`'s deterministic force-route guard always runs first and is never overridden by Jev. 2. Every agent/skill/pipeline name returned to the caller is validated against the live `AGENTS:`/`SKILLS:`/`PIPELINES:` manifest membership before use. 3. Manifest-membership validation is the ONLY thing that can invalidate a primary agent/skill/pipeline pick (2026-09-16, fits-threshold removal): every name returned to the caller must be a real, shortlisted manifest entry, or that dimension is rejected to `null` -> fallback, never a guess. Jev's top-ranked Choice pick is otherwise always used, with no minimum fit score required — the earlier `--fits-threshold`-gated rejection on primary selection (default 0.30) was removed per the owner's explicit instruction ("it picks the most relevant option instead of having some .7 requirement... I don't want there to be any artificial limit"). `--fits-threshold` still exists, scoped down to gating optional fan-out-candidate inclusion only — it no longer gates primary selection. `--confidence-floor` remains accepted on the CLI for backward compatibility but has no effect on routing (superseded since the v1-to-v2 redesign, unchanged by this removal). A stage-1 gate score below `--gate-threshold` (default 0.30) is a distinct terminal state (`jev-trivial-bypass`), not a fallback, and is completely unaffected by this invariant's rewrite — it decides whether routing is needed at all, not which candidate wins among real options. 4. `JEV_TRANSPORT=auto|vercel|direct` selects the transport. Auto prefers `AI_GATEWAY_API_KEY`, then `TYPESAFE_API_KEY`. Credential values are never passed as arguments, logged, printed, or persisted. 5. `skills/meta/do/SKILL.md` and `commands/do.md` are never modified by this skill or its scripts. 6. On any fallback signal, `/d` executes `/do`'s full Phase 1-4 instructions unmodified — never a partial/degraded `/d`-only path. `jev-trivial-bypass` is not a fallback signal; it is handled directly (Phase 1T), matching `/do`'s own Trivial contract. 7. `jev-route.py` uses the shared transport selector. A missing Gateway bridge or unavailable direct API must never crash `/d`; it must fail open to `/do`. 8. Classification uses at most two Vercel AI Gateway Jev evaluations (one for a trivial bypass, two for a routed request). Every matched `/d` route adds one batched runtime intent-alignment evaluation; it never makes one request per question. The Vercel transport is a supported production dependency. ## Dependencies - `scripts/jev-route.py`, `scripts/jev_intent_align.py` (classification and alignment) - `scripts/pre-route.py`, `scripts/routing-manifest.py` (reused, unmodified) - `scripts/build-dispatch.py` (reused, unmodified — Phase 5) - Jev transport selector (`scripts/jev_transport.py`), Vercel AI Gateway bridge (`scripts/jev_vercel.py` and `scripts/jev_gateway/jev_vercel_gateway.mjs`), direct client (`scripts/jev_router_common.py`), and their credentials ## Known limitations (not blocking, tracked) - Hidden prose coupling: `jev-route.py`'s classifier instructions hand-paraphrase `/do`'s Phase 1-3 text with no automated drift check (see `references/jev-classifier-design.md` "Hidden coupling"). - Route classification has no independent verification beyond Jev's own self-reported fit scores (`/do`'s Step 0 has the orchestrator's live reasoning as an implicit check; `/d` does not have an equivalent for a confident-but-wrong pick). Since the 2026-09-16 fits-threshold removal, this is more relevant, not less: a low-fit pick is no longer deferred to `/do`, it is dispatched as-is. Monitored via regression evaluation, not solved structurally in this pass — see `references/jev-classifier-design.md` "Confident-wrong risk," strengthened (not softened) by that change. - Removing the primary fits threshold means fallback no longer acts as an uncertainty backstop for a low-fit but manifest-valid pick. The intent gate constrains outcome drift, but does not prove that route classification is optimal. - The two-stage classifier plus runtime alignment adds network latency. Vercel is preferred in `auto` mode because it is the required, currently cost-free production path; direct Jev remains the explicit alternative. ## Release criteria - The instructions require a matched route to obtain a runtime receipt for the exact proposed intent before acting, unless validation is explicitly reported unavailable and the fail-open contract is followed. - Added work, lost scope, route mismatch, and essential ambiguity exercise the alignment gates in `EVAL.md`. - Vercel `auto` preference, explicit transport isolation, retry bounds, credential redaction, and direct-transport parity remain covered. - Force-route, trivial-bypass, unavailable, error, and invalid-pick paths resolve according to their documented contracts. - `SAFETY_BUCKETS` have no critical regressions in any new full-corpus run. Historical percentages are not release criteria. Only results present in the repository and reproducible with the current corpus and manifest may be cited; missing or superseded verdict files are not evidence against production use.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.