research
Research: structured investigation, fact-checking, explanation traces.
Install
npx skills add https://github.com/notque/vexjoy-agent/tree/main/skills/research/research
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install notque-vexjoy-agent@llmmart
git clone https://github.com/notque/vexjoy-agent.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole notque/vexjoy-agent collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Research Skill
Three modes. Select by request signal:
| Signal | Mode |
|---|---|
| Formal research, investigation, sourced report, gather evidence | Research Pipeline |
| Fact check, verify claims, check facts, is this accurate, verify quote | Fact-Check |
| Why did you, explain routing, show trace, decision log, why that agent | Explanation Traces |
Default: Research Pipeline.
Mode A: Research Pipeline
Mode B: Fact-Check
Verify every factual claim in a draft before publish. Burden of proof sits on the claim, not the checker. Works standalone or as a pre-publish gate. Non-blocking: the report warns; the caller decides whether to publish.
Phase 1: EXTRACT
List every checkable claim: statistics, prices, dates, quotes, attributions, titles, event facts, rankings, causal assertions. Opinions and speculation stay out.
For each claim, record: ID, verbatim text, type (stat/quote/attribution/event/title/price/causal), location. Extract quotes verbatim for exact-words comparison.
Gate: Every checkable assertion has a claim ID. Sweep the document twice.
Phase 2: VERIFY
Work claim by claim.
- Check provided sources first. Search caller-supplied documents before external sources. Record exact passages.
- Read laterally. Judge a source by what other sources say about it, not by its own presentation.
- Climb the source tier. Follow citations upward: primary (study, filing, transcript) > direct secondary (interviews, primary reading) > derived (aggregators, rewrites). A broken citation chain caps the claim at Unverifiable.
- Triangulate contested claims. Two independent sources (separate origins, not wire rewrites). One source suffices for routine facts from a primary document.
- Verify quotes on three axes. All must hold: exact words match, attributed speaker confirmed, original context supports the meaning used.
- Check staleness. Time-sensitive claims expire. Prices/rates: 1 day-1 week. Counts: 1-3 months. Titles/roles: 3-6 months. Event status: until event date. Surveys: 6-12 months. Science: 1-3 years. Laws: 6-12 months. Records/superlatives: re-check every use. Stable history: none.
Gate: Every claim has an evidence record.
Phase 3: ADJUDICATE
Assign each claim one label:
| Label | Assign when |
|---|---|
| Verified | Evidence supports; current within staleness window; sufficient source tier |
| Disputed | Evidence contradicts; newer source supersedes; quote fails any axis |
| Unverifiable | Sources engage the claim but settle nothing |
| Missing-source | No available source addresses it |
Rules: contradiction beats support. Partial verification gets the weakest label. Stale figure superseded by newer = Disputed; merely old with no newer figure = Unverifiable.
Gate: Every claim carries one label and a one-line justification.
Phase 4: REPORT
# Fact-Check Report: [document]
## Summary
Claims: N | Verified: n | Disputed: n | Unverifiable: n | Missing-source: n
Unchecked: n (reason)
## Per-Claim Findings
### C1 -- [label]
Claim: [text] | Evidence: [source + passage] | Reasoning: [why this label]
## Warnings
[Every Disputed/Missing-source claim with correction]
## Publish Recommendation
[Hold / fix-then-publish / clear]
Every time-sensitive Verified claim carries its as-of date.
Gate: Report covers every claim ID. Warnings lists every Disputed and Missing-source finding.
Mode C: Explanation Traces
Read the per-dispatch route event log and present routing decisions as a human-readable timeline. Answer "why did I get routed here?" from recorded events only -- never from reconstruction or rationalization.
Log path: ${CLAUDE_LEARNING_DIR:-$HOME/.claude/learning}/route-events.jsonl (append-only JSONL).
Phase 1: LOCATE
LOG="${CLAUDE_LEARNING_DIR:-$HOME/.claude/learning}/route-events.jsonl"
wc -l "$LOG"
If absent or empty, report the path and the producing hook (hooks/routing-decision-recorder.py). Do not reconstruct from memory.
Phase 2: PARSE
Parse each JSON line. Two event types: DECISION (one per /do-routed dispatch) and OUTCOME (one per finalized dispatch). See references/trace-schema.md for full field semantics.
Filter to the user's query:
| User signal | Filter |
|---|---|
| Names an agent or skill | DECISION/OUTCOME events matching that name |
| "Why routed here" / latest | Most recent DECISION, current session first |
| Outcome question | OUTCOME events, joined to decisions |
| No specific target | Chronological timeline, most recent session |
Join OUTCOME to DECISION on same session AND key == "{agent}:{skill}". File adjacency is unreliable.
Phase 3: PRESENT
Sort by ts. For each decision, show: time, agent+skill, complexity, request snippet, health at decision (three states: numeric, no-weight-row, legacy), alternates, outcome.
Lead with the answer to the user's specific question, then offer surrounding context. Flag gaps honestly: pre-instrumentation entries, unmatched outcomes.
request_snippet is private session data: show to the session's user, keep out of PR bodies, issues, exports.
Deep References
| Signal | Reference | Content |
|---|---|---|
| Event field schema, health states, join rules | references/trace-schema.md |
DECISION/OUTCOME field semantics |
| Diagnosing thin trace data, consumer mistakes | references/preferred-patterns.md |
Failure mode catalog for log reading |
| Parse/read errors, missing log, unmatched outcomes | references/error-handling.md |
Error-fix mappings for trace reading |
Files (vexjoy-agent)
-
references
-
error-handling.md 7.4 KB
# Explanation Traces — Error Handling Error-fix mappings for every error state the skill encounters when reading `route-events.jsonl`. Each entry includes: what the error looks like, root cause, and the exact response to give the user. All commands are read-only. Path used throughout: ```bash LOG="${CLAUDE_LEARNING_DIR:-$HOME/.claude/learning}/route-events.jsonl" ``` --- ## Error: No Log File Found **Trigger**: `$LOG` is absent or zero lines. **Root causes**: | Cause | How to identify | |---|---| | No /do-routed dispatch recorded yet | `~/.claude/learning/` exists but has no `route-events.jsonl` | | Hooks merged but never synced to `~/.claude` | `ls ~/.claude/hooks/ \| grep routing-decision-recorder` returns nothing | | `CLAUDE_LEARNING_DIR` points elsewhere | `echo "$CLAUDE_LEARNING_DIR"` is set; check that directory instead | **Detection** (run to diagnose which cause applies): ```bash echo "resolved: $LOG"; ls -la "$LOG" 2>/dev/null echo "env: CLAUDE_LEARNING_DIR=${CLAUDE_LEARNING_DIR:-<unset>}" ls ~/.claude/hooks/ 2>/dev/null | grep -E "routing-(decision-recorder|outcome-finalizer)" ``` **Response to user**: ``` No route event log found at ~/.claude/learning/route-events.jsonl (or $CLAUDE_LEARNING_DIR/route-events.jsonl when that variable is set). The log is created on the first /do-routed dispatch by the routing-decision-recorder hook (hooks/routing-decision-recorder.py). An empty or missing log means no /do-routed dispatch has been recorded yet — or merged hook changes were never synced to ~/.claude; run hooks/sync-to-user-claude.py or restart the session. ``` Read decisions from the log only. If no file exists, report that there is nothing to read. --- ## Error: Malformed JSONL Line **Trigger**: `json.JSONDecodeError` on an individual line. **Root causes**: | Cause | Symptom | |---|---| | Truncated append (process killed mid-write; rare — per-line appends are atomic) | Last line of the file is a JSON prefix | | Manual edit | Bad line anywhere in the file | **Detection**: ```bash python3 - "$LOG" <<'EOF' import json, sys bad = [] for i, line in enumerate(open(sys.argv[1]), 1): line = line.strip() if not line: continue try: json.loads(line) except json.JSONDecodeError as e: bad.append((i, e.msg)) print("malformed lines:", bad or "none") EOF ``` **Response to user**: ``` route-events.jsonl has [N] unparseable line(s): [line numbers]. Skipping them; [M] valid events remain and are shown below. ``` JSONL fails per line, never whole-file: skip each bad line, keep every parseable event, and report the skip count. Recovery of a truncated final line is unnecessary — the contract loses at most that one event. --- ## Error: Log Has No Decision Events **Trigger**: File parses, but zero lines have `"type": "decision"`. **Detection**: ```bash python3 - "$LOG" <<'EOF' import json, sys, collections c = collections.Counter(json.loads(l)["type"] for l in open(sys.argv[1]) if l.strip()) print("counts by type:", dict(c)) EOF ``` **Root causes**: | Symptom | Likely cause | |---|---| | Only `outcome` events | Recorder hook missing from `~/.claude/hooks/` while the finalizer is present | | File empty | No /do-routed dispatch since the log was created | | Dispatches happened but nothing recorded | Recorder skips dispatches without a `[do-route]` marker (deliberate: keeps route-health's denominator honest) — the dispatches were not /do-routed, or the marker is malformed | **Response to user**: ``` route-events.jsonl exists with [N] outcome event(s) and zero decision events. Decisions are appended by hooks/routing-decision-recorder.py (PostToolUse on Agent dispatch) and only for /do-routed dispatches carrying a [do-route] marker. Check that the recorder is synced to ~/.claude/hooks/ and that the dispatches you expect were /do-routed. ``` --- ## Error: Decision Has No Matched Outcome **Trigger**: A DECISION event has no OUTCOME with the same `session` and `key == "{agent}:{skill}"`. **Root causes**: | Cause | How to identify | |---|---| | Pending — finalizer has not run yet | Decision is recent; the finalizer fires on the next user prompt | | Session ended before finalization | Decision `ts` is old and its session has no later events | | Finalizer dropped it (stale, past max pending age) | Old decision, no outcome ever appears | **Response to user**: Label the entry `pending — not yet finalized` (recent) or `never finalized` (old). Outcomes arrive on a later user prompt, so a missing outcome right after a dispatch is normal, never an error. Report the recorded state; leave the outcome unset rather than inferring one. --- ## Error: Absent Additive Fields **Trigger**: A decision lacks `n`/`failure`/`action`/`alternates` or `gate_inputs_present`; an outcome lacks `reason` or `routing_relevant`. **Cause**: The schema grew append-compatibly — lines written before a field shipped simply lack it. Also, `n`/`failure`/`action`/`alternates` stay `null` unless a real numeric `health=` was read (see trace-schema.md, health states). **Response to user**: Treat absence as "not recorded then". Flag counts: ``` Note: [N] decision(s) predate the health-gate instrumentation — they show WHAT was routed but carry no health data. ``` Present the fields that exist; leave the rest as unknown rather than inventing values. --- ## Error: User Asks About a Dispatch Not in the Log **Trigger**: User asks "why did I get routed to X?" but no DECISION matches X in `agent`, `skill`, or `alternates`. **Detection logic**: ```python query = "golang-general-engineer" matches = [d for d in decisions if query in d.get("agent", "") or query in d.get("skill", "") or query in (d.get("alternates") or [])] ``` **Root causes**: | Cause | Explanation | |---|---| | Dispatch was not /do-routed | Manual Agent calls carry no `[do-route]` marker; the recorder skips them | | Nested fan-out | The recorder records top-level /do-routed dispatches only — recording nested sub-dispatches would inflate route-health's denominator | | Different session or older than the log | Filter widened to all sessions still finds nothing | **Response to user**: ``` No decision event found for [X]. The log records /do-routed top-level dispatches only. This session has [N] recorded decision(s): [list agent+skill pairs]. The dispatch you asked about was either not /do-routed or was a nested sub-dispatch, which the recorder deliberately excludes. ``` Show what IS in the log. Leave unrecorded dispatches unexplained rather than speculating. --- ## Error-Fix Summary Table | Error | Root cause | User message keyword | Fix | |---|---|---|---| | No log file | No dispatch yet, hooks unsynced, or env redirect | "No route event log found" | Name real path + producing hook; suggest sync-to-user-claude.py | | Malformed line | Truncated append or manual edit | "unparseable line(s)" | Skip per line; report count; keep valid events | | No decision events | Recorder unsynced or dispatches not /do-routed | "zero decision events" | Check recorder in ~/.claude/hooks/; confirm [do-route] marker | | Unmatched outcome | Pending or never finalized | "pending — not yet finalized" | Label the state; outcomes arrive on a later prompt | | Absent additive fields | Pre-instrumentation lines | "predate the health-gate instrumentation" | Treat as unknown; flag counts; keep values as recorded | | Dispatch not in log | Not /do-routed or nested fan-out | "not /do-routed" | Show recorded decisions; explain the exclusion | -
preferred-patterns.md 6.5 KB
# Explanation Traces — Patterns to Fix Failure modes when reading `route-events.jsonl`, organized by kind: consumer mistakes (this skill's behavior) and producer-side diagnostics (recognizing why the recorded data is thin). The producers are fixed toolkit code (`hooks/routing-decision-recorder.py`, `hooks/routing-outcome-finalizer.py`, both writing through `hooks/lib/route_events.py`) — thin data usually means an older log line or an uninstrumented marker, and the fix is correct interpretation, never editing the log. Path used throughout: ```bash LOG="${CLAUDE_LEARNING_DIR:-$HOME/.claude/learning}/route-events.jsonl" ``` --- ## Consumer Patterns to Fix (Skill Behavior) ### AP-1: Reconstructing Decisions from Conversation History **What it looks like** (incorrect skill behavior): > "The log doesn't exist, but based on the conversation I can tell the router > probably chose the golang agent because the user mentioned Go..." **Why wrong**: This is exactly the rationalization the skill exists to prevent. The recorded event is the only evidence of what the router saw at decision time. **Correct behavior**: Report the missing log, name the real path and producing hook, and stop. See SKILL.md Phase 1 Step 2 for the exact message. --- ### AP-2: Joining Outcomes by File Adjacency **What it looks like**: Pairing an OUTCOME event with the DECISION on the line above it. **Why wrong**: Appends from parallel sessions interleave at line granularity. The outcome above may belong to a different session's dispatch. An outcome also lands minutes-to-hours after its decision — anything can sit between them. **Correct behavior**: Match on same `session` AND `key == f"{agent}:{skill}"`. Verify join quality: ```bash python3 - "$LOG" <<'EOF' import json, sys dec, out = {}, set() for line in open(sys.argv[1]): if not line.strip(): continue e = json.loads(line) if e["type"] == "decision": dec[(e["session"], f"{e['agent']}:{e['skill']}")] = dec.get((e["session"], f"{e['agent']}:{e['skill']}"), 0) + 1 else: out.add((e["session"], e["key"])) matched = sum(1 for k in dec if k in out) print(f"decision keys: {len(dec)}, with matched outcome: {matched}") EOF ``` --- ### AP-3: Conflating the Three Health States **What it looks like**: Rendering every `null` `health_at_decision` as "no health data". **Why wrong**: `null` carries two different facts, split by `gate_inputs_present`: - `true` → instrumented, but the pick had no weight row (a new pair — valid, expected) - `false`/absent → legacy marker, health never read Collapsing them hides whether the health gate ran. **Correct behavior**: Render the three states distinctly (SKILL.md Phase 3 Step 1 table). Count each state: ```bash python3 - "$LOG" <<'EOF' import json, sys, collections c = collections.Counter() for line in open(sys.argv[1]): if not line.strip(): continue e = json.loads(line) if e["type"] != "decision": continue if e.get("health_at_decision") is not None: c["(a) numeric"] += 1 elif e.get("gate_inputs_present"): c["(b) no weight row"] += 1 else: c["(c) legacy/uninstrumented"] += 1 print(dict(c)) EOF ``` --- ### AP-4: Treating Absent Additive Fields as Corruption **What it looks like**: Flagging lines without `gate_inputs_present` or `reason` as malformed. **Why wrong**: The schema grew append-compatibly; old lines lack new fields by design. `n`/`failure`/`action`/`alternates` are also `null` on any dispatch where no numeric `health=` was read. **Correct behavior**: Absent = "not recorded then". Flag the count in the timeline note; keep the entries. --- ### AP-5: Sorting by File Order Instead of `ts` **What it looks like**: Presenting events in the order they appear in the file. **Why wrong**: Concurrent appends interleave; a slow hook can land its line after a faster one with an earlier `ts`. File order approximates time but does not guarantee it. **Correct behavior**: Sort numerically on `ts` (a float, epoch seconds), then group by session for display. --- ### AP-6: Dumping Raw JSONL Without Filtering **What it looks like**: Printing raw log lines when the user asks a specific question like "why did I get the governance agent?". **Why wrong**: A 500-event log printed raw is noise, not an explanation. **Correct behavior**: Filter to the matching decision, lead with the answer, offer the session timeline as supplementary context. See SKILL.md Phase 2 Step 2 for the filter table. --- ### AP-7: Leaking `request_snippet` Outside the Session **What it looks like**: Pasting `request_snippet` values into a PR body, issue, or exported report while demonstrating the skill. **Why wrong**: The snippet is the first 200 chars of a real user request — private session data. Showing it to the session's own user is the skill's job; sending it elsewhere is a leak. **Correct behavior**: Inside the session, quote snippets freely. In anything that leaves the session, report counts and field-presence statistics only. --- ## Producer-Side Diagnostics (Why the Data Is Thin) ### AP-8: Missing Decisions — the Dispatch Was Never Recorded **Detection**: ```bash ls ~/.claude/hooks/ | grep -E "routing-(decision-recorder|outcome-finalizer)" ``` **Interpretation**: The recorder appends a DECISION only for /do-routed dispatches carrying a `[do-route]` marker; manual Agent calls and nested fan-out are deliberately excluded (they would inflate route-health's denominator). Recorder absent from `~/.claude/hooks/` means merged hook changes were never synced — run `hooks/sync-to-user-claude.py`. --- ### AP-9: High Legacy Rate — Markers Without `health=` **Detection**: Run the AP-3 state counter. A large "(c) legacy/uninstrumented" share among *recent* decisions means current markers lack the `health=` token. **Interpretation**: `gate_inputs_present` is the signal the decommission clock reads. State (c) on old lines is history; state (c) on new lines means the router's Step-1.5 wiring regressed — worth a report against `skills/meta/do/SKILL.md` Phase 4, never a log edit. --- ## Quick Detection Cheatsheet | Question | Command | |---|---| | Event counts by type | `python3 -c "import json,collections,os;p=os.path.expandvars('$LOG');print(collections.Counter(json.loads(l)['type'] for l in open(p) if l.strip()))"` | | Malformed lines | see error-handling.md, "Malformed JSONL Line" | | Health-state split | AP-3 counter above | | Decisions with matched outcomes | AP-2 join checker above | | Recorder synced | `ls ~/.claude/hooks/ \| grep routing-decision-recorder` | -
trace-schema.md 7.4 KB
# route-events.jsonl Schema Event schema for `<CLAUDE_LEARNING_DIR>/route-events.jsonl` (default `~/.claude/learning/route-events.jsonl`), the per-dispatch decision log consumed by the `explanation-traces` skill. Source of truth: `hooks/lib/route_events.py` — when this document and that module disagree, the module wins. --- ## File Contract | Property | Value | |---|---| | Format | JSONL — one JSON object per line, compact separators, UTF-8 | | Write mode | Append-only; POSIX per-line appends are atomic, so concurrent dispatches interleave at line granularity without a lock | | Path | `<CLAUDE_LEARNING_DIR>/route-events.jsonl`; env var unset means `~/.claude/learning/` | | Failure mode | Failure-safe by contract: a write error is swallowed — worst case one lost event line, never a broken hook | | Authority | Auxiliary instrumentation; the aggregate routing rows in learning.db remain authoritative | Why the log exists: the aggregate rows are keyed `(topic, key)` and carry no per-dispatch history, so faithful offline replay of "request → route → outcome" is impossible from them alone. This log adds that history. Two producers: | Producer | Hook event | Writes | |---|---|---| | `hooks/routing-decision-recorder.py` | PostToolUse (Agent) | DECISION event when it records a /do-routed dispatch | | `hooks/routing-outcome-finalizer.py` | UserPromptSubmit | OUTCOME event when it finalizes a pending dispatch | --- ## DECISION Event One per /do-routed dispatch, captured from the `[do-route]` marker at record time — never back-filled from later weights. **Example** (illustrative values): ```json {"type":"decision","ts":1751400000.123,"session":"abc123","request_snippet":"fix the failing router test","agent":"python-general-engineer","skill":"pr-workflow","complexity":"Medium","health_at_decision":0.62,"n":7,"failure":1,"action":"keep","alternates":["python-general-engineer:python-quality-gate"],"gate_inputs_present":true} ``` | Field | Type | Semantics | |-------|------|-----------| | `type` | string | Always `"decision"`. | | `ts` | float | Epoch seconds (`time.time()`) when the event was appended. | | `session` | string | Session id; `""` when unknown. | | `request_snippet` | string | First 200 chars of the routed request. Private session data — keep out of PR bodies, issues, exports. | | `agent` | string | Dispatched agent; `""` when unknown. | | `skill` | string | Paired skill; `""` when unknown. | | `complexity` | string | Complexity class from the marker; `""` when absent. | | `health_at_decision` | float or null | The picked pair's confidence at decision time; `null` when the pair had no weight row or health was never evaluated — `gate_inputs_present` disambiguates (see states below). | | `n` | int or null | Dispatch count for the pair's weight row — a demote-floor input. `null` unless a real numeric `health=` was read. | | `failure` | int or null | Failure count for the pair's weight row — a demote-floor input. Same null rule as `n`. | | `action` | string or null | The Step-1.5 health-gate outcome: `keep`, `demote`, or `tiebreak`. | | `alternates` | list of strings or null | The keys offered as alternatives; `null` when none recorded. | | `gate_inputs_present` | bool | Instrumentation signal the decommission clock reads (see states below). Additive; old readers ignore it, old lines lack it. | The demote floor needs all three gate inputs — `confidence < 0.30 AND failure >= 3 AND n >= 5` — which is why `n` and `failure` are snapshotted alongside health: confidence alone cannot reconstruct the floor. ### The Three Health States The `[do-route]` marker may carry a `health=` token. Its shape at record time yields three states the event must distinguish: | State | Marker | `health_at_decision` | `gate_inputs_present` | Meaning | |---|---|---|---|---| | (a) numeric | `health=<float>` | the float | `true` | Health evaluated from a weight row | | (b) no-row | `health=-` | `null` | `true` | Instrumented, but the pick had no weight row (valid expected data — e.g. a new pair) | | (c) legacy | no `health=` token | `null` | `false` (or field absent on old lines) | Never instrumented — legacy or missing wiring. A malformed `health=` value also lands here | `gate_inputs_present` exists because `null` health alone cannot distinguish (b) from (c). `n`, `failure`, `action`, and `alternates` stay `null` unless a real numeric `health=` was read (state a). --- ## OUTCOME Event One per finalized dispatch. Outcome resolution happens on a later user prompt, so an outcome's `ts` is minutes-to-hours after its decision's. **Example** (illustrative values): ```json {"type":"outcome","ts":1751400900.456,"session":"abc123","key":"python-general-engineer:pr-workflow","outcome":"success","reason":"acceptance","routing_relevant":true} ``` | Field | Type | Semantics | |-------|------|-----------| | `type` | string | Always `"outcome"`. | | `ts` | float | Epoch seconds when the outcome was finalized. | | `session` | string | Session id; `""` when unknown. | | `key` | string | Routing key `{agent}:{skill}`; agent-only `{agent}:` when the skill is unknown. | | `outcome` | string | One of `success`, `failure`, `neutral`. | | `reason` | string, optional | Short cause for the outcome, free of prompt text and secrets. Written only when given — absent in older events. Finalizer values: `tool-errors`, `rejection`, `acceptance`, `reaction-ignored-multi-dispatch`, `neutral-new-topic`; other producers may write other short strings. | | `routing_relevant` | bool, optional | Marks whether the outcome is a routing signal the confidence loop acts on; route-value-eval counts only routing-relevant failures. Written only when asserted — absent means relevance was not asserted. | --- ## Joining Outcomes to Decisions Match on **both** conditions: 1. Same `session` 2. Outcome `key` equals the decision's `f"{agent}:{skill}"` File adjacency is unreliable: parallel sessions interleave lines. A decision with no matching outcome is pending (finalizer runs on the next user prompt) or was never finalized. --- ## Additive-Field History The schema grew append-compatibly. Older lines lack newer fields: | Fields | Absent on | |---|---| | `n`, `failure`, `action`, `alternates` | Decisions recorded before the health-gate inputs shipped | | `gate_inputs_present` | Decisions recorded before the decommission-clock signal shipped | | `reason`, `routing_relevant` | Outcomes from older or relevance-neutral callers | Consumers treat an absent field as "not recorded then" — identical handling to `null` for health, and never a validity error. --- ## Validation Checklist A healthy log satisfies all of these: - [ ] Every line parses as a standalone JSON object - [ ] Every event has `type` in `{decision, outcome}` and a float `ts` - [ ] Every decision has `agent`, `skill`, `request_snippet` keys - [ ] Every outcome has `key` and `outcome` in `{success, failure, neutral}` - [ ] `health_at_decision` is numeric or `null` — a `null` with `gate_inputs_present: true` is valid data, not a defect Detection (read-only, counts only): ```bash LOG="${CLAUDE_LEARNING_DIR:-$HOME/.claude/learning}/route-events.jsonl" python3 - "$LOG" <<'EOF' import json, sys, collections c = collections.Counter(); bad = [] for i, line in enumerate(open(sys.argv[1]), 1): line = line.strip() if not line: continue try: e = json.loads(line) c[e.get("type", "?")] += 1 except json.JSONDecodeError: bad.append(i) print("by type:", dict(c), "malformed lines:", bad or "none") EOF ```
-
-
SKILL.md 6.6 KB
--- name: research description: "Research: structured investigation, fact-checking, explanation traces." user-invocable: true argument-hint: "<research topic or claim to verify>" agent: research-coordinator-engineer context: fork allowed-tools: - Read - Bash - Glob - Grep - Agent - Write - WebFetch - WebSearch routing: force_route: true not_for: "code review (use review), security audit (use security)" triggers: - "research-pipeline" - "research" - "formal research" - "research with artifacts" - "systematic investigation" - "research report" - "gather evidence" - "fact check" - "fact-check" - "verify claims" - "check facts" - "verify this quote" - "is this accurate" - "is this true" - "check this claim" - "verify this" - "are these numbers right" - "why did you" - "explain routing" - "show trace" - "decision log" - "why that agent" - "explain decision" - "show decisions" - "trace log" category: research pairs_with: - review - writing --- # Research Skill Three modes. Select by request signal: | Signal | Mode | |--------|------| | Formal research, investigation, sourced report, gather evidence | Research Pipeline | | Fact check, verify claims, check facts, is this accurate, verify quote | Fact-Check | | Why did you, explain routing, show trace, decision log, why that agent | Explanation Traces | Default: Research Pipeline. --- ## Mode A: Research Pipeline --- ## Mode B: Fact-Check Verify every factual claim in a draft before publish. Burden of proof sits on the claim, not the checker. Works standalone or as a pre-publish gate. Non-blocking: the report warns; the caller decides whether to publish. ### Phase 1: EXTRACT List every checkable claim: statistics, prices, dates, quotes, attributions, titles, event facts, rankings, causal assertions. Opinions and speculation stay out. For each claim, record: ID, verbatim text, type (stat/quote/attribution/event/title/price/causal), location. Extract quotes verbatim for exact-words comparison. **Gate**: Every checkable assertion has a claim ID. Sweep the document twice. ### Phase 2: VERIFY Work claim by claim. 1. **Check provided sources first.** Search caller-supplied documents before external sources. Record exact passages. 2. **Read laterally.** Judge a source by what other sources say about it, not by its own presentation. 3. **Climb the source tier.** Follow citations upward: primary (study, filing, transcript) > direct secondary (interviews, primary reading) > derived (aggregators, rewrites). A broken citation chain caps the claim at Unverifiable. 4. **Triangulate contested claims.** Two independent sources (separate origins, not wire rewrites). One source suffices for routine facts from a primary document. 5. **Verify quotes on three axes.** All must hold: exact words match, attributed speaker confirmed, original context supports the meaning used. 6. **Check staleness.** Time-sensitive claims expire. Prices/rates: 1 day-1 week. Counts: 1-3 months. Titles/roles: 3-6 months. Event status: until event date. Surveys: 6-12 months. Science: 1-3 years. Laws: 6-12 months. Records/superlatives: re-check every use. Stable history: none. **Gate**: Every claim has an evidence record. ### Phase 3: ADJUDICATE Assign each claim one label: | Label | Assign when | |-------|-------------| | **Verified** | Evidence supports; current within staleness window; sufficient source tier | | **Disputed** | Evidence contradicts; newer source supersedes; quote fails any axis | | **Unverifiable** | Sources engage the claim but settle nothing | | **Missing-source** | No available source addresses it | Rules: contradiction beats support. Partial verification gets the weakest label. Stale figure superseded by newer = Disputed; merely old with no newer figure = Unverifiable. **Gate**: Every claim carries one label and a one-line justification. ### Phase 4: REPORT ``` # Fact-Check Report: [document] ## Summary Claims: N | Verified: n | Disputed: n | Unverifiable: n | Missing-source: n Unchecked: n (reason) ## Per-Claim Findings ### C1 -- [label] Claim: [text] | Evidence: [source + passage] | Reasoning: [why this label] ## Warnings [Every Disputed/Missing-source claim with correction] ## Publish Recommendation [Hold / fix-then-publish / clear] ``` Every time-sensitive Verified claim carries its as-of date. **Gate**: Report covers every claim ID. Warnings lists every Disputed and Missing-source finding. --- ## Mode C: Explanation Traces Read the per-dispatch route event log and present routing decisions as a human-readable timeline. Answer "why did I get routed here?" from recorded events only -- never from reconstruction or rationalization. **Log path**: `${CLAUDE_LEARNING_DIR:-$HOME/.claude/learning}/route-events.jsonl` (append-only JSONL). ### Phase 1: LOCATE ```bash LOG="${CLAUDE_LEARNING_DIR:-$HOME/.claude/learning}/route-events.jsonl" wc -l "$LOG" ``` If absent or empty, report the path and the producing hook (`hooks/routing-decision-recorder.py`). Do not reconstruct from memory. ### Phase 2: PARSE Parse each JSON line. Two event types: DECISION (one per /do-routed dispatch) and OUTCOME (one per finalized dispatch). See `references/trace-schema.md` for full field semantics. Filter to the user's query: | User signal | Filter | |-------------|--------| | Names an agent or skill | DECISION/OUTCOME events matching that name | | "Why routed here" / latest | Most recent DECISION, current session first | | Outcome question | OUTCOME events, joined to decisions | | No specific target | Chronological timeline, most recent session | Join OUTCOME to DECISION on same `session` AND `key == "{agent}:{skill}"`. File adjacency is unreliable. ### Phase 3: PRESENT Sort by `ts`. For each decision, show: time, agent+skill, complexity, request snippet, health at decision (three states: numeric, no-weight-row, legacy), alternates, outcome. Lead with the answer to the user's specific question, then offer surrounding context. Flag gaps honestly: pre-instrumentation entries, unmatched outcomes. `request_snippet` is private session data: show to the session's user, keep out of PR bodies, issues, exports. --- ## Deep References | Signal | Reference | Content | |--------|-----------|---------| | Event field schema, health states, join rules | `references/trace-schema.md` | DECISION/OUTCOME field semantics | | Diagnosing thin trace data, consumer mistakes | `references/preferred-patterns.md` | Failure mode catalog for log reading | | Parse/read errors, missing log, unmatched outcomes | `references/error-handling.md` | Error-fix mappings for trace reading |
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.