# Media Claim Ledger

## Brief Boundary

- Decision and objective:
- Output contract / target:
- Rights, privacy, retention, and sharing boundary:
- Decision granularity:
- Required sampling coverage:
- Human review points:

## Claim Closure

| Claim ID | Exact claim and scope | Evidence class | Source URL / locator | Access date | Tool/build and experiment | Artifact/interval | Result | Limitation / counter-evidence | Durable artifacts | Status |
|---|---|---|---|---|---|---|---|---|---|---|
| media-001 | | documented / reproduced / observed / environment-specific / heuristic / unresolved | | | | | | | | open |

## Sampling Audit

| Claim ID | Population / interval claimed | Sampling method | Covered timestamps / streams | Transformations | Blind spots | Supports claim scope? | Next evidence or reviewer |
|---|---|---|---|---|---|---|---|
| media-001 | | | | | | yes / no / partial | |

## Rejected and Untested Sources

| Source / proposed test | Rejected / inaccessible / redundant / untested / blocked | Reason | Consequence for confidence | Revisit trigger |
|---|---|---|---|---|
| | | | | |

## Completion Gate

- [ ] Current technical claims include source URLs and access dates.
- [ ] Reproduced claims include exact versions/builds, fixtures, commands, probes, and raw evidence.
- [ ] Every sample states its stream/interval, selection method, transformations, coverage, and blind spots.
- [ ] No whole-asset or editorial conclusion is based only on sparse technical evidence.
- [ ] Technical findings and attributed editorial approval are separate.
- [ ] Each retained claim links to durable artifacts; exclusions and unresolved questions are logged.

# Research Brief

## Quick Reference

- **Research question:**
- **Depth:** Scan / Light / Moderate / Deep
- **Commissioned by:**
- **Date:**

---

## 1. Executive Summary

*One paragraph answering the research question. Someone who reads only this should understand what you found and how confident you are.*

---

## 2. Key Findings

*3-5 synthesized findings. Each with a confidence tag.*

### Finding 1: [Title]
[2-3 sentence synthesis of what the evidence says]
**Confidence:** High / Medium / Low
**Sources:** [Link 1](), [Link 2]()

### Finding 2: [Title]
...
**Confidence:** ...
**Sources:** ...

---

## 3. Evidence Table

| Finding | Supporting sources | Contradicting sources | Key evidence |
|---------|-------------------|-----------------------|--------------|
| Finding 1 | Source A, Source B | Source C | The benchmark showing X |
| Finding 2 | Source D, Source E | — | The case study of Y |

---

## 4. Confidence Assessment

| Dimension | Assessment |
|-----------|-----------|
| **Source quality** | How reliable are the sources? (CRAAP scores) |
| **Convergence** | How many independent sources agree? |
| **Coverage** | Are there gaps in the evidence? |
| **Stability** | Would new information likely change the conclusion? |

**Overall confidence:** High / Medium / Low

---

## 5. Open Questions

*What you still don't know. Every research project should have these.*

- [ ] [Question 1]
- [ ] [Question 2]

---

## 6. Sources

| # | Title | URL | Type | Key contribution |
|---|-------|-----|------|-----------------|
| 1 | | | Paper / Blog / Doc | |
| 2 | | | | |

---

## 7. Durable Artifact Inventory

*A research brief is a synthesis, not the only deliverable. List the long-lived artifacts created or updated so a future user or agent can find the source-level evidence and reusable claims.*

| Artifact | Type | Destination / path | What it preserves | Source coverage |
|----------|------|--------------------|-------------------|-----------------|
| | Evidence record / claim record / synthesis / dataset / other | | | Source # / source URL |

### Preservation check

- [ ] Every retained source has a durable evidence record or an explicit exclusion decision in the research log.
- [ ] Every material reusable claim, contradiction, and open question has been extracted at an appropriate granularity.
- [ ] Every artifact preserves provenance, access date, and relationships to the research question and synthesis.
- [ ] No essential evidence exists only in chat or a temporary workspace.

# Research Log

A traceable record of every search, source, and decision made during a research engagement.

## Search History

| # | Query | Engine | Date | Result count | Notes |
|---|-------|--------|------|-------------|-------|
| 1 | | Search tool / source name | | | |
| 2 | | | | | |

## Sources Considered

| # | Title | URL | CRAAP score | Decision | Reason |
|---|-------|-----|-------------|----------|--------|
| 1 | | | /25 | Keep / Reject | |
| 2 | | | /25 | Keep / Reject | |

## Citation Chain

*Which sources led to which other sources.*
- Source A → cited Source D, Source E
- Source D → cited Source F

## Key Decisions

| Decision | Rationale | Date |
|----------|-----------|------|
| Included/excluded topic X | | |
| Stopped searching after N sources | | |
| Changed research question | | |

## Extraction and Preservation Ledger

*Research is incomplete until retained evidence is preserved in the host's durable research surface. Track every source so that extraction is exhaustive rather than driven by a convenient round number.*

| Source | Retain / reject | Material claims, data, or contradictions extracted | Durable artifact(s) | Destination / path | Reason if not preserved |
|--------|-----------------|---------------------------------------------------|---------------------|--------------------|-------------------------|
| Source A | Retain | | | | |
| Source B | Reject | | | | |

## Completion Gate

- [ ] Each retained source appears in the preservation ledger.
- [ ] All material reusable findings are represented by durable, navigable artifacts, not only the final report.
- [ ] The synthesis links to the evidence artifacts it depends on.
- [ ] Rejected or inaccessible sources have a recorded reason.

{
  "schema_version": 1,
  "skill_name": "research-methodology",
  "evals": [
    {
      "id": "academic-lifecycle-scoping-to-report",
      "prompt": "We're deciding whether to bet our roadmap on local LLM inference on consumer GPUs. Do comprehensive research on what architectures people actually use for this today and give me something I can defend in front of the team — not a vibe check.",
      "expected_output": "A comprehensive-track deliverable that begins with scoping, not searching: the core question framed as a single falsifiable question, explicit inclusion/exclusion criteria set before any query (date range appropriate for a fast-moving field, acceptable source types, authority threshold, duplication rule), and the depth declared (comprehensive, not a scan). The search strategy starts broad to map the territory, then narrows using multiple query angles (keyword, tool name, problem statement) and citation chaining both backward (to sources the authors relied on) and forward (cited-by). Every retained source is recorded with title + URL, author/source, date, key claims, supporting evidence, gaps/limitations, and connection to the brief. Credibility is assessed before deep work (CRAAP dimensions; reliability tiers — peer-reviewed and official documentation rank above forum posts). Findings are synthesized thematically with confidence levels tied to triangulation: 3+ independent agreeing sources is high confidence, a lone contradicting source is investigated rather than ignored, and single-source key claims are flagged as such. The final brief has the canonical structure: executive summary, 3-5 key findings with confidence levels, an evidence table mapping sources to findings, a confidence assessment separating solid from uncertain from missing, explicit open questions, and full citations with URLs. Unresolved gaps are reported as gaps — the deliverable never implies exhaustive coverage it did not achieve.",
      "assertions": [
        "Scoping precedes searching: a falsifiable core question plus explicit inclusion/exclusion criteria (date range, source types, authority threshold) are defined up front",
        "Search strategy starts broad, uses multiple query angles, and chains citations backward and forward rather than relying on one query",
        "Each retained source is recorded with title + URL, author, date, key claims, supporting evidence, gaps, and connection to the brief",
        "Confidence levels are tied to triangulation counts, with single-source key claims flagged rather than presented as established fact",
        "The report follows the structured brief format and ends with explicit open questions / unresolved gaps instead of implying completeness"
      ]
    },
    {
      "id": "journalistic-prepublication-verification",
      "prompt": "I've drafted an investigative piece saying a startup's founder misled investors about user numbers. It rests mostly on one anonymous former employee plus two articles that covered the same rumor. Editor wants it out tomorrow. What has to happen before this ships?",
      "expected_output": "The piece does not ship on this evidence base. A damaging accusation triggers the journalistic three-source rule: three genuinely independent sources — different people, documents, or methodologies — not three accounts echoing each other; the two articles citing the same rumor count once at best. With only one person of direct knowledge, the fallback is reporting the confidence gap explicitly ('only one person with direct knowledge would speak, but their account was consistent with internal documents reviewed by ...') and corroborating specifics against tier-1 primary documents (emails, memos, filings, commit history). Anonymous-source handling follows the named-source standard: establish why anonymity is requested, record the source's identity internally even though readers cannot see it, corroborate verifiable claims, and disclose as much as possible ('a current employee, speaking anonymously because they were not authorized...'). Before publication, run the 7-step verification protocol: every quote read word-for-word against the recording or notes (not memory), every link in the draft opened and numbers matched, every surprising claim triangulated with at least two sources, statistics recency-checked, paraphrase drift audited so a hedged source statement is never upgraded in strength, remaining single-source claims flagged openly in the draft, and the 'what if I'm wrong' test applied hardest to the most damaging allegations. If independence or corroboration cannot be reached by deadline, the claim is cut or reframed — deadline pressure changes timing, not the evidence bar.",
      "assertions": [
        "The damaging accusation invokes the three-independent-sources rule and identifies the two echo articles as non-independent",
        "The single-direct-source shortfall is handled by reporting the confidence gap and corroborating against primary documents",
        "Anonymous-source discipline is followed: reason established, identity known to the researcher, claims corroborated, attribution disclosed as far as possible",
        "Pre-publication protocol includes quotes verified word-for-word against recordings/notes and every draft link opened",
        "Paraphrase drift is audited so source strength is never inflated (e.g., 'suggests' never becomes 'proves')",
        "The 'what if I'm wrong' test concentrates hardest verification on the most damaging claims, with unresolved single-source claims flagged or cut"
      ]
    },
    {
      "id": "industry-signal-filter-evidence-standard",
      "prompt": "A major cloud vendor just changed its AI pricing to usage-based. Tech Twitter is losing its mind. Does this actually matter for our enterprise spend strategy, and give me an analysis worth reading — not another news recap.",
      "expected_output": "An industry-analysis piece, not a recap. Step one is reading the actual announcement or filing directly — not the coverage of it — then running the signal filter: does this change competitive position, cost structure, who can build what, or open a previously closed market? Noise stops there; signal proceeds through the 'so what' test asked three times until it reaches a decision-level implication (e.g., usage-based pricing shifts TCO calculations for every enterprise evaluating these tools — that is the story). Claims are grounded in corporate evidence standards: percentages about industry behavior require a named survey firm with linked methodology, trend claims need multiple consistent data points from independent sources, and executive rhetoric ('we see a path to...') is separated from committed action — evasive answers and comparison baselines are read as signals. Analysis follows the money: capital flows, talent movement, infrastructure spending, and regulatory attention reveal strategy more honestly than press releases. A competitor reaction scan asks who responded and what silence means. Conclusions obey the one-company rule: one company is an anecdote, two to three show a pattern, four or more across segments justify 'the industry shows' framing — so a single vendor's move is framed accordingly, not extrapolated into a law of nature.",
      "assertions": [
        "The primary announcement or filing is read directly before relying on secondary coverage",
        "The signal filter and triple 'so what' drill land on a decision-level implication for enterprise buyers rather than restating the news",
        "Statistical claims meet the case-study evidence standard (named survey firm, linked methodology) and executive language is distinguished from verified commitments",
        "'Follow the money' evidence (capital, talent, infrastructure, regulatory attention) supports the strategic inference",
        "Framing respects the one-company rule: one vendor is an anecdote, and the write-up does not generalize past the evidence",
        "A competitor reaction scan is included, treating competitor silence as signal"
      ]
    },
    {
      "id": "durable-artifact-gate-preservation",
      "prompt": "Your research subagent wrapped up a competitor API-pricing investigation and dropped a really solid summary right here in chat. We're about to close the ticket and start building. Is there anything left to do before we call the research done?",
      "expected_output": "Yes — under the Durable Artifact Gate the research is not complete, because the only useful output currently lives in chat and will be unrecoverable context. Before closing: identify the durable destination (the host system's normal long-lived research surface — linked knowledge records, a tracked report package, or a project document; never a transient workspace or untracked scratch file). Extract at each source's natural granularity: every distinct reusable claim, data point, contradiction, and open question that materially changes future reasoning, with no fixed note count as a stopping rule, continuing until every retained source is accounted for in the extraction log. Preserve provenance on each artifact: source URL or citation, access date, evidence strength, and links back to the research question and any dependent synthesis. Separate extraction from synthesis: the summary may explain conclusions, but source-level evidence records must be preserved first and must survive alongside it. Record preservation decisions in the research log, including sources that were rejected, too weak, inaccessible, redundant, or out of scope, so a future researcher can tell intentional exclusions from oversights. Only when retained sources and extracted claims are verifiably represented in the destination system — and the brief and log's durable-artifact sections are complete — is the work done.",
      "assertions": [
        "Chat-only output fails the Durable Artifact Gate; findings move to the host's durable long-lived research surface, not scratch files",
        "Extraction preserves provenance per artifact: source URL/citation, access date, evidence strength, and links to the question and dependent synthesis",
        "Extraction is separated from synthesis — source-level evidence records are preserved before and alongside any compacted summary",
        "Rejected, redundant, or out-of-scope sources are recorded in the research log as deliberate exclusions",
        "Completion is confirmed only after verifying every retained source and extracted claim is represented in the durable destination"
      ]
    },
    {
      "id": "worker-timeout-checkpoint-recovery",
      "prompt": "I delegated a deep literature sweep on battery recycling economics to a research subagent. It hit its execution time limit after 20 minutes and returned a half-written partial summary. Do I rerun the whole thing from scratch, assume the topic's a dead end, or what?",
      "expected_output": "Neither rerun-from-scratch nor abandon: a timeout is an interruption, not a research result, and never grounds for closing the investigation or inferring nothing useful exists. First, read the complete delegation transcript and inspect the workspace or scratch directory to establish what was actually produced before deciding what was lost. Recover and verify every partial artifact, source log, and extracted claim already written — long jobs routinely produce valuable partial work before hitting slow extraction, rate limits, or one unresponsive source. Resume from the last durable checkpoint instead of restarting broad discovery. Fix the bottleneck rather than accepting it: narrow or replace the slow operation (large PDF extraction, repeatedly rate-limited search), and write each subsequent stage incrementally so progress survives another interruption. If the worker cannot be resumed safely, continue the missing research directly or split it into smaller bounded tasks — the timeout changes the execution path, not the acceptance criteria. Report completion only when the research question is covered, retained sources and claims are represented in the durable evidence artifacts, and unresolved gaps are explicit. Elapsed time, worker status, and the existence of a plausible partial summary are not acceptance criteria.",
      "assertions": [
        "Timeout is treated as an interruption — never summarized as complete or interpreted as 'no useful result'",
        "The delegation transcript and workspace are inspected before concluding what was lost, and partial artifacts/source logs are recovered and verified",
        "Work resumes from the last durable checkpoint rather than restarting broad discovery",
        "Slow operations (large PDF extraction, repeated rate-limited search) are narrowed or replaced, with later stages written incrementally",
        "Acceptance is gated on evidence completeness and artifact verification with explicit unresolved gaps — not elapsed time or worker status"
      ]
    },
    {
      "id": "technical-claim-reproduction",
      "prompt": "A widely shared blog post claims a 30B model hits 42 tokens/sec on a 24GB consumer GPU, and I've already cited that number in a draft report. Can I trust it, and what should I do before the report goes out?",
      "expected_output": "Do not ship the borrowed number if you have the tools to test it: when a claim rests on a measurable performance figure and reproduction is feasible, test it yourself — your own properly conducted measurement is tier-1 evidence. Follow the reproduction standard: read the claim precisely (exact numbers, versions, flags, conditions); replicate conditions as closely as possible (model and software version, hardware, configuration, measurement methodology), noting that the post likely omits driver/CUDA details, which makes the original benchmark incomplete by definition; run the benchmark at least three times, since variability across runs is itself data. Compare honestly: matching within expected variance verifies the claim; significant divergence means either the claim or your conditions are off — check conditions, then report the discrepancy; inability to reproduce makes the claim unverifiable and flagged as such. Document the reproduction (claim tested, results, conditions, min/max/mean variance, verdict: Verified / Partially supported / Contradicted / Unverifiable). If testing genuinely is not feasible, find independent reproductions, audit the vendor's methodology critically (sample size, confounds, conflicts of interest), and flag the figure in the draft as vendor-reported and not independently verified. Under the Pre-Publication Gateway, a factual performance claim in a report gets this treatment before it is reported as established.",
      "assertions": [
        "Self-reproduction is attempted before the cited number is trusted, with own measurement ranked as tier-1 evidence",
        "Conditions replication covers model/software version, hardware, flags, and methodology, and the source's unstated benchmark conditions are called out",
        "Multiple runs are performed with variance reported, and outcomes map to Verified / Partially supported / Contradicted / Unverifiable verdicts",
        "If reproduction is infeasible, independent reproductions are sought and the vendor-only figure is explicitly flagged in the draft",
        "The claim is withheld from the report as established fact until the reproduction protocol completes"
      ]
    },
    {
      "id": "single-lookup-answered-directly",
      "prompt": "quick one while I'm here — what port does PostgreSQL listen on by default?",
      "expected_output": "Answer immediately and directly: 5432 (configurable via postgresql.conf or the PGPORT environment variable). This is a single factual lookup, and the correct routing is to respond directly without engaging the research lifecycle — spinning up scoping, inclusion/exclusion criteria, source triangulation, and a research log adds cost without adding credibility for a settled, stable fact. The full Scope-Gather-Evaluate-Analyze-Synthesize-Report pipeline, durable-artifact machinery, and pre-publication protocols exist for investigations where method traceability and evidence credibility matter; they are actively wrong-sized here. At most, cite where the answer comes from (official PostgreSQL documentation); no multi-phase process, no brief, no evidence table.",
      "assertions": [
        "Responds directly with the correct default port (5432) instead of launching a research lifecycle",
        "Recognizes the request sits outside the skill's trigger boundary — a single factual lookup answered without the full process",
        "Imposes no unnecessary scoping, triangulation, logging, or artifact machinery on a trivial lookup"
      ]
    },
    {
      "id": "media-claim-source-experiment-closure",
      "prompt": "Research whether sparse FFmpeg frame samples prove that a logo never appears anywhere in a two-hour video. Keep the work reproducible and tell me which skill owns what.",
      "expected_output": "Use research-methodology for question scope, sources/access dates, exclusions, claim closure, sampling audit, and durable artifacts; route FFmpeg commands, probes, extraction, and build-specific experiments to the ffmpeg skill; and route semantic/editorial judgment to an attributed reviewer. Reject the whole-video absence conclusion because sparse samples cover only sampled timestamps, then narrow the claim, collect sufficient bounded evidence, or leave it unresolved.",
      "assertions": [
        "makes the ownership boundary between research-methodology, ffmpeg, and the reviewer explicit",
        "requires source URLs and access dates for current documented claims and exact versions/builds for reproduced claims",
        "records asset/stream/interval, sampling method, transformations, coverage, and blind spots",
        "rejects a whole-video or editorial conclusion from sparse technical samples",
        "maps each claim to source, experiment, limitation, durable artifact, and exclusion or unresolved status"
      ]
    }
  ]
}

# Industry Analysis Track

For the publication's — analyzing company strategy, market signals, and technology trends with data that enterprise decision-makers will scrutinize.

## The Core Difference From Journalism

Journalism asks "what happened and who's responsible." Industry analysis asks "what does this mean, what's coming next, and what should we do about it." The evidence base is more public, the time pressure is higher, and the conclusion is the point.

## Signal vs. Noise

The fundamental skill of industry analysis is distinguishing what matters from what's loud. Most tech news is noise — a product launch, a funding round, an executive departure. Signal is what changes the decision landscape.

### The Signal Filter

Before investing time in any development, run it through this filter:

| Question | If yes | If no |
|----------|--------|-------|
| Does this change a company's competitive position? | Signal | Noise |
| Does this change the cost structure of a technology? | Signal | Noise |
| Does this change who can build what? | Signal | Noise |
| Does this open a market that was previously closed? | Signal | Noise |
| Does this reveal a strategy the company hasn't announced? | Signal | Noise |
| Is this the first time something has happened? | Signal, investigate | Trend-confirming |

### The "So What" Test

After identifying a development, ask "so what?" three times:

1. **So what?** Salesforce's AI Work Ratio hit 30%. (→ They're changing how they measure productivity)
2. **So what?** That means enterprise AI pricing is about to shift from per-seat to outcome-based. (→ Competitors will follow)
3. **So what?** This changes the TCO calculation for every enterprise evaluating AI tools. (→ That's the story.)

If you can't answer the third "so what," you haven't found the angle yet.

## Corporate Evidence Sources

### Financial Documents

| Document | What it reveals | Where to find it |
|----------|----------------|------------------|
| **10-K / Annual report** | Risk factors, strategy letter, segment financials, competitive landscape | SEC EDGAR, company investor relations |
| **10-Q** | Quarterly trends, updated risk factors, management's discussion | SEC EDGAR |
| **Earnings call transcript** | What leadership emphasizes, how they answer hard questions, what they avoid | Seeking Alpha, Fool.com, company IR |
| **Investor day presentations** | Long-term strategy, TAM estimates, product roadmaps | Company IR site |
| **S-1 (IPO filing)** | Full business model, competitive risks, historical financials | SEC EDGAR |

### Executive Signal Analysis

What executives say is strategic communication. What they *don't* say is often more revealing.

| Signal | What to look for |
|--------|-----------------|
| **Evasive answer to a direct question** | They know something but won't say it on the record — the question hit a nerve |
| **Forward-looking language** | "We see a path to..." vs "This is a priority" — specificity indicates commitment |
| **Who's assigned to what** | An executive moved from a core product to an experimental one tells you where the company's center of gravity is shifting |
| **Rhetorical emphasis** | The same phrase appearing across multiple executives' communications is a coordinated message |
| **What they praise competitors for** | "We admire how [competitor] has done X" often signals an upcoming strategy change |
| **Comparison baselines** | "Faster than the market" is weak. "Faster than our three largest competitors combined" is strong. Which baseline they choose reveals how they measure themselves. |

### The "Follow the Money" Pattern

The most reliable signal in technology analysis:

1. **Where is investment capital flowing?** Follow the money — VC funding, R&D spend, acquisition targets.
2. **Where is talent flowing?** Key hires, departures, executive churn. A mass exodus from one company to another tells you where the center of gravity is shifting.
3. **Where is infrastructure spending going?** Cloud contracts, GPU procurement, data center buildout. Capital deployment reveals strategy more honestly than press releases.
4. **Where is regulatory attention focused?** FTC investigations, EU digital regulations, export controls. Policy follows economic concentration.

## The Case Study Standard

Industry analysis articles often use case studies. The standard for evidence:

| Claim type | Minimum evidence | Strong evidence |
|------------|----------------|-----------------|
| "Company X achieved Y" | Public announcement or earnings mention | Independent verification (user survey, third-party audit, bench test) |
| "X% of organizations do Y" | Industry survey (named firm, linked methodology) | Multiple surveys, same finding, different respondents |
| "Trend X is accelerating" | Two data points showing direction | Three+ data points with consistent trajectory, independent sources |
| "X is more effective than Y" | Company A/B test (must state sample size and duration) | Independent academic study or multi-company meta-analysis |
| "Leadership said Z" | Verbatim quote with source (transcript, event video, blog post) | Quote + context of when/why they said it |

### The "One Company" Rule

A single case study is an anecdote. Two companies with different profiles showing the same pattern is a finding. Five companies across different segments is a trend. Frame accordingly:

| Count | Framing |
|-------|---------|
| 1 company | "At Company X, we observed..." |
| 2-3 companies | "Across early adopters, a consistent pattern..." |
| 4+ companies | "The data across the industry shows..." |

## Competitive Intelligence Ethics

| Do | Don't |
|----|-------|
| Use publicly available information | Misrepresent yourself to get information |
| Analyze published products and marketing | Use non-public pricing decks or internal documents |
| Attend public events and conferences | Ask interview subjects to reveal confidential information |
| Track job postings and patent filings | Pay employees of competitors for information |
| Subscribe to competitors' public newsletters | Use a fake identity to access gated competitor content |

## Industry Analysis Research Protocol

### Standard Depth Piece (Moderate Depth)

1. **Scrape the headline** — what's the development? Identify the core event or data release
2. **Primary source check** — find and read the actual announcement, paper, or filing. Not the news about it
3. **Signal filter** — run through the signal/noise filter above. If it's noise, stop
4. **Context layer** — what has happened in the preceding 6 months that changes how to read this? Load the last 2-3 related sources for trajectory
5. **Competitor reaction scan** — have competitors responded? What does their response (or silence) signal?
6. **The "so what" 3x** — drill to the real implication
7. **Write the brief** — load `assets/research-brief.md` and fill for the publication's format

# Journalistic Research Track

For the publication investigative pieces — source-heavy, narrative-driven work where facts must survive public scrutiny.

## The Core Difference From Academic Research

Academic research assumes all sources are published and findable. Journalistic research assumes the best sources are people, documents that weren't meant to be public, and observations the researcher makes firsthand. The evidence is messier, the verification burden is higher, and the stakes for getting it wrong are immediate.

## Primary Source Handling

### The Hierarchy of Journalistic Evidence

| Tier | Source type | How to treat it |
|------|-------------|-----------------|
| 1 | **Primary documents** — emails, internal memos, court filings, financial statements, code commits, leaked data | Highest weight. Corroborate authenticity, then treat as ground truth for what the document says |
| 2 | **Firsthand testimony** — someone who was there, saw it happen, or did the thing | Named sources are stronger than anonymous. Corroborate specifics against documents when possible |
| 3 | **Contemporary records** — meeting notes, chat logs, recordings made at the time | Stronger than memory. Memory is unreliable within weeks |
| 4 | **Published reporting** — other journalists who covered the story | Cite them, don't re-report their work. If you stand on their reporting, make it visible |
| 5 | **Secondhand accounts** — someone who heard from someone who was there | Weakest tier. Only use if primary sources are unavailable and the chain is short |

### The Named Source Standard

**Always prefer named sources.** A named source who is proven wrong damages your credibility. An anonymous source who is proven right also damages your credibility — because the reader can't verify who said it.

When a source requests anonymity:

1. **Establish why.** Is it fear of retaliation, a non-disclosure agreement, or discomfort with public attribution? The reason determines how much weight to give the information.
2. **Know their identity.** The reader may not know who they are, but you must. Record it. The source must be verifiable to you even if not to the reader.
3. **Corroborate their claims.** An anonymous source making a verifiable claim is useful. An anonymous source making an unverifiable claim is hearsay.
4. **Never grant anonymity to a source you haven't spoken to directly.** Secondhand anonymous sourcing is not sourcing.
5. **Disclose what you can.** "A current employee who spoke on condition of anonymity because they were not authorized to discuss internal matters" is better than "sources say."

### Interview Integrity

- **Record the conversation.** Always ask permission. If they decline, take contemporaneous notes and read critical quotes back during the interview for confirmation.
- **Verify identity.** If you haven't met the person before, verify who they are before treating their information as source material. LinkedIn, corporate website, cross-reference with others.
- **Context matters.** What else was happening when they said this? Under what circumstances did they share this information? A source who is angry may overstate. A source who is afraid may understate.
- **Follow up.** The best quote often comes after you've turned off the recorder — or in a second conversation after they've had time to think.

## Document Investigation

Not all valuable evidence comes from people. Some of the best sources are documents that exist in plain sight.

### Types of Documents Worth Finding

| Document type | What to look for | Where to find it |
|--------------|-----------------|------------------|
| **Git history** | When was a feature added? Who committed it? What was the commit message? Was it reviewed? | GitHub, git log, pull request discussions |
| **Earnings transcripts** | What does leadership actually say about their strategy? Compare with what they do. | SEC.gov, investor relations pages, earnings call transcripts |
| **Job postings** | What skills are they hiring for? What does the job description reveal about priorities or problems? | Company career pages, LinkedIn |
| **Support forums** | What are users actually struggling with? What workarounds have they developed? | GitHub issues, community forums, subreddits |
| **Change logs** | What did the product look like 6 months ago vs now? What features were removed? | Changelog archives, Wayback Machine |
| **Filing/pipeline documents** | SEC filings, patent applications, regulatory submissions | EDGAR, patent databases, FCC filings |

### Reading With Intent

Don't just read a document — interrogate it:

- **Who created this, and why?** Every document has an intended audience and a purpose. A press release is meant to make the company look good. An internal memo is meant to communicate operational direction. A patent application stakes a legal claim.
- **What's missing?** Absences are often more revealing than presences. A security audit that doesn't mention authentication is a finding. A press release that doesn't name the CEO is a signal.
- **What assumptions does it rely on?** "Q4 revenue grew 15% year-over-year" assumes the previous year's Q4 is the right baseline. What if they had an acquisition that quarter?
- **Does the headline match the body?** Press releases, blog posts, and reports often have headlines that overstate the findings in the body. Read past the lede.

## The Three-Source Rule (Journalism Version)

For any factual claim that:
- Accuses someone of wrongdoing
- Relies on a surprising statistic
- Is contested by other sources
- Would be damaging if wrong

**Require three independent sources.** Not three articles citing each other. Three genuinely independent sources — different people, different documents, different methodologies — all pointing to the same fact.

If you can't get three, report the confidence gap: "Only one person with direct knowledge would speak about it, but their account was consistent with internal documents reviewed by [publication]."

## Researching Across a Series

When researching article 3 of a 5-part series, the trap is re-researching what was already established.

### Per-Installment Research Map

For each article in a series:

| Dimension | Done in earlier installment | Needs research for this one |
|-----------|---------------------------|----------------------------|
| **Shared context** | What the reader already knows from parts 1-2 | What new context does this installment need? |
| **Sources to revisit** | Experts who were informative earlier | Is there a new angle that warrants a follow-up conversation? |
| **Claims that need updating** | Facts established in earlier parts | Have any changed since publication? |
| **Recurring characters** | People introduced in earlier parts | Do they appear in this installment? Do they need re-introduction? |
| **Thematic throughline** | What theme has connected the series so far | Does this installment advance it, complicate it, or branch? |

### The Series Brief

Before starting research on a new installment, write a one-paragraph "where we are" summary that answers:
1. What has the series established definitively so far?
2. What open questions remain from earlier installments?
3. What new ground does this installment cover?
4. What sources from earlier installments could speak to this new ground?

This prevents re-researching and keeps each installment building on the last.

## Pre-Publication Verification Protocol

Before any the publication piece with investigative stakes ships:

1. **Read every quote in the draft against the source.** Not from memory. From the recording or notes. Word for word.
2. **Open every link in the draft.** Does the source say what the draft claims? Numbers match? Quotes exact?
3. **Triangulate every surprise claim.** If a finding would change what the reader thinks, it needs at least two sources.
4. **Check recency.** Statistics cited as "current" within 2 years for fast-moving tech, 5 years for slower domains. Flag outdated data explicitly.
5. **Audit for paraphrase drift.** The draft may accurately cite a source but mis-state the strength of the claim. A paper that says "suggests" should not be paraphrased as "proves."
6. **Flag single-source claims.** Any claim that rests on a single source should be called out in the draft. The reader deserves to know the evidence base.
7. **Run the "what if I'm wrong" test.** If every claim in this article turned out to be false, which ones would do the most damage? Verify those hardest.

# Media Evidence Routing

Use the shared research lifecycle for media questions, but keep operational media mechanics with the owning tool skill. This reference defines the handoff; it is not an FFmpeg runbook.

## Ownership boundary

| Concern | Owner |
|---|---|
| Research question, inclusion/exclusion rules, source evaluation, current-claim dates, rejected-source log, source-to-claim audit, synthesis confidence, and durable preservation | `research-methodology` |
| FFmpeg/FFprobe commands, build inventories, media intake, bounded extraction, EDLs, rendering, signal measurement, and output/target verification | `ffmpeg` |
| Transcript acquisition and timing quality | The authorized source or transcription skill |
| Semantic frame/audio interpretation and editorial approval | An attributed human or capable reviewer |
| Upload, account, or platform API action | The named platform skill |

Start from `assets/research-brief.md` and `assets/research-log.md`. For consequential media claims, also copy `assets/media-claim-ledger.md`. Route command selection and experiment execution to `ffmpeg`; return its versioned records to the research ledger before synthesis.

## Media brief additions

Record these before gathering evidence:

- editorial or technical objective and the decision it will inform;
- output contract and named downstream consumer, if any;
- rights, authorization, privacy, retention, and sharing boundary;
- decision granularity: whole asset, interval, event, frame, packet, or stream;
- review points and accountable human decisions;
- required coverage and what would make sampling insufficient.

## Evidence classes

Keep unlike evidence separate:

- **Documented semantics:** an official specification or tool manual, with URL, accessed date, relevant version, and exact supported claim.
- **Reproduced behavior:** exact tool/build, environment, fixture/digest, commands, raw results, and repeated observations.
- **Observed artifact:** probe fields, decoded samples, measurements, destination results, or attributed review for the named artifact and interval.
- **Environment-specific claim:** availability or behavior tied to one build, device, operating system, target, or account.
- **Heuristic:** detector threshold, sparse sampling, transcript navigation, inferred continuity, or editorial convention requiring review.
- **Unresolved:** conflicting, inaccessible, untested, undersampled, or authorization-blocked claim.

Official documentation explains documented behavior; it does not prove local availability or the outcome of a command. A reproduction establishes only its recorded fixture and conditions. A target result applies only to that target/version.

## Sampling and editorial guardrails

Every frame, clip, waveform, transcript span, or detector event needs an asset ID, stream/interval, selection method, transformations, and coverage statement. Audit whether the sample can support the claim's scope. Sparse frames cannot establish absence throughout a video; a detector event cannot establish editorial meaning; a transcript cannot establish exact audiovisual continuity; a local decode cannot establish downstream acceptance.

When coverage is insufficient, narrow the claim, collect more bounded evidence, request attributed review, or leave the question unresolved. Technical facts and human editorial approval remain separate ledger entries.

## Worked source-to-claim closure

This compact example demonstrates the method, not a portable compatibility guarantee.

| Field | Record |
|---|---|
| Claim | Explicit stream mapping preserved one MPEG-4 video stream followed by one PCM audio stream in a Matroska remux under the recorded experiment. |
| Classification | Reproduced behavior plus observed artifact. |
| Source | [FFmpeg documentation](https://ffmpeg.org/ffmpeg.html), accessed 2026-09-08; supports `-map` stream-selection semantics, not this run's outcome. |
| Experiment | Synthetic 64x64/10 fps video plus 48 kHz sine audio; FFmpeg and FFprobe 8.1.2; create source, remux with `-map 0:v:0 -map 0:a:0 -c copy`, then probe stream index/type/codec. |
| Observation | FFprobe reported stream 0 as MPEG-4 video and stream 1 as PCM signed 16-bit little-endian audio. |
| Limitation | One local build, synthetic fixture, Matroska container, and one stream order. This does not establish visual/audio quality, exact timing, other builds/containers, or downstream compatibility. |
| Durable artifacts | Claim ledger entry, access-dated source record, exact command log, build record, source/output probe JSON, fixture generator, and rejected/untested-source entries. |

Closure requires the claim to link to every named artifact and the durable log to record missing evidence. Preserve inaccessible, rejected, redundant, and untested sources with reasons so omission cannot masquerade as coverage.

# Research Lifecycle

The full arc of a professional research engagement, from question formulation through reporting.

## Phase 1: Scope — Frame the research question

Before gathering a single source, define what you're looking for and why.

### The Research Brief

Every research project starts with a brief. Load `assets/research-brief.md` and fill it out before proceeding. The brief answers:

- **Core question:** What exactly are we trying to find out? Frame as a single, falsifiable question.
- **Scope boundaries:** What's in and what's out. Define inclusion/exclusion criteria before searching.
- **Depth required:** Quick scan (2-3 sources for an overview) vs comprehensive (exhaustive on a narrow question) vs deep-dive (multi-angle on a complex question)
- **Decision context:** Who needs this information and what will they do with it? A product decision needs different evidence than a research paper.

### Question Types

| Question type | What you're looking for | Example |
|--------------|------------------------|---------|
| **Descriptive** | What's happening? | "What architectures are used for local LLM inference on consumer GPUs?" |
| **Comparative** | How does X compare to Y? | "How does Qwen3.6 compare to Gemma 4 for tool calling?" |
| **Causal** | What drives X? | "Why do MoE models have lower inference latency at high batch sizes?" |
| **Evaluative** | Is X effective? | "What evidence exists that RAG improves accuracy over zero-shot?" |
| **Gap-finding** | What's not known? | "What hasn't been published about neuromorphic edge deployment?" |

### Inclusion/Exclusion Criteria

Before searching, define:
- **Date range:** How recent must sources be?
- **Source type:** Peer-reviewed, industry reports, blog posts, documentation?
- **Authority threshold:** What makes a source credible enough?
- **Language:** English only, or other languages?
- **Duplication rule:** Multiple sources reporting the same finding — count as one or many?

## Phase 2: Gather — Systematic source discovery

### Search Strategy

1. **Start broad, then narrow.** First query should be broad enough to map the territory. Subsequent queries narrow based on what you found.
2. **Use multiple search angles.** Don't rely on one query. Search by: keyword, author/institution, tool name, problem statement, related concept.
3. **Citation chaining.** From each promising source:
   - **Backward:** Follow the citations/bibliography to find the sources the author relied on
   - **Forward:** Search for papers that cite this source (Google Scholar "cited by")
4. **Source diversity.** Don't rely on one type of source. Mix:
   - Primary research (papers, technical reports)
   - Grey literature (blog posts, documentation, forum discussions)
   - Expert commentary (industry analysis, conference talks)
   - Empirical data (benchmarks, datasets, reproducible experiments)

### For Each Source Found, Record

| Field | Purpose |
|-------|---------|
| Title + URL | Find it again |
| Author/source | Credibility assessment |
| Date | Recency check |
| Key claims | What it says that's relevant |
| Supporting evidence | What backs the claims |
| Gaps/limitations | What it doesn't say |
| Connection to brief | How it answers the research question |

## Phase 6: Report — Structure the findings

A research brief should have this structure (see `assets/research-brief.md`):

1. **Executive summary** — One-paragraph answer to the research question
2. **Key findings** — 3-5 synthesized findings with confidence levels
3. **Evidence table** — Sources mapped to findings
4. **Confidence assessment** — What's solid, what's uncertain, what's missing
5. **Open questions** — What you still don't know
6. **Sources** — Full citations with URLs

## Decision Contexts

Different depths of research for different needs:

| Context | Depth | Sources | Time |
|---------|-------|---------|------|
| Quick answer for a decision | Scan | 2-3 targeted sources | Minutes |
| Briefing for a conversation | Light | 5-8 sources, 2 angles | 1 hour |
| Support for a recommendation | Moderate | 10-15 sources, 3+ angles | Half day |
| Foundation for a publication | Deep | 20+ sources, exhaustive | Days |

# Source Evaluation

How to judge whether a source is credible, relevant, and useful before committing to deeper work with it.

## The CRAAP Test

The most widely used framework for evaluating source credibility. Each dimension scored 1 (poor) to 5 (excellent).

| Dimension | What to ask | Low (1-2) | High (4-5) |
|-----------|-------------|-----------|------------|
| **Currency** | When was this published? Has it been updated? Is the field moving fast enough that age matters? | >5 years old in a fast-moving field | Recent, or classic/stable reference |
| **Relevance** | Does this directly address the research question? Who is the intended audience? | Tangential, or wrong audience | Directly addresses the brief |
| **Authority** | Who wrote this? What are their credentials? Is the publisher reputable? | Unknown author, non-reputable publisher | Known expert, peer-reviewed, institutional affiliation |
| **Accuracy** | Is the information supported by evidence? Can it be verified elsewhere? | No sources, claims unsupported | Cited, verifiable, consistent with other sources |
| **Purpose** | Why does this source exist? To inform, persuade, sell, entertain? Is there bias? | Clear bias, advocacy, or commercial intent | Educational or informational, bias acknowledged |

**Thresholds:**
- **18-25:** High-quality source, use confidently
- **13-17:** Adequate, corroborate with another source
- **Below 13:** Low quality, use with caution or discard

## Triangulation

A single source is never enough. Confidence comes from convergence across multiple independent sources.

### Types of Triangulation

| Type | What it means | How to use |
|------|--------------|------------|
| **Source triangulation** | Multiple sources saying the same thing | If 3+ independent sources agree, confidence is high |
| **Methodological triangulation** | Different methods producing the same conclusion | A paper + a benchmark + a case study all pointing the same way |
| **Investigator triangulation** | Different researchers/teams reaching the same conclusion | Multiple labs replicating a finding |
| **Theory triangulation** | Different theoretical frameworks predicting the same outcome | Multiple lenses converging |

### The Triangulation Rule

- **3+ independent sources agree:** Treat as high confidence
- **2 sources agree, 1 contradicts:** Investigate the contradiction — it may be the more interesting finding
- **Sources disagree without clear resolution:** Report the disagreement, don't force consensus
- **Single source on a key claim:** Flag as "single source" in the brief, not as established fact

## Red Flags That Warrant Caution

| Red flag | What to do |
|----------|------------|
| Source is the sole origin of a claim | Corroborate before trusting |
| Source has a clear commercial interest | Discount claims that align with their product |
| Source cites itself or a closed loop | Low credibility for external claims |
| Source makes extraordinary claims without extraordinary evidence | Require higher corroboration standard |
| Source is anonymous or unverifiable | Use only as directional signal |
| Source is from a known advocacy organization | Understand the bias, still use if evidence is sound |

## Source Types by Reliability

| Tier | Source type | Trust |
|------|-------------|-------|
| 1 | Peer-reviewed research, official documentation, primary sources | High |
| 2 | Industry reports, technical blogs by known practitioners, conference talks | Medium-High |
| 3 | News articles, community discussions, forum posts | Medium |
| 4 | Social media, anonymous posts, marketing content | Low-Directional |
| 5 | Opinion without evidence, satire, known unreliable sources | Discard |

# Source index

- **Source repository:** https://github.com/magnus919/hermes-profiles
- **Inspected commit:** `867a555`
- **Imported source directory:** `research-methodology`
- **Porting boundary:** Retained portable methodology, templates, scripts, and references. Removed or generalized Hermes profile, task-orchestration, memory, and rigid response-handoff assumptions.

# Structured Analytic Techniques

Techniques adapted from intelligence analysis to systematically evaluate evidence, challenge assumptions, and avoid cognitive bias.

## Analysis of Competing Hypotheses (ACH)

Use when you have multiple possible explanations for the same evidence and need to decide which is most credible.

### Step-by-Step

1. **Identify all hypotheses.** Brainstorm every plausible explanation for what you're seeing. Include the null hypothesis (nothing unusual is happening). Do NOT pick a favorite yet.

2. **List evidence and assumptions.** For each hypothesis, note what would support it, what would contradict it, and what assumptions it requires.

3. **Build a matrix.** Rows = evidence items. Columns = hypotheses. For each cell, mark:
   - **CC** (consistent): This evidence supports this hypothesis
   - **IC** (inconsistent): This evidence contradicts this hypothesis
   - **N/A** (not applicable): This evidence doesn't bear on this hypothesis

4. **Work across, not down.** The most important step. For each piece of evidence, evaluate it against ALL hypotheses before moving to the next piece. This prevents confirmation bias toward a single hypothesis.

5. **Count inconsistencies.** The hypothesis with the FEWEST inconsistencies is the most likely — NOT the one with the most supporting evidence. ACH is a refutation tool, not a confirmation tool.

6. **Test sensitivity.** What if a key piece of evidence is wrong? If removing it changes your conclusion, that evidence is a linchpin — verify it.

7. **Report.** Present the conclusion, the rejected alternatives, and the linchpin evidence that drove the decision.

### When to Use ACH

| Good for | Not good for |
|----------|-------------|
| Competing technical explanations | Single-hypothesis verification |
| Evaluating competing vendor claims | Exploratory research |
| Root cause analysis | Routine fact-gathering |
| Contradictory evidence sets | Simple yes/no questions |

## Driving Forces Analysis

Use to understand what's shaping a trend, market, or technology trajectory.

1. **List driving forces.** What factors are pushing in one direction? (Technology advances, regulation, market demand, cost curves)
2. **List restraining forces.** What's holding back change? (Incumbent lock-in, technical limitations, talent gaps, infrastructure debt)
3. **Which forces are accelerating?** Are drivers getting stronger or weaker?
4. **What would change the balance?** What event or discovery would shift the equilibrium?
5. **Two scenarios.** If drivers win → what happens? If restrainers hold → what happens?

## Pre-Mortem Analysis

Use before committing to a research conclusion to identify what could be wrong.

1. **Assume the conclusion is wrong.** Imagine it's six months from now and your research finding turned out to be completely incorrect.
2. **Write the failure story.** What happened? What evidence misled you? What assumptions were wrong? What did you miss?
3. **Identify failure modes.** Which specific evidence items, assumptions, or reasoning steps are most vulnerable?
4. **Harden the analysis.** For each failure mode: what additional evidence would rule it out? What alternative explanation would cover it?

## Indicator / Validator Framework

Use to track whether an ongoing development is trending toward or away from a predicted outcome.

1. **Define observable indicators.** What would you see if the prediction is correct? What would you see if it's wrong?
2. **Assign diagnostic value.** Some indicators are stronger than others. An indicator that would exist ONLY under one scenario is highly diagnostic.
3. **Track over time.** Indicators don't fire all at once. Track which are appearing, which haven't, and which are contradictory.
4. **Update confidence.** As evidence accumulates, adjust your confidence in each scenario.

## Linchpin Analysis

Use to identify which single element your entire conclusion rests on.

1. **Trace the reasoning chain.** Conclusion → supporting evidence → foundational assumptions.
2. **Find the linchpin.** Which assumption or evidence item, if wrong, would collapse the entire conclusion?
3. **Test that specific element.** Don't test random alternatives. Test the linchpin.
4. **Report linchpin confidence separately.** "I'm confident in the conclusion IF [linchpin] holds. Here's what would change if it doesn't."

# Synthesis Patterns

Methods for combining findings from multiple sources into something more valuable than any single source.

## Thematic Synthesis

Best for: Combining qualitative findings, expert opinions, and case studies.

1. **Extract claims.** From each source, pull the specific claims relevant to your research question.
2. **Find patterns.** Group claims that point in the same direction. What themes emerge?
3. **Name each theme.** A good theme name is specific enough to be meaningful, broad enough to contain related claims.
4. **Support each theme.** List which sources support it, which contradict it, and which are neutral.
5. **Identify convergent and divergent themes.** Where do sources agree? Where do they disagree?

### Convergence Signals

| Signal | What it means |
|--------|--------------|
| 3+ independent sources agree | High confidence in the finding |
| Sources from different fields converge | Very high confidence — cross-validation across disciplines |
| Different methodologies produce same result | Method-independent finding |
| Even critics concede the point | Defensible claim |

### Divergence Signals

| Signal | What it means |
|--------|--------------|
| Sources disagree without clear pattern | Territory is unsettled — report both sides |
| Disagreement correlates with methodology | Methods may be driving results |
| Disagreement correlates with funding source | Conflict of interest may be a factor |
| Single source contradicts consensus | Investigate deeper — the outlier may be wrong or may have found something others missed |

## Lines of Argument

Best for: Building a case for or against a position.

1. **State the conclusion.** What are you trying to support?
2. **Build independent lines.** Each line of argument is a separate chain of reasoning that supports the conclusion. They should be as independent as possible.
3. **Test each line.** Does each line stand on its own evidence? Or do they share assumptions?
4. **Weight the lines.** Some lines are stronger than others. Score each: strong anchoring evidence, moderate supporting evidence, or weak circumstantial evidence.
5. **Cross-line consistency.** If multiple independent lines all point to the same conclusion, confidence is much higher than if they share a common assumption.

## Constant Comparison

Best for: Iterative research where findings emerge gradually.

1. **Start with the first source.** Extract initial themes.
2. **Compare each new source.** Does it confirm existing themes? Add new ones? Contradict?
3. **Update themes.** Refine, split, or merge themes as evidence accumulates.
4. **Test saturation.** Are new sources adding new themes or just confirming existing ones? When new sources stop adding new themes, you've reached saturation.

## The Convergence-Divergence Matrix

Map findings along two axes:

| | Convergent (sources agree) | Divergent (sources disagree) |
|---|---|---|
| **Well-evidenced** (many sources) | ⭐ High-confidence finding — report as solid | ⚠️ Active debate — report both sides with evidence weights |
| **Thinly-evidenced** (few sources) | 📌 Tentative finding — flag as provisional | 🔍 Gap — identify as area needing more research |

# Technical Verification Track

For claims that can be tested by doing — running code, querying APIs, checking benchmarks, measuring yourself. The most credible evidence is evidence you can reproduce.

## The Core Principle

If a claim rests on a number, benchmark, or observable behavior, and you have the tools to test it, do not rely on the source's claim. Test it yourself. Your own measurement, properly conducted, is a tier-1 source.

## What to Verify by Testing

| Claim type | Test method | Tool |
|------------|-------------|------|
| **Performance numbers** (latency, throughput, VRAM usage) | Reproduce the benchmark with the same parameters | Terminal, Python scripts |
| **API behavior** ("the API returns X when Y") | Call the API with the documented parameters | curl, Python httpx |
| **Model outputs** ("Model A beats Model B on X") | Run the same prompt through both models | llama.cpp, OpenRouter API |
| **Configuration claims** ("Set flag X for best results") | Try it with and without the flag, compare | Terminal, A/B testing |
| **Memory/disk usage** ("This uses less than X") | Build the system, measure actual usage | `du`, `ps`, `nvidia-smi`, `htop` |
| **Availability claims** ("The protocol supports Y") | Read the spec, then try to do it | Source code, protocol docs, actual implementation |
| **Compatibility claims** ("Works on macOS and Linux") | Test on both platforms or verify per-platform CI results | CI logs, Docker |

## The Reproduction Standard

### Step 1: Read the claim carefully
What, exactly, does the source claim? Write down the specific numbers, flags, parameters, and conditions.

### Step 2: Replicate the conditions
Use the same:
- Model version / software version
- Hardware (or comparable)
- Configuration flags
- Input data (or equivalent)
- Measurement methodology

If the source doesn't specify conditions fully, note what's missing. A benchmark that doesn't specify GPU driver version, CUDA version, or `nvidia-smi` output is incomplete.

### Step 3: Run the test
Run it once and observe. Then run it again. Then a third time. Variability across runs is itself data.

### Step 4: Compare results

| Situation | What it means |
|-----------|--------------|
| Your result matches the claim within expected variance | Claim verified — high confidence |
| Your result differs significantly | Either the claim is wrong, or your conditions differ. Check conditions, then report the discrepancy |
| You can't reproduce at all | Claim is unverifiable with available resources. Flag it |
| Your result is better than the claim | Interesting — may mean setup differences, or the claim was conservative |

### Step 5: Document the reproduction

```
## Verification

- Claim tested: [exact claim from source]
- My results: [numbers]
- Conditions: [hardware, software, flags, methodology]
- Variance across runs: [min/max/mean across N runs]
- Verdict: Verified / Partially supported / Contradicted / Unverifiable
- Notes: [any caveats about the test conditions]
```

## When Testing Isn't Feasible

Some claims can't be tested with available resources (requires $10K of cloud credits, proprietary hardware, or access to a system you don't have). In these cases:

1. **Find independent reproductions.** Has someone else tested the same claim? Look for replication studies, community benchmarks, or forum discussions.
2. **Read the methodology critically.** If you can't test it yourself, audit the testing methodology. Was the sample size adequate? Were confounding variables controlled? Was there a conflict of interest?
3. **Flag untested claims in the draft.** "This benchmark was conducted by the vendor and has not been independently verified" is honest and keeps you protected.

## The "I Built It" Standard

For technical tutorials and walkthroughs (like "Running a 35B MoE Model on a 16GB Consumer GPU"):

- Every configuration flag in the article must have been tested by the author
- Every command in the article must produce the stated output
- Every screenshot or terminal output must be from the author's own system
- No "should work" — only "worked for me under these conditions"
- If a configuration didn't work, say so and explain why

This standard distinguishes evidence-led technical writing from generic tutorials. The mistakes and dead ends are often the value.

## Media-specific verification

When a technical claim depends on media streams, sampled frames/audio/transcripts, an editorial interpretation, or a downstream consumer, load `media-evidence-routing.md`. The shared reproduction standard still applies, but the `ffmpeg` skill owns command/build/media mechanics. Preserve its probes, commands, samples, and target results in `../assets/media-claim-ledger.md`; do not infer whole-asset or editorial conclusions from sparse technical samples.