{"slug":"trace-to-training-data","title":"trace-to-training-data","summary":"Convert evaluation traces and production logs into SFT examples and preference pairs. Use when graded traces or failure examples exist and need to become training data, when applying rejection sampling to model outputs, or when building DPO pairs from passing and failing runs.","platform":"Claude","tags":[],"authorName":"LLM Mart","authorSlug":"llm-mart","score":0,"source":"github","price":null,"verified":false,"createdAt":"2026-09-01T18:59:31.251229Z","repo":{"url":"https://github.com/wshobson/agents","stars":40003,"forks":4267,"license":"MIT","updatedAt":"2026-09-26T19:54:17Z"},"bodyHtml":"<hr>\n<h2>name: trace-to-training-data\ndescription: Convert evaluation traces and production logs into SFT examples and preference pairs. Use when graded traces or failure examples exist and need to become training data, when applying rejection sampling to model outputs, or when building DPO pairs from passing and failing runs.</h2>\n<h1>Trace To Training Data</h1>\n<p>This skill assumes <code>eval-harness-first</code>\nalready graded the traces being\nconverted here — goldens, graders,\nand <code>runs/&lt;run-id&gt;/results.json</code>\nall exist before conversion\nstarts. This is the flywheel edge\nthat skill names in its own flow:\n\"the same labeled traces become\nthe training set.\" Conversion\nhappens here; grading already\nhappened upstream.</p>\n<p><strong>Input:</strong> graded traces —\n<code>eval/goldens.jsonl</code> plus\n<code>runs/&lt;run-id&gt;/results.json</code>, each\nrow carrying a <code>task_id</code>, a\n<code>verdict</code> from the grader, and a\n<code>reward</code> when the task supports a\nscalar score (judge score,\nexecution partial-credit, or an\nRLVR verifier):</p>\n<pre><code>{\"task_id\": \"t-042\", \"trace_id\": \"t-042-a3\",\n \"messages\": [{\"role\": \"user\", \"content\": \"...\"}],\n \"verdict\": \"pass\", \"reward\": 0.91,\n \"grader\": \"exact_match\"}\n</code></pre>\n<p><strong>Output format:</strong> rows shaped\nexactly like <code>dataset-curation</code>'s\nFormat Selection table — SFT\n<code>messages</code> rows or DPO\n<code>prompt</code>/<code>chosen</code>/<code>rejected</code>\npairs — so this skill's output is\nthat skill's input with no\nreshaping step in between.</p>\n<h2>The Principle</h2>\n<p>The eval harness already did the\nlabeling work: every trace in\n<code>results.json</code> carries a verdict,\nand often a reward, before this\nskill ever touches it. Converting\na graded trace into a training\nrow is mechanical — pick a shape\nfrom <code>dataset-curation</code>'s table,\nmap fields, write JSONL.\n<strong>Curation is the work that\nremains</strong> — which traces clear a\nquality bar, which pairs are\ninformative, and which rows must\nnever enter the training set at\nall.</p>\n<p>Treat any conversion step that\nrequires re-judging a trace as a\nsign the harness is missing a\ngrader, not a gap this skill\nshould paper over. A trace with\nno verdict or reward isn't\nconvertible yet — route it back\nto <code>eval-harness-first</code> first,\ndon't hand-label it here to\nunblock conversion.</p>\n<h2>SFT From Traces</h2>\n<ul>\n<li><strong>Keep the top-reward fraction\nof successful trajectories</strong>,\nnot every passing one. Rank\npassing traces by reward and\ntake a fraction (the\nAgent-lightning pattern) rather\nthan every trace that merely\ncleared the pass bar — a trace\nthat barely passed is a weaker\nSFT signal than one that scored\nwell above threshold.</li>\n<li><strong>Expert-corrected failures\nbecome gold SFT examples\ndirectly</strong> (the Langfuse\npattern) — when a human edits a\nfailing trace's output into a\ncorrect one, that correction\nneeds no reward threshold; a\nhuman already validated it.\nRoute corrections straight into\nthe SFT set.</li>\n<li><strong>Step-level masking beats\nwhole-trajectory discard for\nmulti-step traces.</strong> When only\nsome steps in a multi-step\ntrajectory are bad, mask the\nloss on the bad steps and keep\nthe good ones, rather than\ndiscarding the whole trajectory.\nSRFT reports 32.2% vs. 30.9% on\nSWE-bench for step-level critic\nmasking over trajectory discard\n— a real, if modest, gap from\nthe finer-grained cut.</li>\n</ul>\n<h2>Preference Pairs From Traces</h2>\n<ul>\n<li><strong>Build pairs from\npassing-vs-failing trajectories\non the SAME task</strong>, never from\nunrelated best- and\nworst-scoring traces pulled\nacross different tasks —\ncross-task pairs teach the\nmodel to prefer one task over\nanother, not one response over\nanother.</li>\n<li><strong>Select the rejected member at\nμ−2σ of the reward distribution\nfor that task, never the\nabsolute minimum.</strong>\n<code>preference-optimization</code>'s\nPair Construction section owns\nthe full selection formula;\nthis skill supplies the graded\ntrajectories it consumes.</li>\n<li><strong>Judge-scored delta selection\ncuts pair volume without\ncutting signal.</strong> Score each\ncandidate pair by\nchosen-minus-rejected judge\ndelta and keep only the\nhighest-delta subset — the top\n5k of a 16.5k candidate pool\nmatched the full pool's\ndownstream result. Build the\nfull candidate set first, then\nfilter by delta; don't cap\ngeneration at 5k up front.</li>\n</ul>\n<h2>Hygiene</h2>\n<ul>\n<li><strong>Scan for secrets and PII before any row ships,\nand redact what's found.</strong> Traces sourced from\nproduction logs can carry credentials, API keys,\ntokens, or customer data — run a secret/PII scan\nover every SFT and DPO row and redact matches;\nconversion fails closed (the row is dropped, not\nshipped with the raw content) if sensitive fields\nremain after redaction. Never commit secrets.</li>\n<li><strong>Eval goldens must never leak\ninto training data.</strong> Hold\nevery <code>eval/goldens.jsonl</code> ID\nout of every converted SFT and\nDPO set — a trace that also\nappears as a golden trains on\nthe exact item the checkpoint\ngets graded against later,\nsilently inflating every\nsubsequent eval run.</li>\n<li><strong>Dedup against the training\nset</strong>, not just within the\nnewly converted rows —\nexact-match or\nembedding-similarity, matching\n<code>dataset-curation</code>'s dedup\nmethod field, run against\nwhatever training data already\nexists before this batch merges\nin.</li>\n<li><strong>Provenance goes into the\ndataset card.</strong> Every converted\nrow must trace back to its\nsource <code>run_id</code> and <code>trace_id</code>\n— <code>dataset-curation</code>'s\nProvenance field checks for\nexactly this link back to\n<code>trace-to-training-data</code>\noutput; a row with no traceable\nsource isn't ready to merge.</li>\n</ul>\n<h2>Related Skills</h2>\n<ul>\n<li><code>eval-harness-first</code> — produces\nthe graded traces this skill\nconverts; a trace with no\nverdict or reward isn't\nconvertible yet, route it back\nthere before conversion.</li>\n<li><code>dataset-curation</code> — owns the\ntarget formats and the dataset\ncard this skill's provenance\ndata feeds; converted rows must\nmatch its Format Selection\ntable field names exactly, not\nan approximation of them.</li>\n<li><code>preference-optimization</code> —\nconsumes the DPO pairs this\nskill builds and owns the full\nμ−2σ rejection-selection\nformula referenced above.</li>\n</ul>\n<p>Worked JSONL-to-JSONL conversions\n— graded trace to SFT row, trace\npair to DPO pair, correction to\nSFT row, the rejection-sampling\nloop, and the goldens-holdout\ncheck — live in\n<code>references/conversion-recipes.md</code>.</p>\n","files":[{"path":"references/conversion-recipes.md","sizeBytes":7770,"isText":true},{"path":"SKILL.md","sizeBytes":6251,"isText":true}],"reviewScore":null,"reviewSummary":null,"trust":{"provenance":"trusted-source-unreviewed","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow.","bodySource":null},"bodyLocked":false,"purchaseUrl":null,"sourceUrl":null,"report":{"provenance":"trusted-source-unreviewed","screen":{"ran":true,"outcome":"clean","suspicious":0,"notes":0,"hiddenCharacters":false},"virusScan":{"engine":"clamav","status":"clean","scannedAt":"2026-09-01T19:00:37.842746Z","sha256":"9760C671193467BB1E294673EFA03A92090BF2BEBF2EE30F366BC45A536B0A39","sizeBytes":6124},"review":null,"source":{"repositoryUrl":"https://github.com/wshobson/agents","path":"plugins/llm-finetuning/skills/trace-to-training-data","license":"MIT","commit":"9b15b34b0bfc13a815cbfc2366e14ea549e09422","subtreeSha":"F653BF0457EBA15592DABE55CB35907FE2744CFAD49D2E46E0FA69CA824AE9EA","lastSyncedAt":"2026-09-26T23:12:03.520842Z"},"reviewedAt":"2026-09-01T19:03:43.78364Z","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow."},"install":[{"target":"skills-cli","command":"npx skills add https://github.com/wshobson/agents/tree/main/plugins/llm-finetuning/skills/trace-to-training-data"},{"target":"claude-code","command":"claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install wshobson-agents@llmmart"},{"target":"git","command":"git clone https://github.com/wshobson/agents.git"}]}