This skill assumes finetuning-method-selection\nalready routed here — the next step is preparing\ndata, not choosing a method. What follows: format\nselection by target method, the template/packing\nmechanics behind the most common silent training\nfailures, rules for mixing in synthetic data\nwithout collapse, and the dataset card that closes\nout Phase 2 before a run starts.
Input: raw examples (demonstrations, preference\njudgments, or task prompts) plus a routing decision\nfrom finetuning-method-selection.\nOutput format: a formatted, packed, validated\nJSONL dataset plus a completed dataset card — the\nPhase 2 artifact /finetune checks before launching\ntraining.
| Method | \nShape | \nRows | \n
|---|---|---|
| SFT, single-turn | \nInstruct (instruction/response or prompt/completion) | \n~1,000+ floor | \n
| SFT, multi-turn | \nConversation / ChatML messages list | \n~1,000+ floor | \n
| DPO / ORPO | \nPreference pair (prompt, chosen, rejected) | \nMethod-dependent, see preference-optimization | \n
| KTO | \nUnpaired (prompt, completion, label) | \nMethod-dependent, see preference-optimization | \n
| GRPO / RLVR | \nPrompt-only (prompt + verifier metadata) | \nMethod-dependent, see grpo-rlvr-training | \n
~1,000+ rows is the recommended floor for SFT,\nnot a target. Below it, a handful of low-quality\nor duplicate examples can dominate the gradient;\nabove it, quality over quantity — a smaller\nverified, deduplicated set beats a larger noisy one.
\nThe ChatML shape, for orientation; the other four\nformats plus a ShareGPT conversion note live in\nreferences/formats-and-templates.md:
{\"messages\": [\n {\"role\": \"user\", \"content\": \"...\"},\n {\"role\": \"assistant\", \"content\": \"...\"}\n]}\nApply the target model's chat template before\nany concatenation or packing, never after — packing\nraw text and templating the packed blob afterward\ncorrupts turn boundaries, landing role markers in\nthe wrong place relative to each example.
\nTrain on assistant responses only. Mask the\nloss (-100 in the labels tensor) over system/user\nturns and the template's own role markers — only\nassistant-turn content tokens contribute to loss.
Template/tokenizer mismatches are a top silent\nfailure mode. A model trained against one chat\ntemplate but served or evaluated with a different\none degrades without erroring. Verify the same\ntemplate string used in training is applied at\ninference and eval time.
\nKeep the dataset in messages shape and let\nthe trainer template and mask it\n(assistant_only_loss=True in current TRL) —\npre-rendering to a flat text field destroys the\nturn boundaries masking needs. Full code sketch:\nreferences/formats-and-templates.md. Sanity-check\nbefore training — decode only unmasked positions;\nexpect only assistant text:
keep = batch[\"labels\"][0] != -100\nprint(tokenizer.decode(batch[\"input_ids\"][0][keep]))\nWithout packing, 40–70% of compute is spent on\npadding — variable-length examples batched at a\nfixed sequence length waste the gap between each\nexample's length and the batch's max. Packing\nconcatenates multiple examples into one sequence\nup to the max length, cutting most of that waste.
\nPacking changes batch semantics. A packed\nsequence can contain several original examples, so\n\"steps per epoch\" and any LR schedule keyed to\nexample count shift once packing is on — recompute\nschedule milestones against packed-sequence count.
\nMANDATORY: decode and manually inspect 5–10\npacked sequences before scaling to a full run.\nConfirm example boundaries land where expected,\ntemplate markers are intact per sub-example, and\nthe loss mask is still assistant-only within each\npacked sequence. Not optional — packing bugs are\nsilent (the loss curve looks normal) and only\nsurface in eval quality, hours later:
\nfor seq in packed_dataset.select(range(10)):\n print(tokenizer.decode(seq[\"input_ids\"]))\nreferences/synthetic-data.md's\nReplay-Mix Construction recipe);\nstate which rows count as \"real\"\nin the dataset card rather than\nleaving the floor structurally\nunmeetable.references/synthetic-data.md.Every dataset that reaches training gets a card —\nthe required Phase 2 artifact /finetune checks\nbefore launching. The card is not free-form\ndocumentation; it MUST carry these fields:
trace-to-training-data output.references/synthetic-data.md.eval-harness-first run back to the checkpoint.A dataset missing any of these six fields isn't\nready for /finetune — the card is a gate, not a\nsummary written after the fact.
Before handing off to /finetune, confirm:
references/formats-and-templates.md — JSONL\nexamples per format, current-TRL masking code,\nand the ShareGPT conversion note.references/synthetic-data.md — generation-method\nranking, filter funnel, replay-mix construction,\nand teacher→student distillation pattern.Related skills: finetuning-method-selection routes\nhere; lora-qlora-recipes, vision-sft, and\npreference-optimization consume the datasets this\nskill produces; trace-to-training-data is the\nprovenance source for graded-trajectory datasets;\neval-harness-first grades the resulting checkpoint.