outline-agent
Step 1 of the PaperOrchestra pipeline (arXiv:2604.05018). Convert (idea.md, experimental_log.md, template.tex, conference_guidelines.md) into a strict JSON outline containing a plotting plan, literature search plan (Intro + Related Work), and section-level writing plan with citat
Install
npx skills add https://github.com/Ar9av/PaperOrchestra/tree/main/skills/outline-agent
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install ar9av-paperorchestra@llmmart
git clone https://github.com/Ar9av/PaperOrchestra.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole ar9av/paperorchestra collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Outline Agent (Step 1)
Faithful implementation of the Outline Agent from PaperOrchestra (Song et al., 2026, arXiv:2604.05018, App. F.1, pp. 40–44).
Cost: 1 LLM call.
Your task
Read four input files from the workspace and produce a single JSON object at
workspace/outline.json with three top-level keys:
plotting_plan— array of figure objectsintro_related_work_plan— object withintroduction_strategyandrelated_work_strategysection_plan— array of section objects, each withsection_titleandsubsections[]
How to do it
Read the verbatim prompt at
references/prompt.md. This is the exact Outline Agent system prompt from the paper. Use it as your system message.Prepend the Anti-Leakage Prompt from
../paper-orchestra/references/anti-leakage-prompt.md.Read the four input files:
workspace/inputs/idea.mdworkspace/inputs/experimental_log.mdworkspace/inputs/template.texworkspace/inputs/conference_guidelines.md
Synthesize across all four — the global instruction in the prompt is "Do not analyze inputs in isolation. You must synthesize information across all provided documents for every step."
Emit a single JSON object following the schema in
references/outline-schema.md. Cross-check againstreferences/outline_schema.json(machine-readable).Save to
workspace/outline.json.Validate:
python skills/outline-agent/scripts/validate_outline.py workspace/outline.jsonIf validation fails, fix the JSON and re-validate. Do not proceed to Step 2 or Step 3 with an invalid outline — every downstream agent depends on this schema.
Append §1 to research_brief.md (see
skills/shared/research_brief_template.md):After
outline.jsonpasses validation, append the §1 section toworkspace/research_brief.md(create the file if absent). Template:## §1 · Core Claim and Narrative _Written by: outline-agent, Step 1_ **Core claim:** <one-sentence contribution> **Narrative tension:** <gap this paper resolves> **Key novelty framing:** <how the contribution is framed relative to prior work> **Outline decisions:** - Plotting plan: <N> figures - Related Work clusters: <names> - Section structure: <section titles> **Potential weaknesses flagged at outline stage:** - <any claim in idea.md that may be hard to support>This is a free-form prose append; no machine-readable schema required.
Hard rules from the prompt (do not violate)
These are excerpted from references/prompt.md. The validator enforces them.
Plotting plan (Directive 1)
plot_typeMUST be exactly one of"plot"or"diagram".data_sourceMUST be exactly one of"idea.md","experimental_log.md", or"both".aspect_ratioMUST be exactly one of:"1:1","1:4","2:3","3:2","3:4","4:1","4:3","4:5","5:4","9:16","16:9","21:9".figure_idMUST be a semantically meaningful snake_case identifier (e.g.,fig_framework_overview,fig_ablation_study_parameter_sensitivity).figure_idMUST NOT contain the word"Figure".
Intro / Related Work strategy (Directive 2)
- Strictly separate Introduction (macro-level context, 10-20 papers, foundational + survey + impact) from Related Work (micro-level technical baselines, 30-50 papers, divided into 2-4 methodology clusters that directly compete with or precede the proposed approach).
- For each Related Work cluster: provide
methodology_cluster,sota_investigation_mission,limitation_hypothesis,limitation_search_queries,bridge_to_our_method. - CRITICAL TIMELINE RULE: Do not instruct searches for any papers
published after
{cutoff_date}. Derivecutoff_datefromconference_guidelines.md(e.g., "ICLR 2025 → cutoff October 2024", "CVPR 2025 → cutoff November 2024"). If unspecified, default to one month before today's date.
Section plan (Directive 3)
- Structural hierarchy: if Subsection X.1 is created, X.2 is mandatory. No orphaned subsections. Omit subsections entirely if a section does not require division.
- Content specificity: each
content_bulletsentry must reference source materials concretely. AVOID "Describe the model". REQUIRE "Formalize the Temporal-Aware Attention mechanism using Eq. 3 from idea.md." - Mandatory citations: every dataset, optimizer, metric, and
foundational architecture/model mentioned in
idea.mdorexperimental_log.mdMUST have a citation hint, no matter how ubiquitous (e.g., AdamW, ResNet, ImageNet, CLIP, Transformer, LLaMA, GPT, LLaVA). - Citation hint format:
- If you know the exact author and title:
"Author (Exact Paper Title)" - Otherwise:
"research paper or technical report introducing '[Exact Model/Dataset/Metric Name]'" - Do NOT guess or hallucinate authors.
- If you know the exact author and title:
Output
Exactly one file: workspace/outline.json. No prose, no code blocks, no
markdown. The Section Writing Agent and Literature Review Agent will parse
this JSON directly.
See references/example-output.json for a complete worked example from the
paper (App. F.1, pp. 43–44).
Resources
references/prompt.md— verbatim Outline Agent prompt from App. F.1references/outline-schema.md— prose explanation of the schemareferences/outline_schema.json— machine-readable JSON Schemareferences/example-output.json— example output from the paperreferences/allowed-values.md— enumerated allowed values for each enum fieldscripts/validate_outline.py— JSON Schema validatorskills/shared/research_brief_template.md— NEW §1 schema; append after outline.json passes validation
Files (paperorchestra)
-
references
-
allowed-values.md 2.2 KB
# Allowed enumerated values Source: arXiv:2604.05018, App. F.1 Outline Agent prompt. These are the only values the validator will accept for each enum field. Anything else will fail `validate_outline.py`. ## `plot_type` | Value | Use for | |---|---| | `"plot"` | Statistical/quantitative charts (line, bar, radar, scatter, box, histogram, heatmap, etc.). The `objective` field MUST also name the specific chart type. | | `"diagram"` | Conceptual / architectural diagrams (block diagram, flowchart, system overview). Rendered with graphviz or matplotlib patches. | ## `data_source` | Value | Meaning | |---|---| | `"idea.md"` | The figure visualizes a concept or architecture from the idea (typically diagrams). | | `"experimental_log.md"` | The figure plots numeric values from the experimental log. | | `"both"` | The figure combines conceptual context with empirical data. | ## `aspect_ratio` Exactly one of these 12 strings: ``` 1:1 1:4 2:3 3:2 3:4 4:1 4:3 4:5 5:4 9:16 16:9 21:9 ``` Common conventions: | Use | Ratio | |---|---| | Square ablation grid | `1:1` | | Single column figure (2-col layout) | `4:3` or `5:4` | | Wide cross-column figure (2-col layout) | `16:9` or `21:9` | | Vertical mobile / tall comparison | `9:16` or `2:3` | | Teaser banner | `21:9` | | Tall stacked subplots | `3:4` or `4:5` | The Plotting Agent's `render_matplotlib.py` translates the ratio into pixel dimensions at 300 DPI. ## `figure_id` rules - Snake_case: `^[a-z0-9_]+$` - Convention: prefix with `fig_` (e.g., `fig_main_results`, `fig_ablation_temperature_sensitivity`). - MUST be semantically meaningful (no `fig_1`, `fig_a`, `chart`). - MUST NOT contain the word `figure` as a substring after the `fig_` prefix. ## Citation hint format Two acceptable forms (per the prompt): | You know | Use | |---|---| | Exact author and title | `"Author (Exact Paper Title)"` — e.g., `"Vaswani et al. (Attention Is All You Need)"` | | You don't know either | `"research paper or technical report introducing '[Exact Model/Dataset/Metric Name]'"` — e.g., `"research paper or technical report introducing 'AdamW optimizer'"` | **Never guess authors.** The Literature Review Agent will use these hints as search queries — wrong authors lead to wrong citations. -
example-output.json 4.8 KB
{ "_comment": "Verbatim example from arXiv:2604.05018, App. F.1, pp. 43-44. Use as a reference for shape and style. Do not copy literally.", "plotting_plan": [ { "figure_id": "fig_teaser_fig_cross_modal_alignment_performance", "title": "Teaser: Cross-Modal Alignment Performance", "plot_type": "plot", "data_source": "experimental_log.md", "objective": "Visual summary (Radar Chart) demonstrating that our method achieves SOTA balance across 5 metrics.", "aspect_ratio": "16:9" } ], "intro_related_work_plan": { "introduction_strategy": { "hook_hypothesis": "Video-LLMs are currently the dominant paradigm for short clips.", "problem_gap_hypothesis": "Context window limits prevent scaling to >5s videos efficiently.", "search_directions": [ "Find highly cited papers establishing the real-world impact of context limits in video generation", "Search for published 'long-context video generation' surveys", "Identify foundational papers establishing causal video generation" ] }, "related_work_strategy": { "overview": "Investigate three specific paradigms to build a graph proving the necessity of our Sliding-Window approach.", "subsections": [ { "subsection_title": "2.1 Autoregressive Video Generation", "methodology_cluster": "Discrete Tokenization & Transformers", "sota_investigation_mission": "Identify the current SOTA autoregressive models from 2024-2025. Determine their maximum stable generation length.", "limitation_hypothesis": "These models suffer from 'drift' or 'error propagation' because they lack bidirectional context.", "limitation_search_queries": [ "Autoregressive video generation error propagation metrics", "Causal masking limitations in temporal video transformers" ], "bridge_to_our_method": "Our method introduces bidirectional blocks to fix the hypothesized drift issue." }, { "subsection_title": "2.2 Diffusion-Based Editing Frameworks", "methodology_cluster": "DDIM Inversion & Cross-Attention", "sota_investigation_mission": "Find recent papers using DDIM inversion for editing. Identify the standard benchmarks they use.", "limitation_hypothesis": "They fail at large structural changes because cross-attention maps are too rigid.", "limitation_search_queries": [ "DDIM inversion failure cases large motion", "Cross-attention control rigidity video editing" ], "bridge_to_our_method": "Our Flow-Guided Attention allows for spatial deformation, addressing rigidity." } ] } }, "section_plan": [ { "section_title": "Abstract", "subsections": [ { "subsection_title": "Abstract Content", "content_bullets": [ "Briefly state the problem of temporal inconsistency.", "Introduce the proposed method.", "Highlight key results." ], "citation_hints": [] } ] }, { "section_title": "3. Methodology", "subsections": [ { "subsection_title": "3.1 Temporal-Aware Attention Mechanism", "content_bullets": [ "Define the query-key matching logic", "Explain the masking strategy" ], "citation_hints": [ "Vaswani et al. (Attention Is All You Need)", "research paper or technical report introducing 'FlashAttention-2'" ] }, { "subsection_title": "3.2 Optimization Objective", "content_bullets": [ "Detail the loss function", "Discuss regularization terms" ], "citation_hints": [] } ] }, { "section_title": "4. Experiments", "subsections": [ { "subsection_title": "4.1 Experimental Setup", "content_bullets": [ "Implementation details", "Hyperparameters and datasets used" ], "citation_hints": [ "research paper or technical report introducing 'WebVid-10M'", "Paszke et al. (PyTorch: An Imperative Style, High-Performance Deep Learning Library)", "research paper or technical report introducing 'AdamW optimizer'", "research paper or technical report introducing 'Jaccard Index'" ] }, { "subsection_title": "4.2 Main Results", "content_bullets": [ "Comparison with Baselines", "Quantitative Analysis" ], "citation_hints": [ "Ho et al. (Denoising Diffusion Probabilistic Models)", "research paper or technical report introducing 'AVSegFormer baseline'" ] } ] } ] } -
outline-schema.md 4.2 KB
# Outline JSON Schema (prose) The Outline Agent emits a single JSON object with three top-level keys: `plotting_plan`, `intro_related_work_plan`, `section_plan`. Source: arXiv:2604.05018, App. F.1, pp. 43–44. The machine-readable JSON Schema is in `outline_schema.json` and is enforced by `scripts/validate_outline.py`. This document is the human-readable view. ## Top-level structure ```json { "plotting_plan": [ <FigureSpec>, ... ], "intro_related_work_plan": <IntroRelWorkPlan>, "section_plan": [ <SectionSpec>, ... ] } ``` ## FigureSpec ```json { "figure_id": "fig_framework_overview", // snake_case, no "Figure" "title": "Framework Overview", "plot_type": "diagram", // "plot" or "diagram" "data_source": "idea.md", // "idea.md" | "experimental_log.md" | "both" "objective": "Visual summary (Block Diagram) of the proposed pipeline.", "aspect_ratio": "16:9" // see allowed-values.md } ``` If `plot_type` is `"plot"`, the `objective` MUST name the specific chart type (e.g., "Visual summary (Radar Chart) demonstrating that our method achieves SOTA balance across 5 metrics."). ## IntroRelWorkPlan ```json { "introduction_strategy": { "hook_hypothesis": "Video-LLMs are currently the dominant paradigm for short clips.", "problem_gap_hypothesis": "Context window limits prevent scaling to >5s videos efficiently.", "search_directions": [ "Find highly cited papers establishing the real-world impact of context limits in video generation", "Search for published 'long-context video generation' surveys", "Identify foundational papers establishing causal video generation" ] }, "related_work_strategy": { "overview": "Investigate three specific paradigms to build a graph proving the necessity of our Sliding-Window approach.", "subsections": [ { "subsection_title": "2.1 Autoregressive Video Generation", "methodology_cluster": "Discrete Tokenization & Transformers", "sota_investigation_mission": "Identify the current SOTA autoregressive models from 2024-2025. Determine their maximum stable generation length.", "limitation_hypothesis": "These models suffer from 'drift' or 'error propagation' because they lack bidirectional context.", "limitation_search_queries": [ "Autoregressive video generation error propagation metrics", "Causal masking limitations in temporal video transformers" ], "bridge_to_our_method": "Our method introduces bidirectional blocks to fix the hypothesized drift issue." } ] } } ``` ## SectionSpec ```json { "section_title": "3. Methodology", "subsections": [ { "subsection_title": "3.1 Temporal-Aware Attention Mechanism", "content_bullets": [ "Define the query-key matching logic", "Explain the masking strategy" ], "citation_hints": [ "Vaswani et al. (Attention Is All You Need)", "research paper or technical report introducing 'FlashAttention-2'" ] }, { "subsection_title": "3.2 Optimization Objective", "content_bullets": [ "Detail the loss function", "Discuss regularization terms" ], "citation_hints": [] } ] } ``` ## Validation rules (enforced by `validate_outline.py`) 1. The top-level object MUST contain exactly the keys `plotting_plan`, `intro_related_work_plan`, `section_plan` (extras allowed but reported). 2. `plotting_plan` is a non-empty array. 3. Each FigureSpec has all six fields with correct enum values. 4. `figure_id` matches `^[a-z0-9_]+$` and does not contain `figure` as a substring after `fig_`. 5. `intro_related_work_plan.introduction_strategy` and `.related_work_strategy` both present. 6. `related_work_strategy.subsections` has 2–4 entries (the paper's "2-4 distinct methodology clusters" rule). 7. `section_plan` is a non-empty array. Each SectionSpec has `section_title` and `subsections`. 8. **Hierarchy rule**: if a section has any subsection, it must have ≥2 (no orphans). The validator emits a WARNING for sections with exactly 1 subsection. 9. `citation_hints` is always an array (may be empty). -
outline_schema.json 4 KB
{ "$schema": "https://json-schema.org/draft/2020-12/schema", "title": "PaperOrchestra Outline", "description": "Schema for outline.json — output of the Outline Agent (Step 1 of arXiv:2604.05018, App. F.1).", "type": "object", "required": ["plotting_plan", "intro_related_work_plan", "section_plan"], "additionalProperties": true, "properties": { "plotting_plan": { "type": "array", "minItems": 1, "items": { "$ref": "#/$defs/FigureSpec" } }, "intro_related_work_plan": { "type": "object", "required": ["introduction_strategy", "related_work_strategy"], "properties": { "introduction_strategy": { "type": "object", "required": ["hook_hypothesis", "problem_gap_hypothesis", "search_directions"], "properties": { "hook_hypothesis": { "type": "string", "minLength": 1 }, "problem_gap_hypothesis": { "type": "string", "minLength": 1 }, "search_directions": { "type": "array", "minItems": 3, "maxItems": 8, "items": { "type": "string", "minLength": 1 } } } }, "related_work_strategy": { "type": "object", "required": ["overview", "subsections"], "properties": { "overview": { "type": "string", "minLength": 1 }, "subsections": { "type": "array", "minItems": 2, "maxItems": 4, "items": { "$ref": "#/$defs/RelWorkSubsection" } } } } } }, "section_plan": { "type": "array", "minItems": 1, "items": { "$ref": "#/$defs/SectionSpec" } } }, "$defs": { "FigureSpec": { "type": "object", "required": ["figure_id", "title", "plot_type", "data_source", "objective", "aspect_ratio"], "properties": { "figure_id": { "type": "string", "pattern": "^[a-z0-9_]+$", "not": { "pattern": "figure(?!_)" } }, "title": { "type": "string", "minLength": 1 }, "plot_type": { "enum": ["plot", "diagram"] }, "data_source": { "enum": ["idea.md", "experimental_log.md", "both"] }, "objective": { "type": "string", "minLength": 1 }, "aspect_ratio": { "enum": ["1:1", "1:4", "2:3", "3:2", "3:4", "4:1", "4:3", "4:5", "5:4", "9:16", "16:9", "21:9"] } } }, "RelWorkSubsection": { "type": "object", "required": [ "subsection_title", "methodology_cluster", "sota_investigation_mission", "limitation_hypothesis", "limitation_search_queries", "bridge_to_our_method" ], "properties": { "subsection_title": { "type": "string", "minLength": 1 }, "methodology_cluster": { "type": "string", "minLength": 1 }, "sota_investigation_mission": { "type": "string", "minLength": 1 }, "limitation_hypothesis": { "type": "string", "minLength": 1 }, "limitation_search_queries": { "type": "array", "minItems": 1, "items": { "type": "string", "minLength": 1 } }, "bridge_to_our_method": { "type": "string", "minLength": 1 } } }, "SectionSpec": { "type": "object", "required": ["section_title", "subsections"], "properties": { "section_title": { "type": "string", "minLength": 1 }, "subsections": { "type": "array", "items": { "type": "object", "required": ["subsection_title", "content_bullets", "citation_hints"], "properties": { "subsection_title": { "type": "string", "minLength": 1 }, "content_bullets": { "type": "array", "minItems": 1, "items": { "type": "string", "minLength": 1 } }, "citation_hints": { "type": "array", "items": { "type": "string", "minLength": 1 } } } } } } } } } -
prompt.md 6.7 KB
# Outline Agent — verbatim prompt **Source: arXiv:2604.05018, Appendix F.1, pages 40–44 (verbatim).** This is the exact system prompt used by the Outline Agent in the paper. Reproduce it as your system message. Substitute `{cutoff_date}` with the research cutoff derived from `conference_guidelines.md`. --- ``` You are a senior AI researcher drafting a paper for a top-tier conference (e.g., NeurIPS, ICML, CVPR, ICLR). Your task is to convert the provided methodology and experimental logs into a detailed, venue-compliant paper outline. You must output a single JSON object. Your inputs are: 1. idea.md: A detailed summary of the methodology, core contributions, and theoretical framework. 2. experimental_log.md: A summary of experimental results, including raw data points, ablation studies, and performance metrics. 3. template.tex: The template structure. You must use the section commands (e.g., \section{...}) found here as your primary skeleton. 4. conference_guidelines.md: Formatting rules, specific page limits (for word count calculation), and mandatory sections. Processing Directives Global Instruction: Do not analyze inputs in isolation. You must synthesize information across all provided documents for every step. Directive 1: Plotting & Visualization Plan Synthesize experimental_log.md and idea.md to identify the most compelling evidence. - Determine which figures are essential to visually prove the hypothesis (e.g., convergence rates, qualitative visual comparisons). - The plot_type MUST be exactly "plot" or "diagram". If it is a plot, specify the specific chart type (e.g., Radar Chart) inside the objective. - The data_source MUST be exactly "idea.md", "experimental_log.md", or "both". - Determine the ideal aspect_ratio for each figure. The aspect_ratio MUST be exactly one of: "1:1", "1:4", "2:3", "3:2", "3:4", "4:1", "4:3", "4:5", "5:4", "9:16", "16:9", "21:9". - The figure_id MUST be a semantically meaningful string identifier summarizing the plot contents, like "fig_framework_overview" or "fig_ablation_study_parameter_sensitivity". It MUST NOT contain the word "Figure". - Output Focus: Create an array of objects for the plotting_plan key. Directive 2: Research Graph & Investigation Strategy (Intro & Related Work) Provide search instructions for a downstream literature review agent to build a Research Graph. Do not write the actual paper content. Prevent Citation Overlap: Strictly separate the scope of the Introduction from Related Work to ensure the agent searches for different tiers of literature. - Introduction: Focuses on macro-level context (foundational papers, surveys). - Related Work: Focuses on micro-level technical comparisons (recent SOTA baselines, benchmarks). Introduction Strategy (Macro-Level Context, 10-20 papers): - Hypotheses: Define the "Hook" (broad context) and "Problem Gap" to be verified. CRITICAL: Strictly scope the problem gap and claims to match the specific datasets and evaluations present in experimental_log.md. Do not over-claim generalization. - Search Directions: Provide 3-5 specific queries to find: 1. Papers establishing the real-world impact or urgency of the problem gap. 2. Good survey or review papers on the topic. 3. 3-5 Foundational papers that established the sub-field. Related Work Strategy (Micro-Level Technical Baselines, 30-50 papers): - Divide the field into 2-4 distinct methodology clusters that directly compete with or precede our approach. - For each cluster, define: 1. Methodology Cluster Name: The technical category. 2. SOTA Investigation: Instructions to find recent papers for conceptual context. CRITICAL TIMELINE RULE: Do not instruct searches for any papers published after {cutoff_date}. Furthermore, do NOT instruct the search for new "competitors" to beat if they are not exclusively in experimental_log.md. 3. Limitation Hypothesis: The suspected failure point of these competing methods, based on idea.md. 4. Limitation Search Queries: Highly specific, narrow queries to find papers documenting these exact limitations. 5. The Bridge: How our proposed method resolves this specific limitation. Output Focus: Populate the intro_related_work_plan key. Directive 3: Section Writing Plan & Sizing Constraints Outline the remaining sections (Abstract, Methodology, Experiments, Conclusion, Appendix) into a detailed structural plan. - Structural Hierarchy: If Subsection X.1 is created, X.2 is mandatory. Do not create orphaned subsections. Omit subsections entirely if a section does not require division. - Content Specificity: Explicitly reference source materials. - Avoid: "Describe the model." - Require: "Formalize the Temporal-Aware Attention mechanism using Eq. 3 from idea.md." - Mandatory Citations (citation_hints): You must provide targeted citation hints for all external dependencies. Every hint must point to a single, unambiguous canonical paper. - Required Coverage (EXHAUSTIVE): You MUST explicitly create a targeted citation_hints query for EVERY SINGLE dataset, optimizer, metric, and foundational architecture/model you mention, no matter how ubiquitous or obvious it seems (e.g., AdamW, ResNet, ImageNet, CLIP, Transformer, LLaMA, GPT, LLaVA). If it is in the experimental_log.md or idea.md, it MUST have a citation hint. 1. All baseline methods compared against. 2. All datasets evaluated on. 3. All standard metrics utilized. 4. All foundational algorithms (e.g., ResNet, Transformer, Diffusion models), foundational models (e.g., LLMs, VLMs), optimizers (e.g., AdamW), or frameworks built upon. - Format Constraint & Anti-Hallucination Rule: If you know the exact author and title, use "Author (Exact Paper Title)". DO NOT guess or hallucinate authors. If you do not know the exact author, use this format: "research paper or technical report introducing '[Exact Model/Dataset/Metric Name]'". - Output Focus: Populate the section_plan key. Guidelines on Scientific Depth & Mathematical Rigor: - Grounded Formalization: Propose explicit subsections for rigorous mathematical formulations (e.g., loss functions, core algorithms, theoretical proofs). You must base these strictly on idea.md and experimental_log.md; do not instruct the writing agent to include hallucinated variables or unsupported math. Strict Output Format (JSON) You must output a single, valid JSON object with the following three top-level keys: "plotting_plan", "intro_related_work_plan", and "section_plan". ``` The full example output JSON from the paper (App. F.1, pp. 43–44) is at `example-output.json`.
-
-
scripts
-
validate_outline.py 5 KB
#!/usr/bin/env python3 """ validate_outline.py — Validate workspace/outline.json against the JSON Schema. Performs the structural checks defined in `references/outline_schema.json` plus a few semantic checks the schema cannot express: - figure_id snake_case + no "figure" infix - "no orphaned subsection" hierarchy rule - related_work_strategy.subsections has 2-4 entries (paper rule) - emits WARNINGS for missing-but-recommended things, not just ERRORS Exit codes: 0 valid (warnings allowed) 1 schema or semantic errors Usage: python validate_outline.py /path/to/outline.json """ import json import os import re import sys try: import jsonschema from jsonschema import Draft202012Validator except ImportError: print("ERROR: jsonschema is required. Install with: pip install jsonschema", file=sys.stderr) sys.exit(2) HERE = os.path.dirname(os.path.abspath(__file__)) SCHEMA_PATH = os.path.normpath( os.path.join(HERE, "..", "references", "outline_schema.json") ) FIGID_RE = re.compile(r"^[a-z0-9_]+$") def load_schema() -> dict: with open(SCHEMA_PATH) as f: return json.load(f) def semantic_checks(outline: dict) -> tuple[list[str], list[str]]: """Return (errors, warnings).""" errors: list[str] = [] warnings: list[str] = [] # Figure ID checks for i, fig in enumerate(outline.get("plotting_plan", [])): fid = fig.get("figure_id", "") if not FIGID_RE.match(fid): errors.append( f"plotting_plan[{i}].figure_id={fid!r} is not snake_case " f"matching ^[a-z0-9_]+$" ) # The word "figure" must not appear (allow the conventional fig_ prefix) body = re.sub(r"^fig_", "", fid) if "figure" in body.lower(): errors.append( f"plotting_plan[{i}].figure_id={fid!r} contains 'figure' — " f"forbidden by Outline Agent prompt (App. F.1)" ) # If plot_type is "plot", objective should mention a specific chart type if fig.get("plot_type") == "plot": obj = fig.get("objective", "").lower() chart_keywords = [ "radar", "bar", "line", "scatter", "box", "violin", "heatmap", "histogram", "pie", "stacked", "grouped", "ridge", "density", "convergence", "training curve", ] if not any(k in obj for k in chart_keywords): warnings.append( f"plotting_plan[{i}] is plot_type='plot' but objective " f"does not name a specific chart type. The Outline Agent " f"prompt requires this." ) # Section hierarchy: no orphan subsections for i, section in enumerate(outline.get("section_plan", [])): subs = section.get("subsections") or [] if len(subs) == 1: warnings.append( f"section_plan[{i}] {section.get('section_title')!r} has only " f"one subsection — this is an orphan. The Outline Agent prompt " f"requires that if X.1 exists, X.2 must too." ) return errors, warnings def main() -> int: if len(sys.argv) != 2: print(__doc__, file=sys.stderr) return 2 outline_path = sys.argv[1] if not os.path.isfile(outline_path): print(f"ERROR: {outline_path} not found", file=sys.stderr) return 1 try: with open(outline_path) as f: outline = json.load(f) except json.JSONDecodeError as e: print(f"ERROR: {outline_path} is not valid JSON: {e}", file=sys.stderr) return 1 schema = load_schema() validator = Draft202012Validator(schema) schema_errors = sorted(validator.iter_errors(outline), key=lambda e: e.path) sem_errors, sem_warnings = semantic_checks(outline) if schema_errors: print("SCHEMA ERRORS:", file=sys.stderr) for e in schema_errors: path = "/".join(str(p) for p in e.absolute_path) or "(root)" print(f" {path}: {e.message}", file=sys.stderr) if sem_errors: print("SEMANTIC ERRORS:", file=sys.stderr) for msg in sem_errors: print(f" {msg}", file=sys.stderr) if sem_warnings: print("WARNINGS:") for msg in sem_warnings: print(f" {msg}") if schema_errors or sem_errors: print(f"\nFAIL: {outline_path}", file=sys.stderr) return 1 n_figs = len(outline.get("plotting_plan", [])) n_relwork = len( outline.get("intro_related_work_plan", {}) .get("related_work_strategy", {}) .get("subsections", []) ) n_sections = len(outline.get("section_plan", [])) n_hints = sum( len(sub.get("citation_hints", [])) for sec in outline.get("section_plan", []) for sub in (sec.get("subsections") or []) ) print( f"OK: {outline_path} — {n_figs} figures, " f"{n_relwork} related-work clusters, {n_sections} sections, " f"{n_hints} citation hints" ) return 0 if __name__ == "__main__": sys.exit(main())
-
-
SKILL.md 6.2 KB
--- name: outline-agent description: Step 1 of the PaperOrchestra pipeline (arXiv:2604.05018). Convert (idea.md, experimental_log.md, template.tex, conference_guidelines.md) into a strict JSON outline containing a plotting plan, literature search plan (Intro + Related Work), and section-level writing plan with citation hints. TRIGGER when the orchestrator delegates Step 1 or when the user asks to "outline a paper from raw materials" or "generate the paper structure". --- # Outline Agent (Step 1) Faithful implementation of the Outline Agent from PaperOrchestra (Song et al., 2026, arXiv:2604.05018, App. F.1, pp. 40–44). **Cost: 1 LLM call.** ## Your task Read four input files from the workspace and produce a single JSON object at `workspace/outline.json` with three top-level keys: - `plotting_plan` — array of figure objects - `intro_related_work_plan` — object with `introduction_strategy` and `related_work_strategy` - `section_plan` — array of section objects, each with `section_title` and `subsections[]` ## How to do it 1. **Read the verbatim prompt at `references/prompt.md`.** This is the exact Outline Agent system prompt from the paper. Use it as your system message. 2. **Prepend the Anti-Leakage Prompt** from `../paper-orchestra/references/anti-leakage-prompt.md`. 3. **Read the four input files**: - `workspace/inputs/idea.md` - `workspace/inputs/experimental_log.md` - `workspace/inputs/template.tex` - `workspace/inputs/conference_guidelines.md` 4. **Synthesize across all four** — the global instruction in the prompt is "Do not analyze inputs in isolation. You must synthesize information across all provided documents for every step." 5. **Emit a single JSON object** following the schema in `references/outline-schema.md`. Cross-check against `references/outline_schema.json` (machine-readable). 6. **Save to** `workspace/outline.json`. 7. **Validate**: ```bash python skills/outline-agent/scripts/validate_outline.py workspace/outline.json ``` If validation fails, fix the JSON and re-validate. Do not proceed to Step 2 or Step 3 with an invalid outline — every downstream agent depends on this schema. 8. **Append §1 to research_brief.md** (see `skills/shared/research_brief_template.md`): After `outline.json` passes validation, append the §1 section to `workspace/research_brief.md` (create the file if absent). Template: ```markdown ## §1 · Core Claim and Narrative _Written by: outline-agent, Step 1_ **Core claim:** <one-sentence contribution> **Narrative tension:** <gap this paper resolves> **Key novelty framing:** <how the contribution is framed relative to prior work> **Outline decisions:** - Plotting plan: <N> figures - Related Work clusters: <names> - Section structure: <section titles> **Potential weaknesses flagged at outline stage:** - <any claim in idea.md that may be hard to support> ``` This is a free-form prose append; no machine-readable schema required. ## Hard rules from the prompt (do not violate) These are excerpted from `references/prompt.md`. The validator enforces them. ### Plotting plan (Directive 1) - `plot_type` MUST be exactly one of `"plot"` or `"diagram"`. - `data_source` MUST be exactly one of `"idea.md"`, `"experimental_log.md"`, or `"both"`. - `aspect_ratio` MUST be exactly one of: `"1:1"`, `"1:4"`, `"2:3"`, `"3:2"`, `"3:4"`, `"4:1"`, `"4:3"`, `"4:5"`, `"5:4"`, `"9:16"`, `"16:9"`, `"21:9"`. - `figure_id` MUST be a semantically meaningful snake_case identifier (e.g., `fig_framework_overview`, `fig_ablation_study_parameter_sensitivity`). - `figure_id` MUST NOT contain the word `"Figure"`. ### Intro / Related Work strategy (Directive 2) - Strictly separate Introduction (macro-level context, 10-20 papers, foundational + survey + impact) from Related Work (micro-level technical baselines, 30-50 papers, divided into 2-4 methodology clusters that directly compete with or precede the proposed approach). - For each Related Work cluster: provide `methodology_cluster`, `sota_investigation_mission`, `limitation_hypothesis`, `limitation_search_queries`, `bridge_to_our_method`. - **CRITICAL TIMELINE RULE**: Do not instruct searches for any papers published after `{cutoff_date}`. Derive `cutoff_date` from `conference_guidelines.md` (e.g., "ICLR 2025 → cutoff October 2024", "CVPR 2025 → cutoff November 2024"). If unspecified, default to one month before today's date. ### Section plan (Directive 3) - **Structural hierarchy**: if Subsection X.1 is created, X.2 is mandatory. No orphaned subsections. Omit subsections entirely if a section does not require division. - **Content specificity**: each `content_bullets` entry must reference source materials concretely. AVOID "Describe the model". REQUIRE "Formalize the Temporal-Aware Attention mechanism using Eq. 3 from idea.md." - **Mandatory citations**: every dataset, optimizer, metric, and foundational architecture/model mentioned in `idea.md` or `experimental_log.md` MUST have a citation hint, no matter how ubiquitous (e.g., AdamW, ResNet, ImageNet, CLIP, Transformer, LLaMA, GPT, LLaVA). - **Citation hint format**: - If you know the exact author and title: `"Author (Exact Paper Title)"` - Otherwise: `"research paper or technical report introducing '[Exact Model/Dataset/Metric Name]'"` - **Do NOT guess or hallucinate authors.** ## Output Exactly one file: `workspace/outline.json`. No prose, no code blocks, no markdown. The Section Writing Agent and Literature Review Agent will parse this JSON directly. See `references/example-output.json` for a complete worked example from the paper (App. F.1, pp. 43–44). ## Resources - `references/prompt.md` — verbatim Outline Agent prompt from App. F.1 - `references/outline-schema.md` — prose explanation of the schema - `references/outline_schema.json` — machine-readable JSON Schema - `references/example-output.json` — example output from the paper - `references/allowed-values.md` — enumerated allowed values for each enum field - `scripts/validate_outline.py` — JSON Schema validator - `skills/shared/research_brief_template.md` — **NEW** §1 schema; append after outline.json passes validation
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.