Claude Skill

outline-agent

Step 1 of the PaperOrchestra pipeline (arXiv:2604.05018). Convert (idea.md, experimental_log.md, template.tex, conference_guidelines.md) into a strict JSON outline containing a plotting plan, literature search plan (Intro + Related Work), and section-level writing plan with citat

LLM Mart · 0 points · 0 views 0 listing impressions 0 install-command copies
Virus-scanned Reviewed automatically before listing.

Full trust report

Download ar9av-paperorchestra-skills_outline-agent-36c3cc4.zip · 14 KB
Part of ar9av/paperorchestra — 8 skills

Install

skills CLI npx skills add https://github.com/Ar9av/PaperOrchestra/tree/main/skills/outline-agent
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install ar9av-paperorchestra@llmmart
Git git clone https://github.com/Ar9av/PaperOrchestra.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole ar9av/paperorchestra collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Outline Agent (Step 1)

Faithful implementation of the Outline Agent from PaperOrchestra (Song et al., 2026, arXiv:2604.05018, App. F.1, pp. 40–44).

Cost: 1 LLM call.

Your task

Read four input files from the workspace and produce a single JSON object at workspace/outline.json with three top-level keys:

  • plotting_plan — array of figure objects
  • intro_related_work_plan — object with introduction_strategy and related_work_strategy
  • section_plan — array of section objects, each with section_title and subsections[]

How to do it

  1. Read the verbatim prompt at references/prompt.md. This is the exact Outline Agent system prompt from the paper. Use it as your system message.

  2. Prepend the Anti-Leakage Prompt from ../paper-orchestra/references/anti-leakage-prompt.md.

  3. Read the four input files:

    • workspace/inputs/idea.md
    • workspace/inputs/experimental_log.md
    • workspace/inputs/template.tex
    • workspace/inputs/conference_guidelines.md
  4. Synthesize across all four — the global instruction in the prompt is "Do not analyze inputs in isolation. You must synthesize information across all provided documents for every step."

  5. Emit a single JSON object following the schema in references/outline-schema.md. Cross-check against references/outline_schema.json (machine-readable).

  6. Save to workspace/outline.json.

  7. Validate:

    python skills/outline-agent/scripts/validate_outline.py workspace/outline.json
    

    If validation fails, fix the JSON and re-validate. Do not proceed to Step 2 or Step 3 with an invalid outline — every downstream agent depends on this schema.

  8. Append §1 to research_brief.md (see skills/shared/research_brief_template.md):

    After outline.json passes validation, append the §1 section to workspace/research_brief.md (create the file if absent). Template:

    ## §1 · Core Claim and Narrative
    _Written by: outline-agent, Step 1_
    
    **Core claim:** <one-sentence contribution>
    **Narrative tension:** <gap this paper resolves>
    **Key novelty framing:** <how the contribution is framed relative to prior work>
    **Outline decisions:**
    - Plotting plan: <N> figures
    - Related Work clusters: <names>
    - Section structure: <section titles>
    **Potential weaknesses flagged at outline stage:**
    - <any claim in idea.md that may be hard to support>
    

    This is a free-form prose append; no machine-readable schema required.

Hard rules from the prompt (do not violate)

These are excerpted from references/prompt.md. The validator enforces them.

Plotting plan (Directive 1)

  • plot_type MUST be exactly one of "plot" or "diagram".
  • data_source MUST be exactly one of "idea.md", "experimental_log.md", or "both".
  • aspect_ratio MUST be exactly one of: "1:1", "1:4", "2:3", "3:2", "3:4", "4:1", "4:3", "4:5", "5:4", "9:16", "16:9", "21:9".
  • figure_id MUST be a semantically meaningful snake_case identifier (e.g., fig_framework_overview, fig_ablation_study_parameter_sensitivity).
  • figure_id MUST NOT contain the word "Figure".

Intro / Related Work strategy (Directive 2)

  • Strictly separate Introduction (macro-level context, 10-20 papers, foundational + survey + impact) from Related Work (micro-level technical baselines, 30-50 papers, divided into 2-4 methodology clusters that directly compete with or precede the proposed approach).
  • For each Related Work cluster: provide methodology_cluster, sota_investigation_mission, limitation_hypothesis, limitation_search_queries, bridge_to_our_method.
  • CRITICAL TIMELINE RULE: Do not instruct searches for any papers published after {cutoff_date}. Derive cutoff_date from conference_guidelines.md (e.g., "ICLR 2025 → cutoff October 2024", "CVPR 2025 → cutoff November 2024"). If unspecified, default to one month before today's date.

Section plan (Directive 3)

  • Structural hierarchy: if Subsection X.1 is created, X.2 is mandatory. No orphaned subsections. Omit subsections entirely if a section does not require division.
  • Content specificity: each content_bullets entry must reference source materials concretely. AVOID "Describe the model". REQUIRE "Formalize the Temporal-Aware Attention mechanism using Eq. 3 from idea.md."
  • Mandatory citations: every dataset, optimizer, metric, and foundational architecture/model mentioned in idea.md or experimental_log.md MUST have a citation hint, no matter how ubiquitous (e.g., AdamW, ResNet, ImageNet, CLIP, Transformer, LLaMA, GPT, LLaVA).
  • Citation hint format:
    • If you know the exact author and title: "Author (Exact Paper Title)"
    • Otherwise: "research paper or technical report introducing '[Exact Model/Dataset/Metric Name]'"
    • Do NOT guess or hallucinate authors.

Output

Exactly one file: workspace/outline.json. No prose, no code blocks, no markdown. The Section Writing Agent and Literature Review Agent will parse this JSON directly.

See references/example-output.json for a complete worked example from the paper (App. F.1, pp. 43–44).

Resources

  • references/prompt.md — verbatim Outline Agent prompt from App. F.1
  • references/outline-schema.md — prose explanation of the schema
  • references/outline_schema.json — machine-readable JSON Schema
  • references/example-output.json — example output from the paper
  • references/allowed-values.md — enumerated allowed values for each enum field
  • scripts/validate_outline.py — JSON Schema validator
  • skills/shared/research_brief_template.md — NEW §1 schema; append after outline.json passes validation
Files (paperorchestra)
  • references
    • allowed-values.md 2.2 KB
      # Allowed enumerated values
      
      Source: arXiv:2604.05018, App. F.1 Outline Agent prompt.
      
      These are the only values the validator will accept for each enum field.
      Anything else will fail `validate_outline.py`.
      
      ## `plot_type`
      
      | Value | Use for |
      |---|---|
      | `"plot"` | Statistical/quantitative charts (line, bar, radar, scatter, box, histogram, heatmap, etc.). The `objective` field MUST also name the specific chart type. |
      | `"diagram"` | Conceptual / architectural diagrams (block diagram, flowchart, system overview). Rendered with graphviz or matplotlib patches. |
      
      ## `data_source`
      
      | Value | Meaning |
      |---|---|
      | `"idea.md"` | The figure visualizes a concept or architecture from the idea (typically diagrams). |
      | `"experimental_log.md"` | The figure plots numeric values from the experimental log. |
      | `"both"` | The figure combines conceptual context with empirical data. |
      
      ## `aspect_ratio`
      
      Exactly one of these 12 strings:
      
      ```
      1:1   1:4   2:3   3:2   3:4   4:1   4:3   4:5   5:4   9:16   16:9   21:9
      ```
      
      Common conventions:
      
      | Use | Ratio |
      |---|---|
      | Square ablation grid | `1:1` |
      | Single column figure (2-col layout) | `4:3` or `5:4` |
      | Wide cross-column figure (2-col layout) | `16:9` or `21:9` |
      | Vertical mobile / tall comparison | `9:16` or `2:3` |
      | Teaser banner | `21:9` |
      | Tall stacked subplots | `3:4` or `4:5` |
      
      The Plotting Agent's `render_matplotlib.py` translates the ratio into pixel
      dimensions at 300 DPI.
      
      ## `figure_id` rules
      
      - Snake_case: `^[a-z0-9_]+$`
      - Convention: prefix with `fig_` (e.g., `fig_main_results`,
        `fig_ablation_temperature_sensitivity`).
      - MUST be semantically meaningful (no `fig_1`, `fig_a`, `chart`).
      - MUST NOT contain the word `figure` as a substring after the `fig_` prefix.
      
      ## Citation hint format
      
      Two acceptable forms (per the prompt):
      
      | You know | Use |
      |---|---|
      | Exact author and title | `"Author (Exact Paper Title)"` — e.g., `"Vaswani et al. (Attention Is All You Need)"` |
      | You don't know either | `"research paper or technical report introducing '[Exact Model/Dataset/Metric Name]'"` — e.g., `"research paper or technical report introducing 'AdamW optimizer'"` |
      
      **Never guess authors.** The Literature Review Agent will use these hints as
      search queries — wrong authors lead to wrong citations.
      
    • example-output.json 4.8 KB
      {
        "_comment": "Verbatim example from arXiv:2604.05018, App. F.1, pp. 43-44. Use as a reference for shape and style. Do not copy literally.",
        "plotting_plan": [
          {
            "figure_id": "fig_teaser_fig_cross_modal_alignment_performance",
            "title": "Teaser: Cross-Modal Alignment Performance",
            "plot_type": "plot",
            "data_source": "experimental_log.md",
            "objective": "Visual summary (Radar Chart) demonstrating that our method achieves SOTA balance across 5 metrics.",
            "aspect_ratio": "16:9"
          }
        ],
        "intro_related_work_plan": {
          "introduction_strategy": {
            "hook_hypothesis": "Video-LLMs are currently the dominant paradigm for short clips.",
            "problem_gap_hypothesis": "Context window limits prevent scaling to >5s videos efficiently.",
            "search_directions": [
              "Find highly cited papers establishing the real-world impact of context limits in video generation",
              "Search for published 'long-context video generation' surveys",
              "Identify foundational papers establishing causal video generation"
            ]
          },
          "related_work_strategy": {
            "overview": "Investigate three specific paradigms to build a graph proving the necessity of our Sliding-Window approach.",
            "subsections": [
              {
                "subsection_title": "2.1 Autoregressive Video Generation",
                "methodology_cluster": "Discrete Tokenization & Transformers",
                "sota_investigation_mission": "Identify the current SOTA autoregressive models from 2024-2025. Determine their maximum stable generation length.",
                "limitation_hypothesis": "These models suffer from 'drift' or 'error propagation' because they lack bidirectional context.",
                "limitation_search_queries": [
                  "Autoregressive video generation error propagation metrics",
                  "Causal masking limitations in temporal video transformers"
                ],
                "bridge_to_our_method": "Our method introduces bidirectional blocks to fix the hypothesized drift issue."
              },
              {
                "subsection_title": "2.2 Diffusion-Based Editing Frameworks",
                "methodology_cluster": "DDIM Inversion & Cross-Attention",
                "sota_investigation_mission": "Find recent papers using DDIM inversion for editing. Identify the standard benchmarks they use.",
                "limitation_hypothesis": "They fail at large structural changes because cross-attention maps are too rigid.",
                "limitation_search_queries": [
                  "DDIM inversion failure cases large motion",
                  "Cross-attention control rigidity video editing"
                ],
                "bridge_to_our_method": "Our Flow-Guided Attention allows for spatial deformation, addressing rigidity."
              }
            ]
          }
        },
        "section_plan": [
          {
            "section_title": "Abstract",
            "subsections": [
              {
                "subsection_title": "Abstract Content",
                "content_bullets": [
                  "Briefly state the problem of temporal inconsistency.",
                  "Introduce the proposed method.",
                  "Highlight key results."
                ],
                "citation_hints": []
              }
            ]
          },
          {
            "section_title": "3. Methodology",
            "subsections": [
              {
                "subsection_title": "3.1 Temporal-Aware Attention Mechanism",
                "content_bullets": [
                  "Define the query-key matching logic",
                  "Explain the masking strategy"
                ],
                "citation_hints": [
                  "Vaswani et al. (Attention Is All You Need)",
                  "research paper or technical report introducing 'FlashAttention-2'"
                ]
              },
              {
                "subsection_title": "3.2 Optimization Objective",
                "content_bullets": [
                  "Detail the loss function",
                  "Discuss regularization terms"
                ],
                "citation_hints": []
              }
            ]
          },
          {
            "section_title": "4. Experiments",
            "subsections": [
              {
                "subsection_title": "4.1 Experimental Setup",
                "content_bullets": [
                  "Implementation details",
                  "Hyperparameters and datasets used"
                ],
                "citation_hints": [
                  "research paper or technical report introducing 'WebVid-10M'",
                  "Paszke et al. (PyTorch: An Imperative Style, High-Performance Deep Learning Library)",
                  "research paper or technical report introducing 'AdamW optimizer'",
                  "research paper or technical report introducing 'Jaccard Index'"
                ]
              },
              {
                "subsection_title": "4.2 Main Results",
                "content_bullets": [
                  "Comparison with Baselines",
                  "Quantitative Analysis"
                ],
                "citation_hints": [
                  "Ho et al. (Denoising Diffusion Probabilistic Models)",
                  "research paper or technical report introducing 'AVSegFormer baseline'"
                ]
              }
            ]
          }
        ]
      }
      
    • outline-schema.md 4.2 KB
      # Outline JSON Schema (prose)
      
      The Outline Agent emits a single JSON object with three top-level keys:
      `plotting_plan`, `intro_related_work_plan`, `section_plan`. Source:
      arXiv:2604.05018, App. F.1, pp. 43–44.
      
      The machine-readable JSON Schema is in `outline_schema.json` and is enforced
      by `scripts/validate_outline.py`. This document is the human-readable view.
      
      ## Top-level structure
      
      ```json
      {
        "plotting_plan": [ <FigureSpec>, ... ],
        "intro_related_work_plan": <IntroRelWorkPlan>,
        "section_plan": [ <SectionSpec>, ... ]
      }
      ```
      
      ## FigureSpec
      
      ```json
      {
        "figure_id":     "fig_framework_overview",            // snake_case, no "Figure"
        "title":         "Framework Overview",
        "plot_type":     "diagram",                           // "plot" or "diagram"
        "data_source":   "idea.md",                           // "idea.md" | "experimental_log.md" | "both"
        "objective":     "Visual summary (Block Diagram) of the proposed pipeline.",
        "aspect_ratio":  "16:9"                               // see allowed-values.md
      }
      ```
      
      If `plot_type` is `"plot"`, the `objective` MUST name the specific chart type
      (e.g., "Visual summary (Radar Chart) demonstrating that our method achieves
      SOTA balance across 5 metrics.").
      
      ## IntroRelWorkPlan
      
      ```json
      {
        "introduction_strategy": {
          "hook_hypothesis":        "Video-LLMs are currently the dominant paradigm for short clips.",
          "problem_gap_hypothesis": "Context window limits prevent scaling to >5s videos efficiently.",
          "search_directions": [
            "Find highly cited papers establishing the real-world impact of context limits in video generation",
            "Search for published 'long-context video generation' surveys",
            "Identify foundational papers establishing causal video generation"
          ]
        },
        "related_work_strategy": {
          "overview": "Investigate three specific paradigms to build a graph proving the necessity of our Sliding-Window approach.",
          "subsections": [
            {
              "subsection_title":          "2.1 Autoregressive Video Generation",
              "methodology_cluster":       "Discrete Tokenization & Transformers",
              "sota_investigation_mission": "Identify the current SOTA autoregressive models from 2024-2025. Determine their maximum stable generation length.",
              "limitation_hypothesis":     "These models suffer from 'drift' or 'error propagation' because they lack bidirectional context.",
              "limitation_search_queries": [
                "Autoregressive video generation error propagation metrics",
                "Causal masking limitations in temporal video transformers"
              ],
              "bridge_to_our_method": "Our method introduces bidirectional blocks to fix the hypothesized drift issue."
            }
          ]
        }
      }
      ```
      
      ## SectionSpec
      
      ```json
      {
        "section_title": "3. Methodology",
        "subsections": [
          {
            "subsection_title": "3.1 Temporal-Aware Attention Mechanism",
            "content_bullets": [
              "Define the query-key matching logic",
              "Explain the masking strategy"
            ],
            "citation_hints": [
              "Vaswani et al. (Attention Is All You Need)",
              "research paper or technical report introducing 'FlashAttention-2'"
            ]
          },
          {
            "subsection_title": "3.2 Optimization Objective",
            "content_bullets": [
              "Detail the loss function",
              "Discuss regularization terms"
            ],
            "citation_hints": []
          }
        ]
      }
      ```
      
      ## Validation rules (enforced by `validate_outline.py`)
      
      1. The top-level object MUST contain exactly the keys `plotting_plan`,
         `intro_related_work_plan`, `section_plan` (extras allowed but reported).
      2. `plotting_plan` is a non-empty array.
      3. Each FigureSpec has all six fields with correct enum values.
      4. `figure_id` matches `^[a-z0-9_]+$` and does not contain `figure` as a
         substring after `fig_`.
      5. `intro_related_work_plan.introduction_strategy` and `.related_work_strategy`
         both present.
      6. `related_work_strategy.subsections` has 2–4 entries (the paper's
         "2-4 distinct methodology clusters" rule).
      7. `section_plan` is a non-empty array. Each SectionSpec has `section_title`
         and `subsections`.
      8. **Hierarchy rule**: if a section has any subsection, it must have ≥2
         (no orphans). The validator emits a WARNING for sections with exactly 1
         subsection.
      9. `citation_hints` is always an array (may be empty).
      
    • outline_schema.json 4 KB
      {
        "$schema": "https://json-schema.org/draft/2020-12/schema",
        "title": "PaperOrchestra Outline",
        "description": "Schema for outline.json — output of the Outline Agent (Step 1 of arXiv:2604.05018, App. F.1).",
        "type": "object",
        "required": ["plotting_plan", "intro_related_work_plan", "section_plan"],
        "additionalProperties": true,
        "properties": {
          "plotting_plan": {
            "type": "array",
            "minItems": 1,
            "items": { "$ref": "#/$defs/FigureSpec" }
          },
          "intro_related_work_plan": {
            "type": "object",
            "required": ["introduction_strategy", "related_work_strategy"],
            "properties": {
              "introduction_strategy": {
                "type": "object",
                "required": ["hook_hypothesis", "problem_gap_hypothesis", "search_directions"],
                "properties": {
                  "hook_hypothesis":        { "type": "string", "minLength": 1 },
                  "problem_gap_hypothesis": { "type": "string", "minLength": 1 },
                  "search_directions": {
                    "type": "array",
                    "minItems": 3,
                    "maxItems": 8,
                    "items": { "type": "string", "minLength": 1 }
                  }
                }
              },
              "related_work_strategy": {
                "type": "object",
                "required": ["overview", "subsections"],
                "properties": {
                  "overview": { "type": "string", "minLength": 1 },
                  "subsections": {
                    "type": "array",
                    "minItems": 2,
                    "maxItems": 4,
                    "items": { "$ref": "#/$defs/RelWorkSubsection" }
                  }
                }
              }
            }
          },
          "section_plan": {
            "type": "array",
            "minItems": 1,
            "items": { "$ref": "#/$defs/SectionSpec" }
          }
        },
        "$defs": {
          "FigureSpec": {
            "type": "object",
            "required": ["figure_id", "title", "plot_type", "data_source", "objective", "aspect_ratio"],
            "properties": {
              "figure_id": {
                "type": "string",
                "pattern": "^[a-z0-9_]+$",
                "not": { "pattern": "figure(?!_)" }
              },
              "title":     { "type": "string", "minLength": 1 },
              "plot_type": { "enum": ["plot", "diagram"] },
              "data_source": { "enum": ["idea.md", "experimental_log.md", "both"] },
              "objective": { "type": "string", "minLength": 1 },
              "aspect_ratio": {
                "enum": ["1:1", "1:4", "2:3", "3:2", "3:4", "4:1", "4:3", "4:5", "5:4", "9:16", "16:9", "21:9"]
              }
            }
          },
          "RelWorkSubsection": {
            "type": "object",
            "required": [
              "subsection_title", "methodology_cluster", "sota_investigation_mission",
              "limitation_hypothesis", "limitation_search_queries", "bridge_to_our_method"
            ],
            "properties": {
              "subsection_title":           { "type": "string", "minLength": 1 },
              "methodology_cluster":        { "type": "string", "minLength": 1 },
              "sota_investigation_mission": { "type": "string", "minLength": 1 },
              "limitation_hypothesis":      { "type": "string", "minLength": 1 },
              "limitation_search_queries": {
                "type": "array",
                "minItems": 1,
                "items": { "type": "string", "minLength": 1 }
              },
              "bridge_to_our_method":       { "type": "string", "minLength": 1 }
            }
          },
          "SectionSpec": {
            "type": "object",
            "required": ["section_title", "subsections"],
            "properties": {
              "section_title": { "type": "string", "minLength": 1 },
              "subsections": {
                "type": "array",
                "items": {
                  "type": "object",
                  "required": ["subsection_title", "content_bullets", "citation_hints"],
                  "properties": {
                    "subsection_title": { "type": "string", "minLength": 1 },
                    "content_bullets": {
                      "type": "array",
                      "minItems": 1,
                      "items": { "type": "string", "minLength": 1 }
                    },
                    "citation_hints": {
                      "type": "array",
                      "items": { "type": "string", "minLength": 1 }
                    }
                  }
                }
              }
            }
          }
        }
      }
      
    • prompt.md 6.7 KB
      # Outline Agent — verbatim prompt
      
      **Source: arXiv:2604.05018, Appendix F.1, pages 40–44 (verbatim).**
      
      This is the exact system prompt used by the Outline Agent in the paper.
      Reproduce it as your system message. Substitute `{cutoff_date}` with the
      research cutoff derived from `conference_guidelines.md`.
      
      ---
      
      ```
      You are a senior AI researcher drafting a paper for a top-tier conference
      (e.g., NeurIPS, ICML, CVPR, ICLR). Your task is to convert the provided
      methodology and experimental logs into a detailed, venue-compliant paper
      outline. You must output a single JSON object.
      
      Your inputs are:
        1. idea.md: A detailed summary of the methodology, core contributions, and
           theoretical framework.
        2. experimental_log.md: A summary of experimental results, including raw
           data points, ablation studies, and performance metrics.
        3. template.tex: The template structure. You must use the section commands
           (e.g., \section{...}) found here as your primary skeleton.
        4. conference_guidelines.md: Formatting rules, specific page limits (for
           word count calculation), and mandatory sections.
      
      Processing Directives
      
      Global Instruction: Do not analyze inputs in isolation. You must synthesize
      information across all provided documents for every step.
      
      Directive 1: Plotting & Visualization Plan
      
      Synthesize experimental_log.md and idea.md to identify the most compelling
      evidence.
      
        - Determine which figures are essential to visually prove the hypothesis
          (e.g., convergence rates, qualitative visual comparisons).
        - The plot_type MUST be exactly "plot" or "diagram". If it is a plot,
          specify the specific chart type (e.g., Radar Chart) inside the objective.
        - The data_source MUST be exactly "idea.md", "experimental_log.md", or
          "both".
        - Determine the ideal aspect_ratio for each figure. The aspect_ratio MUST
          be exactly one of: "1:1", "1:4", "2:3", "3:2", "3:4", "4:1", "4:3",
          "4:5", "5:4", "9:16", "16:9", "21:9".
        - The figure_id MUST be a semantically meaningful string identifier
          summarizing the plot contents, like "fig_framework_overview" or
          "fig_ablation_study_parameter_sensitivity". It MUST NOT contain the word
          "Figure".
        - Output Focus: Create an array of objects for the plotting_plan key.
      
      Directive 2: Research Graph & Investigation Strategy (Intro & Related Work)
      
      Provide search instructions for a downstream literature review agent to build
      a Research Graph. Do not write the actual paper content.
      
      Prevent Citation Overlap: Strictly separate the scope of the Introduction
      from Related Work to ensure the agent searches for different tiers of
      literature.
      
        - Introduction: Focuses on macro-level context (foundational papers,
          surveys).
        - Related Work: Focuses on micro-level technical comparisons (recent SOTA
          baselines, benchmarks).
      
      Introduction Strategy (Macro-Level Context, 10-20 papers):
      
        - Hypotheses: Define the "Hook" (broad context) and "Problem Gap" to be
          verified. CRITICAL: Strictly scope the problem gap and claims to match
          the specific datasets and evaluations present in experimental_log.md.
          Do not over-claim generalization.
        - Search Directions: Provide 3-5 specific queries to find:
          1. Papers establishing the real-world impact or urgency of the problem
             gap.
          2. Good survey or review papers on the topic.
          3. 3-5 Foundational papers that established the sub-field.
      
      Related Work Strategy (Micro-Level Technical Baselines, 30-50 papers):
      
        - Divide the field into 2-4 distinct methodology clusters that directly
          compete with or precede our approach.
        - For each cluster, define:
          1. Methodology Cluster Name: The technical category.
          2. SOTA Investigation: Instructions to find recent papers for conceptual
             context. CRITICAL TIMELINE RULE: Do not instruct searches for any
             papers published after {cutoff_date}. Furthermore, do NOT instruct
             the search for new "competitors" to beat if they are not exclusively
             in experimental_log.md.
          3. Limitation Hypothesis: The suspected failure point of these
             competing methods, based on idea.md.
          4. Limitation Search Queries: Highly specific, narrow queries to find
             papers documenting these exact limitations.
          5. The Bridge: How our proposed method resolves this specific limitation.
      
      Output Focus: Populate the intro_related_work_plan key.
      
      Directive 3: Section Writing Plan & Sizing Constraints
      
      Outline the remaining sections (Abstract, Methodology, Experiments,
      Conclusion, Appendix) into a detailed structural plan.
      
        - Structural Hierarchy: If Subsection X.1 is created, X.2 is mandatory.
          Do not create orphaned subsections. Omit subsections entirely if a
          section does not require division.
        - Content Specificity: Explicitly reference source materials.
          - Avoid: "Describe the model."
          - Require: "Formalize the Temporal-Aware Attention mechanism using
            Eq. 3 from idea.md."
        - Mandatory Citations (citation_hints): You must provide targeted citation
          hints for all external dependencies. Every hint must point to a single,
          unambiguous canonical paper.
          - Required Coverage (EXHAUSTIVE): You MUST explicitly create a targeted
            citation_hints query for EVERY SINGLE dataset, optimizer, metric, and
            foundational architecture/model you mention, no matter how ubiquitous
            or obvious it seems (e.g., AdamW, ResNet, ImageNet, CLIP, Transformer,
            LLaMA, GPT, LLaVA). If it is in the experimental_log.md or idea.md,
            it MUST have a citation hint.
            1. All baseline methods compared against.
            2. All datasets evaluated on.
            3. All standard metrics utilized.
            4. All foundational algorithms (e.g., ResNet, Transformer, Diffusion
               models), foundational models (e.g., LLMs, VLMs), optimizers
               (e.g., AdamW), or frameworks built upon.
          - Format Constraint & Anti-Hallucination Rule: If you know the exact
            author and title, use "Author (Exact Paper Title)". DO NOT guess or
            hallucinate authors. If you do not know the exact author, use this
            format: "research paper or technical report introducing '[Exact
            Model/Dataset/Metric Name]'".
        - Output Focus: Populate the section_plan key.
      
      Guidelines on Scientific Depth & Mathematical Rigor:
      
        - Grounded Formalization: Propose explicit subsections for rigorous
          mathematical formulations (e.g., loss functions, core algorithms,
          theoretical proofs). You must base these strictly on idea.md and
          experimental_log.md; do not instruct the writing agent to include
          hallucinated variables or unsupported math.
      
      Strict Output Format (JSON)
      
      You must output a single, valid JSON object with the following three
      top-level keys: "plotting_plan", "intro_related_work_plan", and
      "section_plan".
      ```
      
      The full example output JSON from the paper (App. F.1, pp. 43–44) is at
      `example-output.json`.
      
  • scripts
    • validate_outline.py 5 KB
      #!/usr/bin/env python3
      """
      validate_outline.py — Validate workspace/outline.json against the JSON Schema.
      
      Performs the structural checks defined in
      `references/outline_schema.json` plus a few semantic checks the schema
      cannot express:
      
        - figure_id snake_case + no "figure" infix
        - "no orphaned subsection" hierarchy rule
        - related_work_strategy.subsections has 2-4 entries (paper rule)
        - emits WARNINGS for missing-but-recommended things, not just ERRORS
      
      Exit codes:
          0  valid (warnings allowed)
          1  schema or semantic errors
      
      Usage:
          python validate_outline.py /path/to/outline.json
      """
      import json
      import os
      import re
      import sys
      
      try:
          import jsonschema
          from jsonschema import Draft202012Validator
      except ImportError:
          print("ERROR: jsonschema is required. Install with: pip install jsonschema",
                file=sys.stderr)
          sys.exit(2)
      
      HERE = os.path.dirname(os.path.abspath(__file__))
      SCHEMA_PATH = os.path.normpath(
          os.path.join(HERE, "..", "references", "outline_schema.json")
      )
      
      FIGID_RE = re.compile(r"^[a-z0-9_]+$")
      
      
      def load_schema() -> dict:
          with open(SCHEMA_PATH) as f:
              return json.load(f)
      
      
      def semantic_checks(outline: dict) -> tuple[list[str], list[str]]:
          """Return (errors, warnings)."""
          errors: list[str] = []
          warnings: list[str] = []
      
          # Figure ID checks
          for i, fig in enumerate(outline.get("plotting_plan", [])):
              fid = fig.get("figure_id", "")
              if not FIGID_RE.match(fid):
                  errors.append(
                      f"plotting_plan[{i}].figure_id={fid!r} is not snake_case "
                      f"matching ^[a-z0-9_]+$"
                  )
              # The word "figure" must not appear (allow the conventional fig_ prefix)
              body = re.sub(r"^fig_", "", fid)
              if "figure" in body.lower():
                  errors.append(
                      f"plotting_plan[{i}].figure_id={fid!r} contains 'figure' — "
                      f"forbidden by Outline Agent prompt (App. F.1)"
                  )
              # If plot_type is "plot", objective should mention a specific chart type
              if fig.get("plot_type") == "plot":
                  obj = fig.get("objective", "").lower()
                  chart_keywords = [
                      "radar", "bar", "line", "scatter", "box", "violin",
                      "heatmap", "histogram", "pie", "stacked", "grouped",
                      "ridge", "density", "convergence", "training curve",
                  ]
                  if not any(k in obj for k in chart_keywords):
                      warnings.append(
                          f"plotting_plan[{i}] is plot_type='plot' but objective "
                          f"does not name a specific chart type. The Outline Agent "
                          f"prompt requires this."
                      )
      
          # Section hierarchy: no orphan subsections
          for i, section in enumerate(outline.get("section_plan", [])):
              subs = section.get("subsections") or []
              if len(subs) == 1:
                  warnings.append(
                      f"section_plan[{i}] {section.get('section_title')!r} has only "
                      f"one subsection — this is an orphan. The Outline Agent prompt "
                      f"requires that if X.1 exists, X.2 must too."
                  )
      
          return errors, warnings
      
      
      def main() -> int:
          if len(sys.argv) != 2:
              print(__doc__, file=sys.stderr)
              return 2
      
          outline_path = sys.argv[1]
          if not os.path.isfile(outline_path):
              print(f"ERROR: {outline_path} not found", file=sys.stderr)
              return 1
      
          try:
              with open(outline_path) as f:
                  outline = json.load(f)
          except json.JSONDecodeError as e:
              print(f"ERROR: {outline_path} is not valid JSON: {e}", file=sys.stderr)
              return 1
      
          schema = load_schema()
          validator = Draft202012Validator(schema)
          schema_errors = sorted(validator.iter_errors(outline), key=lambda e: e.path)
      
          sem_errors, sem_warnings = semantic_checks(outline)
      
          if schema_errors:
              print("SCHEMA ERRORS:", file=sys.stderr)
              for e in schema_errors:
                  path = "/".join(str(p) for p in e.absolute_path) or "(root)"
                  print(f"  {path}: {e.message}", file=sys.stderr)
      
          if sem_errors:
              print("SEMANTIC ERRORS:", file=sys.stderr)
              for msg in sem_errors:
                  print(f"  {msg}", file=sys.stderr)
      
          if sem_warnings:
              print("WARNINGS:")
              for msg in sem_warnings:
                  print(f"  {msg}")
      
          if schema_errors or sem_errors:
              print(f"\nFAIL: {outline_path}", file=sys.stderr)
              return 1
      
          n_figs = len(outline.get("plotting_plan", []))
          n_relwork = len(
              outline.get("intro_related_work_plan", {})
              .get("related_work_strategy", {})
              .get("subsections", [])
          )
          n_sections = len(outline.get("section_plan", []))
          n_hints = sum(
              len(sub.get("citation_hints", []))
              for sec in outline.get("section_plan", [])
              for sub in (sec.get("subsections") or [])
          )
          print(
              f"OK: {outline_path} — {n_figs} figures, "
              f"{n_relwork} related-work clusters, {n_sections} sections, "
              f"{n_hints} citation hints"
          )
          return 0
      
      
      if __name__ == "__main__":
          sys.exit(main())
      
  • SKILL.md 6.2 KB
    ---
    name: outline-agent
    description: Step 1 of the PaperOrchestra pipeline (arXiv:2604.05018). Convert (idea.md, experimental_log.md, template.tex, conference_guidelines.md) into a strict JSON outline containing a plotting plan, literature search plan (Intro + Related Work), and section-level writing plan with citation hints. TRIGGER when the orchestrator delegates Step 1 or when the user asks to "outline a paper from raw materials" or "generate the paper structure".
    ---
    
    # Outline Agent (Step 1)
    
    Faithful implementation of the Outline Agent from PaperOrchestra
    (Song et al., 2026, arXiv:2604.05018, App. F.1, pp. 40–44).
    
    **Cost: 1 LLM call.**
    
    ## Your task
    
    Read four input files from the workspace and produce a single JSON object at
    `workspace/outline.json` with three top-level keys:
    
    - `plotting_plan` — array of figure objects
    - `intro_related_work_plan` — object with `introduction_strategy` and `related_work_strategy`
    - `section_plan` — array of section objects, each with `section_title` and `subsections[]`
    
    ## How to do it
    
    1. **Read the verbatim prompt at `references/prompt.md`.** This is the exact
       Outline Agent system prompt from the paper. Use it as your system message.
    2. **Prepend the Anti-Leakage Prompt** from
       `../paper-orchestra/references/anti-leakage-prompt.md`.
    3. **Read the four input files**:
       - `workspace/inputs/idea.md`
       - `workspace/inputs/experimental_log.md`
       - `workspace/inputs/template.tex`
       - `workspace/inputs/conference_guidelines.md`
    4. **Synthesize across all four** — the global instruction in the prompt is
       "Do not analyze inputs in isolation. You must synthesize information across
       all provided documents for every step."
    5. **Emit a single JSON object** following the schema in
       `references/outline-schema.md`. Cross-check against
       `references/outline_schema.json` (machine-readable).
    6. **Save to** `workspace/outline.json`.
    7. **Validate**:
       ```bash
       python skills/outline-agent/scripts/validate_outline.py workspace/outline.json
       ```
       If validation fails, fix the JSON and re-validate. Do not proceed to Step 2
       or Step 3 with an invalid outline — every downstream agent depends on this
       schema.
    
    8. **Append §1 to research_brief.md** (see `skills/shared/research_brief_template.md`):
    
       After `outline.json` passes validation, append the §1 section to
       `workspace/research_brief.md` (create the file if absent). Template:
    
       ```markdown
       ## §1 · Core Claim and Narrative
       _Written by: outline-agent, Step 1_
    
       **Core claim:** <one-sentence contribution>
       **Narrative tension:** <gap this paper resolves>
       **Key novelty framing:** <how the contribution is framed relative to prior work>
       **Outline decisions:**
       - Plotting plan: <N> figures
       - Related Work clusters: <names>
       - Section structure: <section titles>
       **Potential weaknesses flagged at outline stage:**
       - <any claim in idea.md that may be hard to support>
       ```
    
       This is a free-form prose append; no machine-readable schema required.
    
    ## Hard rules from the prompt (do not violate)
    
    These are excerpted from `references/prompt.md`. The validator enforces them.
    
    ### Plotting plan (Directive 1)
    
    - `plot_type` MUST be exactly one of `"plot"` or `"diagram"`.
    - `data_source` MUST be exactly one of `"idea.md"`, `"experimental_log.md"`,
      or `"both"`.
    - `aspect_ratio` MUST be exactly one of:
      `"1:1"`, `"1:4"`, `"2:3"`, `"3:2"`, `"3:4"`, `"4:1"`, `"4:3"`, `"4:5"`,
      `"5:4"`, `"9:16"`, `"16:9"`, `"21:9"`.
    - `figure_id` MUST be a semantically meaningful snake_case identifier
      (e.g., `fig_framework_overview`, `fig_ablation_study_parameter_sensitivity`).
    - `figure_id` MUST NOT contain the word `"Figure"`.
    
    ### Intro / Related Work strategy (Directive 2)
    
    - Strictly separate Introduction (macro-level context, 10-20 papers,
      foundational + survey + impact) from Related Work (micro-level technical
      baselines, 30-50 papers, divided into 2-4 methodology clusters that
      directly compete with or precede the proposed approach).
    - For each Related Work cluster: provide `methodology_cluster`,
      `sota_investigation_mission`, `limitation_hypothesis`,
      `limitation_search_queries`, `bridge_to_our_method`.
    - **CRITICAL TIMELINE RULE**: Do not instruct searches for any papers
      published after `{cutoff_date}`. Derive `cutoff_date` from
      `conference_guidelines.md` (e.g., "ICLR 2025 → cutoff October 2024",
      "CVPR 2025 → cutoff November 2024"). If unspecified, default to one month
      before today's date.
    
    ### Section plan (Directive 3)
    
    - **Structural hierarchy**: if Subsection X.1 is created, X.2 is mandatory.
      No orphaned subsections. Omit subsections entirely if a section does not
      require division.
    - **Content specificity**: each `content_bullets` entry must reference source
      materials concretely. AVOID "Describe the model". REQUIRE "Formalize the
      Temporal-Aware Attention mechanism using Eq. 3 from idea.md."
    - **Mandatory citations**: every dataset, optimizer, metric, and
      foundational architecture/model mentioned in `idea.md` or
      `experimental_log.md` MUST have a citation hint, no matter how ubiquitous
      (e.g., AdamW, ResNet, ImageNet, CLIP, Transformer, LLaMA, GPT, LLaVA).
    - **Citation hint format**:
      - If you know the exact author and title:
        `"Author (Exact Paper Title)"`
      - Otherwise: `"research paper or technical report introducing '[Exact Model/Dataset/Metric Name]'"`
      - **Do NOT guess or hallucinate authors.**
    
    ## Output
    
    Exactly one file: `workspace/outline.json`. No prose, no code blocks, no
    markdown. The Section Writing Agent and Literature Review Agent will parse
    this JSON directly.
    
    See `references/example-output.json` for a complete worked example from the
    paper (App. F.1, pp. 43–44).
    
    ## Resources
    
    - `references/prompt.md` — verbatim Outline Agent prompt from App. F.1
    - `references/outline-schema.md` — prose explanation of the schema
    - `references/outline_schema.json` — machine-readable JSON Schema
    - `references/example-output.json` — example output from the paper
    - `references/allowed-values.md` — enumerated allowed values for each enum field
    - `scripts/validate_outline.py` — JSON Schema validator
    - `skills/shared/research_brief_template.md` — **NEW** §1 schema; append after outline.json passes validation
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related