{"slug":"plotting-agent","title":"plotting-agent","summary":"Step 2 of the PaperOrchestra pipeline (arXiv:2604.05018). Execute the visualization plan from outline.json — render plots and conceptual diagrams from experimental_log.md and idea.md, optionally refine via VLM critique loop, and produce context-aware captions. Runs in parallel wi","platform":"Claude","tags":[],"authorName":"LLM Mart","authorSlug":"llm-mart","score":0,"source":"github","price":null,"verified":false,"createdAt":"2026-09-24T15:42:14.965819Z","repo":{"url":"https://github.com/Ar9av/PaperOrchestra","stars":663,"forks":92,"license":null,"updatedAt":"2026-09-21T17:10:41Z"},"bodyHtml":"<hr>\n<h2>name: plotting-agent\ndescription: Step 2 of the PaperOrchestra pipeline (arXiv:2604.05018). Execute the visualization plan from outline.json — render plots and conceptual diagrams from experimental_log.md and idea.md, optionally refine via VLM critique loop, and produce context-aware captions. Runs in parallel with the literature-review-agent. TRIGGER when the orchestrator delegates Step 2 or when the user asks to \"generate the figures for my paper\" or \"render the plots from this experiment log\".</h2>\n<h1>Plotting Agent (Step 2)</h1>\n<p>Faithful implementation of the Plotting Agent from PaperOrchestra\n(Song et al., 2026, arXiv:2604.05018, §4 Step 2 and App. F.1 p.45).</p>\n<p><strong>Cost: ~20–30 LLM calls.</strong> The paper uses PaperBanana (Zhu et al., 2026) as\nthe default backbone with a closed-loop VLM-critique refinement. This skill\nexpresses that loop in host-agent terms: you (the host agent) generate\nmatplotlib code with your own LLM, render via your Bash/Python tool,\noptionally critique the rendered PNG with your vision model, redraw, and\nfinally caption.</p>\n<h2>Inputs</h2>\n<ul>\n<li><code>workspace/outline.json</code> — specifically the <code>plotting_plan</code> array</li>\n<li><code>workspace/inputs/idea.md</code> and <code>workspace/inputs/experimental_log.md</code> —\nthe source data</li>\n<li><code>workspace/inputs/figures/</code> — optional pre-existing figures (<code>PlotOn</code> mode)</li>\n</ul>\n<h2>Outputs</h2>\n<ul>\n<li><code>workspace/figures/&lt;figure_id&gt;.png</code> — one PNG per <code>plotting_plan</code> entry\n(300 DPI, sized to the requested aspect ratio)</li>\n<li><code>workspace/figures/captions.json</code> — <code>{figure_id: caption_text}</code> map</li>\n</ul>\n<h2>Workflow</h2>\n<h3>Per figure (executed independently per <code>figure_id</code>)</h3>\n<ol>\n<li><p><strong>Read the figure spec</strong> from <code>outline.json</code>:</p>\n<pre><code>{\n  \"figure_id\": \"fig_main_results\",\n  \"title\": \"Main Results on Dataset X\",\n  \"plot_type\": \"plot\",\n  \"data_source\": \"experimental_log.md\",\n  \"objective\": \"Visual summary (Grouped Bar Chart) demonstrating ...\",\n  \"aspect_ratio\": \"5:4\"\n}\n</code></pre>\n</li>\n<li><p><strong>Few-shot retrieval (visual planning)</strong>: pick the matching pattern from\n<code>references/chart-patterns.md</code> (for <code>plot_type==\"plot\"</code>) or\n<code>references/diagram-patterns.md</code> (for <code>plot_type==\"diagram\"</code>).</p>\n</li>\n<li><p><strong>Extract data</strong>: parse <code>idea.md</code> and/or <code>experimental_log.md</code>\n(<code>data_source</code> field tells you which) to obtain the numeric values or\nconceptual entities the figure needs. For <code>experimental_log.md</code>, the\n<code>## 2. Raw Numeric Data</code> section contains markdown tables.</p>\n</li>\n<li><p><strong>Render</strong>:</p>\n<p><strong>If <code>PAPERBANANA_PATH</code> is set</strong> — use the PaperBanana backbone\n(Zhu et al., 2026). It runs a Retriever → Planner → Stylist → Visualizer\n→ Critic loop and is especially good for <code>plot_type == \"diagram\"</code>.\nSee <code>references/paperbanana-cookbook.md</code> for setup (needs a Gemini API key).</p>\n<pre><code>python skills/plotting-agent/scripts/paperbanana_render.py \\\n    --figure-id &lt;figure_id&gt; \\\n    --caption   \"&lt;objective from figure spec&gt;\" \\\n    --content-file workspace/inputs/idea.md \\\n    --task      &lt;diagram|plot&gt; \\\n    --aspect-ratio &lt;aspect_ratio&gt; \\\n    --out       workspace/figures/&lt;figure_id&gt;.png\n</code></pre>\n<p><strong>Otherwise</strong> — write a matplotlib script and run it via your Bash tool,\nor use the bundled helper:</p>\n<pre><code>python skills/plotting-agent/scripts/render_matplotlib.py \\\n    --spec spec.json \\\n    --out workspace/figures/&lt;figure_id&gt;.png\n</code></pre>\n<p>The script must apply the academic style from <code>chart-patterns.md</code>, use the\ncorrect pixel size from <code>aspect-ratios.md</code>, save at 300 DPI, and call\n<code>plt.close()</code> after <code>savefig</code>.</p>\n</li>\n<li><p><strong>VLM critique loop (optional, only if your host has vision)</strong>:</p>\n<ul>\n<li>Reload the rendered PNG as a multimodal input to your LLM.</li>\n<li>Critique it against the figure's <code>objective</code> from the outline. Look for:\nvisual artifacts, mislabeled axes, illegible text, color clashes,\nmisleading scaling, missing legend, overlapping labels.</li>\n<li>If problems are found, regenerate the matplotlib script with corrections\nand re-render. Cap at 3 critique iterations per figure.</li>\n<li>This is the closed-loop refinement step the paper inherits from\nPaperBanana. See <code>references/plotting-pipeline.md</code> for the full loop\ndescription.</li>\n<li><strong>If your host has no vision input, skip this step entirely.</strong> The\nfigure will still render correctly, just without iterative refinement.</li>\n</ul>\n</li>\n<li><p><strong>Generate the caption</strong> using the verbatim Caption Generation prompt at\n<code>references/caption-prompt.md</code>. Inputs to the caption prompt:</p>\n<ul>\n<li><code>task_name</code> — the section the figure belongs to (e.g., \"Methodology\",\n\"Experiments\")</li>\n<li><code>raw_content</code> — the surrounding section text (or content_bullets from\nthe section_plan if the section isn't drafted yet)</li>\n<li><code>description</code> — the <code>objective</code> field from the figure spec</li>\n<li><code>figure_desc</code> — a 1-sentence description of what the rendered figure\nactually shows (from your VLM critique pass, or from the script's plan\nif no vision)</li>\n</ul>\n<p>Write the caption to <code>workspace/figures/captions.json</code> keyed by\n<code>figure_id</code>. <strong>Captions must NOT contain <code>Figure N:</code> or <code>Caption N:</code>\nprefixes</strong> — the LaTeX template handles numbering. Plain text only, no\nmarkdown.</p>\n</li>\n</ol>\n<h2>Conceptual diagrams</h2>\n<p>For <code>plot_type == \"diagram\"</code>, prefer PaperBanana when available — its\nRetriever grounds the Planner in real published paper diagrams.  If\n<code>PAPERBANANA_PATH</code> is unset, follow <code>references/diagram-patterns.md</code>.\nPatterns include block diagrams, system overviews, flowcharts, and\nalgorithm-as-graph. The bundled helper:</p>\n<pre><code>python skills/plotting-agent/scripts/render_diagram.py \\\n    --spec diagram_spec.json \\\n    --out workspace/figures/&lt;figure_id&gt;.png\n</code></pre>\n<p>handles the simple cases (boxes-and-arrows). For complex Fig-1-style\noverview diagrams, write matplotlib patches code yourself.</p>\n<h2>Hard rules</h2>\n<ul>\n<li><strong>300 DPI</strong> for every figure. Lower DPI gets rejected at the LaTeX compile\nstep on conference templates.</li>\n<li><strong>Aspect ratio is exact</strong>. The figure spec's <code>aspect_ratio</code> is one of 12\nenumerated strings. Use the pixel targets in <code>references/aspect-ratios.md</code>.</li>\n<li><strong>Hide top and right spines</strong> for plots. (Diagrams: no spines at all.)</li>\n<li><strong>Muted academic colors</strong> only. The palette is in <code>chart-patterns.md</code>.\nNever use matplotlib defaults (too saturated for print).</li>\n<li><strong>No 3D, no pie charts, no decorative visuals.</strong> The paper's evaluators\npenalize these.</li>\n<li><strong>Every figure MUST have a caption</strong> in <code>captions.json</code>. The Section\nWriting Agent will fail-stop if a caption is missing for any figure\nreferenced from the outline.</li>\n<li><strong>No <code>Figure N:</code> prefix</strong> in captions — LaTeX adds it.</li>\n<li><strong>Never describe data you didn't plot.</strong> The Plotting Agent must not\nhallucinate axes, baselines, or trends. Source-of-truth is\n<code>experimental_log.md</code> or <code>idea.md</code>.</li>\n</ul>\n<h2>Verification gate (run before handing off to Step 3/4)</h2>\n<p>The hard rules above are stated everywhere and enforced nowhere. This gate\nmakes the mechanical half checkable:</p>\n<pre><code>python skills/plotting-agent/scripts/figure_lint.py \\\n    --figures workspace/figures \\\n    --captions workspace/figures/captions.json\n</code></pre>\n<p>ERRORs: a rendered figure with no caption, a caption with no file, an empty\ncaption, a raster too small to print. WARNs: resolution under ~300 DPI at\nsingle-column width, aspect ratios past 4:1, captions that number themselves\n(<code>Figure 3: ...</code>), captions under eight words, and a figure set with no\narchitecture/pipeline/overview figure in it.</p>\n<p>PNG geometry is read from the IHDR and pHYs chunks directly — no imaging\nlibrary, consistent with the repo's deterministic-helpers-only rule.</p>\n<p>After Step 4 has produced <code>paper.tex</code>, re-run with <code>--paper</code> to confirm the\ndraft uses every figure Step 2 rendered and references no file that does not\nexist:</p>\n<pre><code>python skills/plotting-agent/scripts/figure_lint.py \\\n    --figures workspace/figures \\\n    --paper   workspace/drafts/paper.tex\n</code></pre>\n<p>Fix ERRORs before continuing. A missing caption is the one failure that\npropagates silently: Step 4 splices the figure with whatever caption it\ninvents, and Step 5 has no way to know the caption was never grounded.</p>\n<h2>Pre-existing figures (PlotOn mode)</h2>\n<p>If <code>workspace/inputs/figures/</code> is non-empty, check whether any pre-existing\nfile matches a <code>figure_id</code> in the outline (by filename prefix). If so,\n<strong>copy</strong> it into <code>workspace/figures/</code> as-is and <strong>still generate a caption</strong>\nusing the caption prompt. Only generate from scratch the figure_ids that\nhave no pre-existing counterpart.</p>\n<h2>Resources</h2>\n<ul>\n<li><code>references/caption-prompt.md</code> — verbatim Caption Generation prompt from App. F.1</li>\n<li><code>references/plotting-pipeline.md</code> — the full few-shot → render → critique → caption loop</li>\n<li><code>references/chart-patterns.md</code> — matplotlib style + chart type recipes</li>\n<li><code>references/diagram-patterns.md</code> — conceptual diagram recipes</li>\n<li><code>references/aspect-ratios.md</code> — pixel targets for each of the 12 allowed ratios at 300 DPI</li>\n<li><code>references/paperbanana-cookbook.md</code> — <strong>NEW</strong> PaperBanana setup, usage, cost notes, attribution</li>\n<li><code>scripts/render_matplotlib.py</code> — render a JSON plot spec → PNG (matplotlib fallback)</li>\n<li><code>scripts/render_diagram.py</code> — render a JSON diagram spec → PNG (matplotlib fallback)</li>\n<li><code>scripts/paperbanana_render.py</code> — <strong>NEW</strong> PaperBanana backbone wrapper (reads <code>PAPERBANANA_PATH</code> from env)</li>\n<li><code>scripts/figure_lint.py</code> — <strong>NEW</strong> resolution / aspect / caption-coverage gate; <code>--paper</code> cross-checks <code>\\includegraphics</code></li>\n</ul>\n","files":[{"path":"references/aspect-ratios.md","sizeBytes":2094,"isText":true},{"path":"references/caption-prompt.md","sizeBytes":2735,"isText":true},{"path":"references/chart-patterns.md","sizeBytes":6732,"isText":true},{"path":"references/diagram-patterns.md","sizeBytes":3943,"isText":true},{"path":"references/paperbanana-cookbook.md","sizeBytes":6122,"isText":true},{"path":"references/plotting-pipeline.md","sizeBytes":5961,"isText":true},{"path":"scripts/figure_lint.py","sizeBytes":8273,"isText":true},{"path":"scripts/paperbanana_render.py","sizeBytes":12987,"isText":true},{"path":"scripts/render_diagram.py","sizeBytes":4225,"isText":true},{"path":"scripts/render_matplotlib.py","sizeBytes":7253,"isText":true},{"path":"scripts/test_figure_lint.py","sizeBytes":6637,"isText":true},{"path":"SKILL.md","sizeBytes":9395,"isText":true}],"reviewScore":null,"reviewSummary":null,"trust":{"provenance":"trusted-source-unreviewed","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow.","bodySource":null},"bodyLocked":false,"purchaseUrl":null,"sourceUrl":null,"report":{"provenance":"trusted-source-unreviewed","screen":{"ran":true,"outcome":"clean","suspicious":0,"notes":0,"hiddenCharacters":false},"virusScan":{"engine":"clamav","status":"clean","scannedAt":"2026-09-24T15:42:28.875044Z","sha256":"CD9DD40B2139DE845B7CEEE5D36A89DFFA146E6B82F222C0DDE22A38626C5CB0","sizeBytes":30160},"review":null,"source":{"repositoryUrl":"https://github.com/Ar9av/PaperOrchestra","path":"skills/plotting-agent","license":null,"commit":"36c3cc4b10370b1f905adcd4e5601e8b624c2dc3","subtreeSha":"E5BE6786900D55B2626375BA54413EBE1CBED03856F514D58403CFFEB9AF05EB","lastSyncedAt":"2026-09-24T15:42:09.852277Z"},"reviewedAt":"2026-09-24T15:42:58.502411Z","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow."},"install":[{"target":"skills-cli","command":"npx skills add https://github.com/Ar9av/PaperOrchestra/tree/main/skills/plotting-agent"},{"target":"claude-code","command":"claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install ar9av-paperorchestra@llmmart"},{"target":"git","command":"git clone https://github.com/Ar9av/PaperOrchestra.git"}]}