# Figure Integration

Conventions for including figures in the LaTeX paper, per the Section
Writing Agent prompt (App. F.1 p.48, item 5 "Figures and Visual Fidelity").

## Where the figures live

After Step 2 (Plotting Agent), figures are at:

```
workspace/figures/
├── fig_framework_overview.png
├── fig_main_results.png
├── fig_ablation_temperature.png
└── captions.json
```

The Section Writing Agent must reference them with the exact filenames,
including the `.png` extension:

```latex
\begin{figure}[t]
\centering
\includegraphics[width=0.95\linewidth]{figures/fig_framework_overview.png}
\caption{Overview of the proposed pipeline. Raw video frames flow into the
frozen SAM encoder; aligned audio cues are projected through the temporal
modality fusion layer before being injected into the mask decoder.}
\label{fig:framework_overview}
\end{figure}
```

## Single-column vs full-width

The prompt is explicit: in 2-column conference templates, prefer
`\begin{figure}` (single-column) unless the figure is very wide. Use
`\begin{figure*}` only for cross-column figures.

| Figure aspect ratio | Recommended environment |
|---|---|
| `1:1`, `2:3`, `3:2`, `3:4`, `4:3`, `4:5`, `5:4` | `figure` (single column) |
| `16:9`, `21:9`, `4:1` | `figure*` (cross column) |
| `9:16`, `1:4` | `figure` (very tall, hangs over multiple text lines) |

## Caption placement

For figures, `\caption` goes **AFTER** `\includegraphics`:

```latex
\begin{figure}[t]
\centering
\includegraphics[width=0.95\linewidth]{figures/fig_main_results.png}
\caption{Comparison of three temporal-attention variants on the Ref-AVS
Seen split. Bars show Jaccard index; error bars are 95% confidence
intervals over five seeds.}
\label{fig:main_results}
\end{figure}
```

(For tables, `\caption` goes BEFORE `\begin{tabular}`. Different
conventions; both correct in their respective contexts.)

## Caption text

- Pull from `workspace/figures/captions.json` keyed by `figure_id`.
- Do **not** include `Figure N:` in the caption text. LaTeX adds the prefix
  via `\caption`.
- Plain text only. No markdown bold/italic.
- 1-3 sentences. State what the figure shows AND why (the takeaway).

## Reference in prose

Every figure must be referenced in the prose:

```latex
As shown in Figure~\ref{fig:main_results}, our method outperforms both
baselines across all five splits.
```

Use `~` (non-breaking space) before `\ref{...}`. Use `Figure` capitalized
when starting a sentence; lowercase `figure` mid-sentence.

## DO NOT merge figures

The prompt forbids combining multiple figures into one display. Each
`figure_id` from the outline corresponds to exactly one `\begin{figure}`
environment.

## All figures before Conclusion

Per the prompt: "all figures must appear before the Conclusion section,
unless they are placed in an Appendix." If you have a figure that
contextually belongs in the Appendix, move it there explicitly — do not
leave it floating after the Conclusion in the main body.

## Multi-panel figures

If a figure has multiple panels (a, b, c) — e.g., a 1×3 grid showing
ablations across three settings — the panels are part of the **same** PNG
file (rendered by Step 2). Use a single `\includegraphics` and reference
sub-panels in the caption text:

```latex
\caption{Ablation study. (a) Effect of temperature. (b) Effect of dropout.
(c) Effect of layer count. Bars show validation accuracy on the Seen split.}
```

If you really need separate sub-figure environments (`subcaption` package),
that's allowed but adds complexity — prefer single-PNG multi-panel from
Step 2.

# LaTeX Table Patterns

Conventions for building LaTeX tables from `experimental_log.md` raw numeric
data, per the Section Writing Agent prompt requirements (App. F.1 p.47, item
2 "Data & Tables").

## Required: booktabs

Always use the `booktabs` package. The preamble in a typical conference
template already includes it; if not, add:

```latex
\usepackage{booktabs}
```

## Three rules only

Booktabs uses **only** three horizontal rules: `\toprule`, `\midrule`,
`\bottomrule`. No `\hline`. No vertical bars.

```latex
\begin{table}[t]
\centering
\caption{Comparison of methods on Dataset X.}
\label{tab:main_results}
\begin{tabular}{lccc}
\toprule
Method      & Accuracy & F1 & Latency (ms) \\
\midrule
Baseline    & 78.2     & 0.79 & 12.3 \\
\textbf{Ours} & \textbf{85.4} & \textbf{0.87} & \textbf{8.1} \\
\bottomrule
\end{tabular}
\end{table}
```

## From experimental_log markdown table → LaTeX

`experimental_log.md` contains tables in plain markdown:

```markdown
## 2. Raw Numeric Data

### Table 1: Performance comparison on Dataset X

| Method   | Accuracy | F1   | Latency (ms) |
|----------|----------|------|--------------|
| Baseline | 78.2     | 0.79 | 12.3         |
| Ours-S   | 82.1     | 0.83 | 9.4          |
| Ours-L   | 85.4     | 0.87 | 8.1          |
```

The `extract_metrics.py` helper parses these into JSON:

```json
{
  "tables": [
    {
      "label": "Performance comparison on Dataset X",
      "headers": ["Method", "Accuracy", "F1", "Latency (ms)"],
      "rows": [
        ["Baseline", "78.2", "0.79", "12.3"],
        ["Ours-S", "82.1", "0.83", "9.4"],
        ["Ours-L", "85.4", "0.87", "8.1"]
      ]
    }
  ]
}
```

The Section Writing Agent then converts each entry to a `table` environment
verbatim. Important rules from the prompt:

- **Do not hallucinate numbers.** Copy the exact values from
  `extract_metrics.py`'s output.
- **Bold the best result** in each column (the convention for top-tier ML
  papers).
- **Use `\multicolumn{N}{c}{...}` for grouped headers** when the table has
  metric families (e.g., "Seen (J%)", "Seen F", "Unseen (J%)", "Unseen F").
- **Right-align numeric columns** with `r`, left-align text columns with `l`.
  Use `c` only for narrow centered identifiers.
- **Use `\textbf{...}` for bold**, never `**...**` (markdown).

## Readability rules

Booktabs gets the rules right; these get the table *read* right. A reviewer
spends seconds on a results table and forms an opinion from it.

- **Label metric direction in the header** — `Accuracy $\uparrow$`,
  `LPIPS $\downarrow$`, `Latency (ms) $\downarrow$`. Never make a reader infer
  which way is better. `table_lint.py` warns on any numeric column whose
  header carries no arrow.
- **Put units in the header**, not in the cells: `Latency (ms)`, not
  `12.3 ms` repeated down the column.
- **Keep decimal precision constant within a column.** `85.4` and `8.14` in
  one column reads as two different measurements. Pick the precision the
  weakest measurement justifies and hold it.
- **One table, one message.** Unrelated results in one table means neither
  lands. Split rather than widen.
- **Group multi-dataset results with `\multicolumn` + `\cmidrule`**, never
  with vertical separators:

```latex
\toprule
& \multicolumn{2}{c}{Seen} & \multicolumn{2}{c}{Unseen} \\
\cmidrule(lr){2-3} \cmidrule(lr){4-5}
Method & J $\uparrow$ & F $\uparrow$ & J $\uparrow$ & F $\uparrow$ \\
\midrule
```

- **Highlight sparingly.** Bold the best result; if a second emphasis is
  genuinely needed, underline the runner-up. Colored cells beyond one or two
  rows stop being emphasis.
- **Captions carry the setting, protocol, and notation** — not discussion. A
  six-word caption is a label, not a caption.

## Checking a draft

```bash
python skills/section-writing-agent/scripts/table_lint.py workspace/drafts/paper.tex
```

ERRORs are booktabs/LaTeX rule violations (vertical rules, `\hline`, caption
below the tabular, missing rules); WARNs are the readability conventions
above. Exit 1 on any ERROR, or on any WARN with `--strict`.

## Wide tables (2-column conference templates)

For tables that don't fit single-column width, use `table*` and `tabular*`
or `tabularx`:

```latex
\begin{table*}[t]
\centering
\caption{Ablation across all 6 components on 4 splits.}
\label{tab:ablation}
\begin{tabular}{lcccccc}
\toprule
Variant       & Seen J & Seen F & Unseen J & Unseen F & Mix J & Mix F \\
\midrule
Full          & 43.43  & 0.568  & 54.58    & 0.664    & 49.01 & 0.616 \\
- TB          & 33.05  & 0.507  & 50.48    & 0.657    & 41.77 & 0.582 \\
- TMFL        & 40.35  & 0.579  & 45.54    & 0.627    & 42.95 & 0.603 \\
\bottomrule
\end{tabular}
\end{table*}
```

The closing `\end{table*}` must match the opening `\begin{table*}`. The
`latex_sanity.py` script catches mismatches.

## Caption placement

```latex
\begin{table}[t]
\centering
\caption{Caption text here.}        % BEFORE the tabular for tables
\label{tab:my_label}
\begin{tabular}{...}
...
\end{tabular}
\end{table}
```

(For figures, `\caption` goes AFTER `\includegraphics`, not before. See
`figure-integration.md`.)

## Common pitfalls

| Issue | Fix |
|---|---|
| `\hline` everywhere | Replace with `\toprule` (top), `\midrule` (between header and body), `\bottomrule` (bottom). |
| Column too wide, runs off page | Switch to `table*` + `tabular*`. |
| Vertical bars | Remove. Booktabs forbids vertical rules. |
| Misaligned decimals | Use `S[table-format=2.2]` from `siunitx` if available, else right-align with `r`. |
| Table after Conclusion | Move it before. The prompt mandates this. |
| Hallucinated values | Cross-check against `extract_metrics.py` output. |

## Figures floating into or after the References section

**This is the most common final-layout bug.** When many figures appear in the
Experiments section and the bibliography is near the end, LaTeX cannot place all
floats in the text body and defers them past `\bibliography`.

**Fix**: always emit `\clearpage` immediately before `\bibliographystyle{...}`:

```latex
% Flush all pending floats before the bibliography
\clearpage

\bibliographystyle{plainnat}
\bibliography{refs}
```

The `\clearpage` forces LaTeX to output every deferred float on their own pages
before starting the reference list. Without it, figures that could not fit in the
Experiments section will appear between the References heading and the reference
entries, or after the last reference.

## Cross-referencing without cleveref

When the conference template uses `\usepackage[capitalize]{cleveref}`, the
Section Writing Agent should produce `\cref{fig:X}` and `\cref{tab:Y}`.
However, if `cleveref` is stripped (e.g., due to a minimal TeX installation),
bare `\ref{}` produces only the number with no "Figure" or "Table" prefix,
which reads as isolated numbers in the prose.

**Pattern to use when `cleveref` is absent**:

```latex
Figure~\ref{fig:overview}   % tilde prevents line break before number
Table~\ref{tab:main-results}
```

Never write just `\ref{fig:overview}` without a prefix; readers will see "...see 3."

The host agent must check whether `cleveref.sty` is available in the TeX
installation before choosing `\cref{}` vs `Figure~\ref{}`. A safe default is
to always use the `Figure~\ref{}` form; it degrades gracefully and works
everywhere.

# Section Writing Agent — verbatim prompt

**Source: arXiv:2604.05018, Appendix F.1, pages 47–49 (verbatim).**

Use this as your system message for the **single multimodal LLM call** that
drafts the remaining sections of the paper. The Anti-Leakage Prompt
(`../paper-orchestra/references/anti-leakage-prompt.md`) MUST be prepended.

---

```
Role: Senior AI Researcher.

Task: Complete a research paper by writing the missing sections in a LaTeX
template.

You will be given a template.tex file where some sections (e.g.,
Introduction, Related Work) are already written, and others are empty or
missing. Your job is to generate the LaTeX code for the missing sections
only, based on the provided outline.json, and merge them into the final
document.

Inputs

  - outline.json: Your MASTER PLAN. Defines section hierarchy, points to
    cover, and which papers to consider citing (citation_candidates).
  - idea.md: Technical details of the methodology.
  - experimental_log.md: Raw data for tables and qualitative analysis for
    text.
  - citation_map.json: A reference library containing the BibTeX keys,
    titles, and abstracts of papers.
  - conference_guidelines.md: Formatting rules.
  - figures_list: Available figure files.

Critical Instructions

1. Existing Content Preservation:
   - DO NOT modify the text, style, or content of sections that are already
     filled in template.tex.
   - Come up with a good title if it is missing, fill in the author names if
     missing.
   - Keep the preamble (packages) exactly as is.

2. Data & Tables:
   - You are responsible for creating LaTeX tables.
   - Extract numerical data directly from experimental_log.md.
   - Use the booktabs package format (\toprule, \midrule, \bottomrule).
   - Do not hallucinate numbers. Use the exact values provided in the log.
   - Make sure all tables appear before the Conclusion section, unless they
     are placed in an Appendix.

3. Citations:
   - The outline.json provides a list of citation_candidates for specific
     subsections.
   - You MUST use the exact keys found in citation_map.json (e.g.,
     \cite{Hu2021LoraLowrank}).
   - Content Enrichment: Read the abstract provided in citation_map.json
     for the papers you are citing. Use this context to write accurate,
     specific sentences about those works.

4. Writing Content:
   - Write the missing sections following the outline.json structure.
   - Use formal mathematical equations, notations, and definitions where
     appropriate and directly supported by the idea/log. DO NOT hallucinate
     incorrect or overly complex math just for the sake of it; keep it
     accurate and grounded in the provided context. Avoid overly colloquial
     summaries.
   - Always provide detailed ablation studies and qualitative analysis of
     the experimental results: what worked, what does not, and why.
   - Nice to have: discuss the limitations and future work at the end.
   - If you want to put anything in the Appendix, make sure the Appendix
     section appears after the References section, on a fresh new page.

5. Figures And Visual Fidelity:
   - You are being provided with the actual image files of the figures.
     You MUST describe them faithfully and accurately. DO NOT hallucinate
     interpretations that contradict the visual evidence in the plots.
   - Make sure to use ALL of the figures provided in figures_list. Note:
     figures are stored in the figures/ subdirectory. IMPORTANT: use the
     exact filenames including their extensions (e.g., .png) in your
     \includegraphics commands.
   - DO NOT merge or group multiple figures into one for display.
   - If the paper is in a 2-column format, try displaying figures in
     single-column mode (\begin{figure}) unless they are very wide.
   - Ensure that all figures are correctly referenced in the text.
   - Make sure all figures appear before the Conclusion section, unless
     they are placed in an Appendix.
   - You can refine the captions if necessary.
   - Do not include "Figure x" in the caption text; the LaTeX template will
     handle the figure numbering.

6. Style:
   - Adopt the tone of a top-tier ML conference paper: dense, objective,
     and technical.
   - Ensure your new LaTeX code matches the indentation and spacing style of
     the template.tex. Do not change the given style.

Output Format

  - Return the full code for the completed template.tex.
  - The sections that were previously empty should now be filled.
  - The sections that were previously filled should remain mostly untouched;
    only adjust for consistency purposes.
  - Wrap the code with ```latex content ```.

Important Note

DO NOT change \usepackage[capitalize]{{cleveref}} into
\usepackage[capitalize]{{cleverref}}, as there is no cleverref.sty.
Ensure the LaTeX code compiles without errors, e.g., all the begin and end
statements match correctly (e.g., \begin{{figure*}} must be closed with
\end{{figure*}}, not \end{{figure}}).
```

---

## Multimodal call — image inputs

This call should pass the actual figure PNGs as image content blocks
alongside the text inputs above. The model uses them to (a) verify it isn't
describing a chart that doesn't exist, (b) write factually-grounded captions,
(c) accurately interpret what each plot shows in the prose. If your host LLM
lacks vision, document the degradation in your run report and proceed
text-only.

#!/usr/bin/env python3
"""
extract_metrics.py — Parse markdown tables out of experimental_log.md's
"## 2. Raw Numeric Data" section into structured JSON.

The Section Writing Agent uses this to construct LaTeX booktabs tables
without re-deriving numeric values from raw markdown text. Per the App. F.1
prompt, "do not hallucinate numbers; use the exact values provided in the
log" — this script makes that mechanical.

Output JSON shape:
    {
      "tables": [
        {
          "label": "Performance comparison on Dataset X",
          "headers": ["Method", "Accuracy", "F1", "Latency (ms)"],
          "rows": [
            ["Baseline", "78.2", "0.79", "12.3"],
            ...
          ]
        },
        ...
      ]
    }

Usage:
    python extract_metrics.py --log experimental_log.md --out metrics.json
"""
import argparse
import json
import re
import sys


def find_raw_data_section(text: str) -> str:
    """Return the slice of text from '## 2. Raw Numeric Data' to the next H2."""
    m = re.search(r"^##\s+2\.?\s*Raw Numeric Data\s*$", text, re.M)
    if not m:
        return ""
    start = m.end()
    next_h2 = re.search(r"^##\s+", text[start:], re.M)
    end = start + next_h2.start() if next_h2 else len(text)
    return text[start:end]


def parse_markdown_tables(section: str) -> list[dict]:
    """Walk the section, extracting markdown tables and their preceding labels."""
    lines = section.split("\n")
    tables: list[dict] = []
    current_label: str | None = None
    i = 0
    while i < len(lines):
        line = lines[i].strip()

        # Track table labels: ### Table N: Foo  /  ### Table: Foo  /  **Table 1: Foo**
        m = re.match(r"^#+\s*Table[^:]*:\s*(.+?)\s*$", line)
        if m:
            current_label = m.group(1).strip()
            i += 1
            continue
        m = re.match(r"^\*\*Table[^:]*:\s*(.+?)\*\*\s*$", line)
        if m:
            current_label = m.group(1).strip()
            i += 1
            continue

        # Detect table start: a header row followed by a separator row.
        if "|" in line and i + 1 < len(lines):
            sep = lines[i + 1].strip()
            if re.fullmatch(r"\|?\s*[:\-]+\s*(\|\s*[:\-]+\s*)+\|?", sep):
                headers = [c.strip() for c in line.strip("|").split("|")]
                rows: list[list[str]] = []
                j = i + 2
                while j < len(lines) and "|" in lines[j].strip():
                    cells = [c.strip() for c in lines[j].strip().strip("|").split("|")]
                    if len(cells) >= 2:
                        rows.append(cells)
                    j += 1
                tables.append({
                    "label": current_label or f"Table {len(tables) + 1}",
                    "headers": headers,
                    "rows": rows,
                })
                current_label = None
                i = j
                continue

        i += 1

    return tables


def main() -> int:
    p = argparse.ArgumentParser(description=__doc__)
    p.add_argument("--log", required=True, help="experimental_log.md path")
    p.add_argument("--out", required=True, help="metrics.json output path")
    args = p.parse_args()

    text = open(args.log).read()
    section = find_raw_data_section(text)
    if not section:
        print("WARN: no '## 2. Raw Numeric Data' section found", file=sys.stderr)
        with open(args.out, "w") as f:
            json.dump({"tables": []}, f, indent=2)
        return 0

    tables = parse_markdown_tables(section)
    out = {"tables": tables}
    with open(args.out, "w") as f:
        json.dump(out, f, indent=2, ensure_ascii=False)

    print(f"OK: extracted {len(tables)} table(s) → {args.out}")
    for t in tables:
        print(f"  - {t['label']}: {len(t['headers'])} cols × {len(t['rows'])} rows")
    return 0


if __name__ == "__main__":
    sys.exit(main())

#!/usr/bin/env python3
"""
latex_sanity.py — Deterministic structural checks on a generated LaTeX file.

Catches the most common ways the Section Writing Agent's output can fail to
compile, before invoking latexmk:

  1. Unmatched braces (counts \\{ and \\} but ignores escaped ones)
  2. Mismatched \\begin{X} / \\end{X} environments
  3. Unescaped special characters (& % _ outside math/verbatim contexts)
     — heuristic only; common false positives in tabular cells
  4. Duplicate \\label{...}
  5. Missing \\documentclass

Exit codes:
    0  no errors found
    1  one or more errors

Usage:
    python latex_sanity.py path/to/paper.tex
"""
import re
import sys


def check_braces(text: str) -> list[str]:
    # Strip escaped braces and comments
    stripped = re.sub(r"%[^\n]*", "", text)
    stripped = stripped.replace("\\{", "").replace("\\}", "")
    n_open = stripped.count("{")
    n_close = stripped.count("}")
    if n_open != n_close:
        return [f"unmatched braces: {{ × {n_open}, }} × {n_close} (delta {n_open - n_close})"]
    return []


def check_environments(text: str) -> list[str]:
    starred = lambda s: s.replace("*", r"\*")  # noqa: E731
    starts = re.findall(r"\\begin\{([^}]+)\}", text)
    ends = re.findall(r"\\end\{([^}]+)\}", text)
    errors: list[str] = []

    stack: list[str] = []
    pos = 0
    # Walk in order, push starts, pop on ends
    for m in re.finditer(r"\\(begin|end)\{([^}]+)\}", text):
        kind, env = m.group(1), m.group(2)
        if kind == "begin":
            stack.append(env)
        else:
            if not stack:
                errors.append(f"\\end{{{env}}} with no matching \\begin")
                continue
            top = stack.pop()
            if top != env:
                errors.append(f"\\begin{{{top}}} closed by \\end{{{env}}}")

    if stack:
        errors.append(f"unclosed environments: {stack}")
    return errors


def check_documentclass(text: str) -> list[str]:
    if not re.search(r"\\documentclass", text):
        return ["missing \\documentclass — not a complete LaTeX document"]
    return []


def check_duplicate_labels(text: str) -> list[str]:
    labels = re.findall(r"\\label\{([^}]+)\}", text)
    seen: dict[str, int] = {}
    for l in labels:
        seen[l] = seen.get(l, 0) + 1
    dupes = [l for l, n in seen.items() if n > 1]
    if dupes:
        return [f"duplicate labels: {dupes}"]
    return []


def check_unescaped_specials(text: str) -> list[str]:
    """Heuristic: look for & % _ that appear OUTSIDE tabular/math/verbatim
    environments. False positives are common; we only emit WARNINGS, not errors."""
    warnings: list[str] = []
    # Strip math, tabular, verbatim, comments
    s = re.sub(r"\\begin\{tabular\*?\}.*?\\end\{tabular\*?\}", "", text, flags=re.S)
    s = re.sub(r"\\begin\{(equation|align|array|matrix|verbatim|lstlisting)\*?\}.*?\\end\{\1\*?\}", "", s, flags=re.S)
    s = re.sub(r"\$[^$]*\$", "", s)
    s = re.sub(r"%[^\n]*", "", s)
    s = re.sub(r"\\[%&_#$]", "", s)  # remove already-escaped
    bad = re.findall(r"[%&_]", s)
    if bad:
        warnings.append(f"WARN: {len(bad)} potentially unescaped %, &, or _ outside math/tabular")
    return warnings


def main() -> int:
    if len(sys.argv) != 2:
        print(__doc__, file=sys.stderr)
        return 2
    path = sys.argv[1]
    text = open(path).read()

    errors: list[str] = []
    errors += check_documentclass(text)
    errors += check_braces(text)
    errors += check_environments(text)
    errors += check_duplicate_labels(text)
    warnings = check_unescaped_specials(text)

    for w in warnings:
        print(w)

    if errors:
        print(f"\nFAIL: {len(errors)} latex sanity error(s) in {path}", file=sys.stderr)
        for e in errors:
            print(f"  - {e}", file=sys.stderr)
        return 1

    print(f"OK: {path} passes structural sanity checks")
    return 0


if __name__ == "__main__":
    sys.exit(main())

#!/usr/bin/env python3
"""
orphan_cite_gate.py — Verify every \\cite{KEY} in a LaTeX file resolves to
an entry in refs.bib.

The Section Writing Agent prompt mandates "use ONLY the keys found in
citation_map.json". This script enforces it deterministically.

Exit codes:
    0  every cite key resolves
    1  one or more orphan cite keys

Usage:
    python orphan_cite_gate.py paper.tex refs.bib
"""
import re
import sys

CITE_RE = re.compile(
    r"\\(?:cite|citep|citet|citeauthor|citeyear|autocite|parencite|textcite)"
    r"(?:\[[^\]]*\])?"
    r"\{([^}]+)\}"
)
BIB_KEY_RE = re.compile(r"^@\w+\{\s*([^,\s]+)", re.M)


def main() -> int:
    if len(sys.argv) != 3:
        print(__doc__, file=sys.stderr)
        return 2

    tex_path, bib_path = sys.argv[1], sys.argv[2]
    tex = open(tex_path).read()
    bib = open(bib_path).read()

    bib_keys = set(BIB_KEY_RE.findall(bib))
    if not bib_keys:
        print(f"ERROR: no @entry keys found in {bib_path}", file=sys.stderr)
        return 1

    cite_keys: set[str] = set()
    for m in CITE_RE.finditer(tex):
        for k in m.group(1).split(","):
            k = k.strip()
            if k:
                cite_keys.add(k)

    orphans = sorted(cite_keys - bib_keys)
    unused = sorted(bib_keys - cite_keys)

    print(f"refs.bib has {len(bib_keys)} entries; {tex_path} cites {len(cite_keys)} unique keys")

    if orphans:
        print(f"\nFAIL: {len(orphans)} orphan \\cite key(s) (not in refs.bib):", file=sys.stderr)
        for k in orphans:
            print(f"  - {k}", file=sys.stderr)
        return 1

    if unused:
        # Just informational. The literature-review-agent's citation_coverage.py
        # is the gate that enforces ≥90% integration.
        print(f"INFO: {len(unused)} bib entries not yet cited (informational)")

    print("OK: no orphan cite keys")
    return 0


if __name__ == "__main__":
    sys.exit(main())

#!/usr/bin/env python3
"""
table_lint.py — Check LaTeX tables against the conventions in
references/latex-table-patterns.md.

Tables are the first thing a reviewer looks at and the last thing a generated
draft gets right. The failures are mechanical and therefore checkable: vertical
rules, \\hline stacks, captions below the tabular, metric columns with no
direction marker, mixed decimal precision down a column.

Severity:
  ERROR — violates a booktabs/LaTeX rule the prompt states outright
  WARN  — violates a readability convention a reviewer will notice

Exit codes:
    0 — no ERRORs (and no WARNs, if --strict)
    1 — at least one ERROR, or any WARN under --strict
    2 — input file missing or unreadable

Usage:
    python table_lint.py workspace/drafts/paper.tex [--json report.json] [--strict]
"""
import argparse
import json
import os
import re
import sys

METRIC_ARROWS = ("↑", "↓", r"\uparrow", r"\downarrow", r"\textuparrow", r"\textdownarrow")

RULE_MACROS = ("\\toprule", "\\midrule", "\\bottomrule", "\\cmidrule", "\\hline", "\\cline")


def find_tables(tex: str) -> list[dict]:
    """Return one record per table/table* environment."""
    out = []
    for m in re.finditer(r"\\begin\{(table\*?)\}(.*?)\\end\{\1\}", tex, re.DOTALL):
        out.append({
            "env": m.group(1),
            "body": m.group(2),
            "start": m.start(),
            "line": tex[:m.start()].count("\n") + 1,
        })
    return out


def strip_cell(cell: str) -> str:
    cell = re.sub(r"\\(?:textbf|textit|emph|texttt|underline|mathbf)\{([^{}]*)\}", r"\1", cell)
    cell = re.sub(r"\\cellcolor\{[^}]*\}|\\rowcolor\{[^}]*\}", " ", cell)
    cell = re.sub(r"\\multicolumn\{\d+\}\{[^}]*\}\{([^{}]*)\}", r"\1", cell)
    cell = cell.replace("$", "").replace("~", " ").replace("\\%", "%").replace("\\&", "&")
    cell = re.sub(r"\\[a-zA-Z]+\*?", " ", cell)
    return cell.strip(" {}")


NUMERIC = re.compile(r"^[+-]?\d+(?:\.\d+)?%?$")


def is_numeric(cell: str) -> bool:
    return bool(NUMERIC.match(strip_cell(cell).replace(",", "")))


def decimals(cell: str) -> int:
    body = strip_cell(cell).replace(",", "").rstrip("%")
    return len(body.split(".")[1]) if "." in body else 0


def split_rows(tabular_body: str) -> list[list[str]]:
    rows = []
    for raw in re.split(r"\\\\", tabular_body):
        line = raw
        for macro in RULE_MACROS:
            line = re.sub(re.escape(macro) + r"(?:\([^)]*\))?(?:\{[^}]*\})?", " ", line)
        if not line.strip():
            continue
        rows.append([c for c in re.split(r"(?<!\\)&", line)])
    return rows


def lint_table(t: dict, after_conclusion: bool) -> list[dict]:
    findings: list[dict] = []
    body = t["body"]

    def add(sev: str, code: str, msg: str) -> None:
        findings.append({"severity": sev, "code": code, "line": t["line"],
                         "env": t["env"], "message": msg})

    tab = re.search(r"\\begin\{(tabular\*?|tabularx|tabulary)\}(?:\{[^}]*\})?\{([^}]*)\}(.*?)"
                    r"\\end\{\1\}", body, re.DOTALL)
    if not tab:
        add("ERROR", "no-tabular", "table environment contains no tabular")
        return findings

    colspec, rows_src = tab.group(2), tab.group(3)

    if "|" in colspec:
        add("ERROR", "vertical-rule",
            f"column spec {{{colspec}}} uses vertical rules; booktabs forbids them")
    if "\\hline" in body or "\\cline" in body:
        add("ERROR", "hline", "uses \\hline/\\cline; use \\toprule/\\midrule/\\bottomrule")
    if "\\toprule" not in body or "\\bottomrule" not in body:
        add("ERROR", "missing-rules", "missing \\toprule and/or \\bottomrule")

    cap = body.find("\\caption")
    tab_start = body.find("\\begin{" + tab.group(1))
    if cap == -1:
        add("ERROR", "no-caption", "table has no \\caption")
    elif cap > tab_start:
        add("ERROR", "caption-below",
            "\\caption appears after the tabular; table captions go above")
    else:
        caption_text = strip_cell(body[cap:body.find("\n\\label") if "\\label" in body else cap + 400])
        words = len(re.sub(r"^caption", "", caption_text, flags=re.I).split())
        if words < 6:
            add("WARN", "thin-caption",
                f"caption is {words} words; state the setting, protocol, and notation")

    if "\\label" not in body:
        add("WARN", "no-label", "table has no \\label, so it cannot be cross-referenced")

    if after_conclusion:
        add("WARN", "late-table",
            "table appears after the Conclusion; the prompt requires tables before it")

    # ── column-level checks ────────────────────────────────────────────────
    rows = split_rows(rows_src)
    if len(rows) < 2:
        return findings
    header, data = rows[0], rows[1:]
    width = len(header)

    for col in range(width):
        cells = [r[col] for r in data if len(r) == width]
        numeric = [c for c in cells if is_numeric(c)]
        if not cells or len(numeric) < max(2, int(0.8 * len(cells))):
            continue  # not a numeric metric column

        head = header[col] if col < len(header) else ""
        if not any(a in head for a in METRIC_ARROWS):
            add("WARN", "no-metric-direction",
                f"numeric column {col + 1} ({strip_cell(head)!r}) has no ↑/↓ marker; "
                "a reviewer should not have to infer which direction is better")

        precisions = {decimals(c) for c in numeric}
        if len(precisions) > 1:
            add("WARN", "mixed-precision",
                f"numeric column {col + 1} ({strip_cell(head)!r}) mixes decimal precision "
                f"{sorted(precisions)}; keep it constant within a metric column")

    return findings


def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("paper", help="Path to the LaTeX draft")
    ap.add_argument("--json", help="Optional path for the machine-readable report")
    ap.add_argument("--strict", action="store_true", help="Exit 1 on WARNs too")
    args = ap.parse_args()

    if not os.path.exists(args.paper):
        print(f"ERROR: file not found: {args.paper}", file=sys.stderr)
        return 2
    with open(args.paper) as f:
        tex = f.read()

    concl = re.search(r"\\section\*?\{\s*Conclusion", tex, re.IGNORECASE)
    concl_at = concl.start() if concl else len(tex)

    tables = find_tables(tex)
    findings: list[dict] = []
    for t in tables:
        findings.extend(lint_table(t, after_conclusion=t["start"] > concl_at))

    errors = [f for f in findings if f["severity"] == "ERROR"]
    warns = [f for f in findings if f["severity"] == "WARN"]

    print(f"table_lint: {len(tables)} table(s), {len(errors)} error(s), {len(warns)} warning(s)")
    for f in findings:
        print(f"  {f['severity']:5s} line {f['line']:>5} [{f['code']}] {f['message']}")

    if args.json:
        os.makedirs(os.path.dirname(os.path.abspath(args.json)) or ".", exist_ok=True)
        with open(args.json, "w") as fh:
            json.dump({"tables": len(tables), "findings": findings}, fh, indent=2,
                      ensure_ascii=False)

    if errors or (warns and args.strict):
        return 1
    if not findings:
        print("OK: all tables follow the booktabs conventions")
    return 0


if __name__ == "__main__":
    sys.exit(main())

#!/usr/bin/env python3
"""
test_table_lint.py — regression tests for the LaTeX table linter.

Stdlib only:

    python3 skills/section-writing-agent/scripts/test_table_lint.py
"""
import unittest

from table_lint import decimals, find_tables, is_numeric, lint_table, split_rows

GOOD = r"""
\begin{table}[t]
\centering
\caption{Accuracy and latency on the held-out split of Dataset X, averaged over five seeds.}
\label{tab:main}
\begin{tabular}{lrr}
\toprule
Method & Accuracy $\uparrow$ & Latency (ms) $\downarrow$ \\
\midrule
Baseline & 78.2 & 12.3 \\
Ours     & 85.4 & 8.1 \\
\bottomrule
\end{tabular}
\end{table}
"""


def codes(tex: str, after_conclusion: bool = False) -> set[str]:
    tables = find_tables(tex)
    out: set[str] = set()
    for t in tables:
        out |= {f["code"] for f in lint_table(t, after_conclusion)}
    return out


class TestGoodTable(unittest.TestCase):
    def test_clean_table_has_no_findings(self):
        self.assertEqual(codes(GOOD), set())

    def test_unicode_arrows_also_count(self):
        self.assertNotIn("no-metric-direction",
                         codes(GOOD.replace(r"$\uparrow$", "↑").replace(r"$\downarrow$", "↓")))


class TestErrors(unittest.TestCase):
    def test_vertical_rules(self):
        self.assertIn("vertical-rule", codes(GOOD.replace("{lrr}", "{l|r|r}")))

    def test_hline(self):
        self.assertIn("hline", codes(GOOD.replace(r"\midrule", r"\hline")))

    def test_missing_rules(self):
        self.assertIn("missing-rules", codes(GOOD.replace(r"\bottomrule", " ")))

    def test_caption_below_tabular(self):
        broken = r"""
\begin{table}[t]
\begin{tabular}{lr}
\toprule
Method & Accuracy $\uparrow$ \\
\midrule
Ours & 85.4 \\
\bottomrule
\end{tabular}
\caption{Accuracy on the held-out split of Dataset X across five random seeds.}
\label{tab:x}
\end{table}
"""
        self.assertIn("caption-below", codes(broken))

    def test_missing_caption(self):
        self.assertIn("no-caption", codes(
            GOOD.replace(r"\caption{Accuracy and latency on the held-out split of "
                         r"Dataset X, averaged over five seeds.}", " ")))

    def test_table_without_tabular(self):
        self.assertIn("no-tabular", codes(r"\begin{table}\caption{Nothing here at all.}\end{table}"))


class TestWarnings(unittest.TestCase):
    def test_missing_label(self):
        self.assertIn("no-label", codes(GOOD.replace(r"\label{tab:main}", " ")))

    def test_thin_caption(self):
        self.assertIn("thin-caption", codes(GOOD.replace(
            r"\caption{Accuracy and latency on the held-out split of Dataset X, "
            r"averaged over five seeds.}", r"\caption{Results.}")))

    def test_missing_metric_direction(self):
        found = codes(GOOD.replace(r"Accuracy $\uparrow$", "Accuracy")
                          .replace(r"Latency (ms) $\downarrow$", "Latency (ms)"))
        self.assertIn("no-metric-direction", found)

    def test_mixed_decimal_precision(self):
        self.assertIn("mixed-precision", codes(GOOD.replace("& 8.1 ", "& 8.14 ")))

    def test_table_after_conclusion(self):
        self.assertIn("late-table", codes(GOOD, after_conclusion=True))

    def test_text_column_is_not_checked_for_direction(self):
        # The Method column is text; it must not be asked for an arrow.
        found = codes(GOOD)
        self.assertNotIn("no-metric-direction", found)


class TestCellHelpers(unittest.TestCase):
    def test_bold_numbers_are_numeric(self):
        self.assertTrue(is_numeric(r"\textbf{85.4}"))

    def test_math_and_percent_are_numeric(self):
        self.assertTrue(is_numeric(r"$92.1$"))
        self.assertTrue(is_numeric(r"3.2\%"))

    def test_text_is_not_numeric(self):
        self.assertFalse(is_numeric("Critical"))
        self.assertFalse(is_numeric("Mar 2025"))

    def test_decimal_counting(self):
        self.assertEqual(decimals("85.4"), 1)
        self.assertEqual(decimals(r"\textbf{85.40}"), 2)
        self.assertEqual(decimals("12"), 0)

    def test_escaped_ampersand_is_not_a_column_break(self):
        rows = split_rows(r"R\&D & 12.0 \\")
        self.assertEqual(len(rows[0]), 2)


if __name__ == "__main__":
    unittest.main(verbosity=2)