# P0 #3 — GEML generation fluency

Can current Claude models emit GEML that the reference parser accepts, zero-shot vs. with the one-page [`SKILL.md`](SKILL.md)? Each cell is one generation per [fixture](fixtures.mjs); outputs are parsed unmodified.

- **parse-clean** — zero *error* diagnostics (refs resolve, fences close, formulas valid, ids unique).
- **feature-correct** — the requested construct was actually emitted (not avoided).
- **both** — feature-correct *and* parse-clean: did the task, in valid GEML.

| model | condition | parse-clean | feature-correct | both | warnings |
|---|---|---|---|---|---|
| haiku | skill | 67% (4/6) | 83% (5/6) | 67% (4/6) | 0 |
| haiku | zeroshot | 67% (4/6) | 17% (1/6) | 0% (0/6) | 3 |
| sonnet | skill | 83% (5/6) | 100% (6/6) | 83% (5/6) | 0 |
| sonnet | zeroshot | 83% (5/6) | 67% (4/6) | 50% (3/6) | 2 |

## Error breakdown (all error diagnostics across all runs)

| category | count |
|---|---|
| table formula (compute/summary) | 4 |
| unterminated fence (nesting) | 3 |
| chart binding | 2 |

## Findings (after the footgun fixes)

This round re-measures after two parser fixes — **footnote definitions** (`[^id]: text` now resolves, *Note: later removed in favor of `=== note {#id}`*) and **labeled close fences** (`=== #id` closes by name, independent of fence length) — plus the matching skill updates. Against the pre-fix baseline the error mix went from `{footnote 4, fence 3, formula 3, chart 2}` to `{formula 4, fence 3, chart 2}`: the footnote class is gone.

1. **The footnote fix landed cleanly.** Models reach for Markdown footnotes by habit; making `[^id]: text` a real definition removed the entire "unresolved footnote" category (4 → 0) and lifted haiku zero-shot parse-clean 50% → 67% — every `cross-refs` cell is now clean. *(Note: `[^id]: text` was subsequently removed to simplify the grammar. Models are now instructed to use `=== note {#id}` instead.)*
2. **The labeled close helps the capable model, less the weak one.** With the `=== #id` recipe, Sonnet+skill nests a code block inside a note correctly; Haiku still miscounts fences even with the recipe (the unterminated-block error now *names* the labeled close, but a small model doesn't take it). Fence nesting stays the footgun for small models.
3. **The compute/summary formula DSL is now the leading error category** (4): wrong column names and malformed formulas (`unknown column`, `missing )`). It was *not* changed this round — the next fix is to lift `compute`/`summary` out of the quoted attribute string onto their own body lines.
4. **A short skill still owns the *vocabulary* problem** (feature-correct: haiku 17%→83%, sonnet 67%→100% from zero-shot to skill). parse-clean is still short of high-90s (haiku 67%, sonnet 83%) — now bounded mainly by the formula DSL and weak-model fence nesting, not footnotes.

## Per-output detail

| model | condition | fixture | parse-clean | feature | first error |
|---|---|---|---|---|---|
| haiku | skill | chart-bound | ✗ | ✓ | `compute `Total`: unknown column `North`` |
| haiku | skill | cross-refs | ✓ | ✓ |  |
| haiku | skill | diagram-math | ✓ | ✓ |  |
| haiku | skill | fy-table | ✓ | ✓ |  |
| haiku | skill | nested-fences | ✗ | ✗ | `unterminated `code` block (no matching === or `=== #example`` |
| haiku | skill | nested-structure | ✓ | ✓ |  |
| haiku | zeroshot | chart-bound | ✓ | ✗ |  |
| haiku | zeroshot | cross-refs | ✓ | ✗ |  |
| haiku | zeroshot | diagram-math | ✓ | ✗ |  |
| haiku | zeroshot | fy-table | ✗ | ✓ | `summary `FY Total`: missing )` |
| haiku | zeroshot | nested-fences | ✗ | ✗ | `unterminated `callout` block (no matching ===)` |
| haiku | zeroshot | nested-structure | ✓ | ✗ |  |
| sonnet | skill | chart-bound | ✗ | ✓ | `summary targets unknown column `Region`` |
| sonnet | skill | cross-refs | ✓ | ✓ |  |
| sonnet | skill | diagram-math | ✓ | ✓ |  |
| sonnet | skill | fy-table | ✓ | ✓ |  |
| sonnet | skill | nested-fences | ✓ | ✓ |  |
| sonnet | skill | nested-structure | ✓ | ✓ |  |
| sonnet | zeroshot | chart-bound | ✓ | ✓ |  |
| sonnet | zeroshot | cross-refs | ✓ | ✗ |  |
| sonnet | zeroshot | diagram-math | ✓ | ✓ |  |
| sonnet | zeroshot | fy-table | ✓ | ✓ |  |
| sonnet | zeroshot | nested-fences | ✗ | ✓ | `unterminated `geml` block (no matching ======)` |
| sonnet | zeroshot | nested-structure | ✓ | ✗ |  |

## Method & caveats

- Sample size is small (one generation per cell); this is a directional measurement, not a benchmark.
- "parse-clean" is the load-bearing metric: GEML degrades unknown input to paragraphs/warnings, so almost anything *parses* — the question is whether the build-time **checks** accept it.
- **Tool-access caveat (important).** Generation subagents had tools; the more capable ones (Sonnet) ran the reference parser and iterated, so their parse-clean reflects "an agent with the build-checks in the loop" — an *upper bound*, not a single-shot completion. The Haiku cells that did a single `Write` are closest to one-shot, and they are the weakest. A true one-shot parse-clean is likely **below** the numbers above; a cleaner run would disable tool access during generation.
- Fixtures deliberately stress the two footguns the design review flagged: **fence-length nesting** (`nested-fences`) and the **compute/summary formula DSL** (`fy-table`).
- Reproduce: generate `bench/outputs/<model>__<cond>__<fixture>.geml`, then `node bench/score.mjs`.

# Writing GEML — one-page cheatsheet

GEML is plain text. Every structured thing is one **typed block**:

```
=== <type> {#id .class key=val}
<body>
===
```

The opening fence is **≥ 3** `=`. The **closing fence must be the same length**.
To nest a block inside a block, make the **outer fence longer**:

```
==== note
Example code:
=== code {lang=python}
print("hi")
===
====
```

**Easier — close by name.** Give a block an `#id` and close it with `=== #id`.
That closes by *name*, not by length, so you can nest with all `===` fences and
never miscount:

```
=== note {#ex}
Example code:
=== code {#snippet lang=python}
print("hi")
=== #snippet
=== #ex
```

Body modes: `code` / `diagram` / `math` / `table` / `embed` are **raw**;
`note` are **parsed prose**; `meta` is one `key=val` per line.

## Metadata
```
=== meta
title = "My doc"
version = 0.1
===
```
In prose, `{{title}}` inserts a meta value (an unknown key is a build **error**).

## Headings & lists (ATX only)
```
# Title          ## Section {#explicit-id}
- bullet         1. ordered
  - nested by indentation (2 spaces)
- [ ] task       - [x] done
```

## Inline
`*em*`  `**strong**`  `` `code` ``  `~~strike~~`  `$a^2$` (inline math)
`[text](https://x)`  `[text](#id)`  `[[#id]]` (auto text)  `[^note]`  `![alt](pic.png)`
A reference to a missing id is a build **error**. Define a footnote with
a `note` block (e.g. `=== note {#note}`), then cite it inline with `[^note]`.

## Tables
```
=== table {#fy caption="Sales" format=csv header=1}
Seg, Q1, Q2, Q3, Q4
Cloud, 1, 2, 3, 4
===

=== view {#fy-report src=#fy compute="FY [%.1f] = Q1 + Q2 + Q3 + Q4" summary="Seg = 'Total'; Q1 = sum(Q1); FY = sum(FY)"}
===
```
- A `table` holds FACTS. `compute`/`summary` — and `where`/`order`/`limit`/
  `select`/`by`/`aggregate` — belong to a `view` whose `src=` names the table.
- The attribute object `{…}` is **one physical line** (don't wrap it).
- `compute`: `Name = expr` over `+ - * / ( )`, columns by header name (quote names
  with spaces: `'Unit Price'`) or letter `A,B,…`; `;`-separate multiple formulas;
  `[%.1f]` sets number display.
- `summary`: one foot row; each cell is a literal (`'Total'`) or an aggregate
  expression over `sum/avg/min/max/count`. A bare (non-aggregated) column is an
  **error**.
- Visual form also works: `| a | b |` rows with a `|---|` separator.

## Diagrams & charts
```
=== diagram {#flow format=mermaid caption="Flow"}
graph LR
  A --> B
===
```
Chart drawn from a table or a view (empty body — the spec is in the attributes):
```
=== diagram {format=geml-chart data=#fy-report type=bar x=Seg y=FY}
===
```
`type` ∈ `bar|line|area|pie|scatter`; `x`/`y` are column names; `data=#id` must
point at a table, a `view`, or a record-array `data` block — a computed column
like `FY` exists only on the `view`, so bind there; a typo'd column or dangling
id is a build **error**.

## Math
```
=== math {#g caption="Gaussian"}
\int_{-\infty}^{\infty} e^{-x^2} dx = \sqrt{\pi}
===
```

## Things that are build ERRORS — avoid them
- A reference / footnote / chart column / `{{meta}}` key that does not resolve.
- A duplicate `#id`.
- A block not closed by an **equal-length** fence (when nesting, the **outer**
  fence must be longer than any fence inside it).
- A `summary` cell using a column that isn't wrapped in an aggregate.
- Raw HTML (there is none in GEML).