kaggle

<!-- file: composition.md — mode routing source of truth -->

# Mode composition contract

Select exactly one row. Read each named contract once from left to right and generate no unlisted section.

| Mode | Ordered contracts |
| -- | -- |
| `full` | `foundation.md(full)` → `eda.md` → `training.md` → `inference.md(attached)` → `submission.md` |
| `eda-only` | `foundation.md(eda-only)` → `eda.md` |
| `inference-only` | `foundation.md(inference-only)` → `inference.md(standalone)` → `submission.md` |

Apply `style-rules.md` to every row. Load `modality-dispatch.md` only when a selected section requests a modality branch.

Write `.experiments/kaggle/<competition>.py`; add the `-inference` suffix only for `inference-only`.

<!-- file: eda.md — selected by composition.md -->

# EDA section contract

Generate the EDA section after the foundation. Use only grounded paths and schema fields.

## Section 3: EDA

Open with a `# %% [markdown]` EDA header.

### Just-in-time configuration

Define only EDA constants, including the grounded target column and sample count:

```python
# %%
SAMPLE_N = 9
TARGET_COL = "<grounded-target-column>"
```

### Dataset overview

- Load the grounded training table or file index.
- Display shape, head, dtypes, missing values, and appropriate descriptive statistics.
- Confirm referenced files exist on a representative sample.
- Fail fast on required resources — primary training table, file index, grounded target column: `assert not df_train.empty, "..."`. A competition guarantees these exist, so empty/missing means the load is broken, not a state to print and roll past. Never `if df_train.empty: print("SKIP — ..."); ` and continue.
- `if <resource> missing: ...` conditional-skip is valid **only** for optional resources — supplementary/external datasets, pretrained checkpoints, additional model storages — never for required ones. Even then, never skip silently: print what's missing and what section/analysis it skips.
- Treat absent columns, duplicate identifiers, and unreadable samples explicitly (report the finding) — these are not competition-guaranteed and may legitimately vary.

### Target distribution

Plot the target distribution. For regression, include robust quantiles/outlier context; for segmentation/detection, summarize annotation prevalence and empty-target frequency.

### Hypothesis validation

Create a markdown hypothesis cell followed by an executable check for each decision-driving question. At minimum consider:

- class/target balance → loss, sampling, or stratification;
- spatial/sequence dimensions → resize, crop, padding, or batching;
- duplicates, leakage, or grouped entities → split strategy;
- missing/corrupt files → dataset guards;
- label noise or empty annotations → augmentation and evaluation behavior.

Every check ends with a printed finding and explicit design implication. Do not infer a conclusion from a plot without recording the observed statistic.

### Modality display

Load `modality-dispatch.md`:

```bash
_KAGGLE_MODES="${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/skills/kaggle/modes"
cat "$_KAGGLE_MODES/modality-dispatch.md"  # timeout: 5000
```

Select only the grounded branch, and define its visualization helper immediately before first use. Adapt every placeholder column and path from the fact table. Show representative samples and, where applicable, width/height, volume-shape, sequence-length, or point-count distributions.

### EDA lens

Display representative records/samples and print the grounded schema, target properties, missingness, duplicate/leakage checks, and the decisions carried into later stages. In EDA-only mode, retain these implications even though no later sections are generated.

<!-- file: foundation.md — selected by composition.md -->

# Notebook foundation contract

Generate the header, environment setup, imports, and path constants. Use the variant selected by `composition.md`.

## Variant matrix

| Variant | Title suffix | Package setup | Scope |
| -- | -- | -- | -- |
| `full` | `⚡PTL + <ModelLibrary>` for neural training | online download by default; frozen offline packages when requested | training and inference dependencies |
| `eda-only` | `— EDA` | online download; never offline-only | exploration dependencies only |
| `inference-only` | `— Inference` | frozen offline packages | inference dependencies only |

## Section 1: Header and setup

Start with a `# %% [markdown]` cell containing the grounded competition title, 2–3 sentences describing the selected scope and approach, and the competition URL when known. In inference-only mode, name the grounded checkpoint path or Kaggle input dataset.

Follow with one setup cell:

- Online: `# ! pip download -q <library> --dest frozen_packages/`, then install from that directory with online fallback.
- Offline: `# ! cp -r ../input/python-packages/frozen_packages .`, then install with `--no-index --find-links frozen_packages/` and a clearly disclosed fallback only when internet is allowed.
- Put modality-specific packages selected by `modality-dispatch.md` here, never later in the notebook.
- In inference-only mode, exclude training callbacks, logger packages, and training metrics not needed to deserialize the model.

## Section 2: Imports and paths

Use one `# %%` cell containing imports and global paths only:

- Standard library first (`glob`, `os`, `Path` as needed), then NumPy/pandas/plotting, then torch/model packages, then sklearn/XGBoost.
- Use `from tqdm.auto import tqdm`.
- Always import `torch` for neural inference/training notebooks.
- Suppress only a specific noisy warning category; do not blanket-ignore exceptions.
- Define grounded `PATH_DATASET`, `PATH_OUTPUT`, and when needed `PATH_MODELS`/`PATH_CHECKPOINT` as ALL_CAPS.
- Print package and device versions immediately after imports.
- For neural training, call `pl.seed_everything(42)` and seed every non-Lightning split/sampler explicitly.

Keep stage configuration out of this cell. EDA, data, model, training, and inference constants belong immediately before their respective stage.

## Foundation lens

Print resolved paths, device, and package versions. Check required input paths without claiming Kaggle-only paths exist locally during generation.

<!-- file: inference.md — selected by composition.md -->

# Inference section contract

Use the context selected by `composition.md`:

- `attached`: Section 7 of a full notebook, after training.
- `standalone`: Sections 3–6 of an inference-only notebook, starting from a grounded checkpoint.

## Attached context

Include both paths:

1. Run inference with the trained in-memory model under `torch.no_grad()`.
2. Load the best saved checkpoint/model artifact in a separate cell and run an equivalent prediction smoke check.

Use the test loader from the training pipeline, move only inference outputs to CPU, retain stable sample identifiers, and verify prediction count/shape before submission.

## Standalone context

### Section 3: Load model

Ground checkpoint path, format, model class, and required constructor arguments. Prefer importing `<competition>_model.py` emitted by training. If unavailable, define the complete model class with verified imports rather than inventing an API.

Choose the loader by evidence:

- Lightning `.ckpt`: `Model.load_from_checkpoint(...)` with the importable class.
- State dict `.pt`/`.pth`: construct the verified architecture, load the state dict, and check missing/unexpected keys.
- Serialized module: use `torch.load(..., map_location=DEVICE)` only when the artifact is known to contain a full trusted module.
- Custom detector/MONAI model: verify the installed constructor and checkpoint contract first.

Fail clearly when no checkpoint matches; never index `sorted(...)[-1]` without an empty-match guard. Set evaluation mode, move to the selected device, and print model type, device, and parameter count.

### Section 4: Test data

Build a label-free test Dataset/DataLoader or grounded modality equivalent:

- preserve sample ordering and stable IDs;
- use evaluation transforms matching training;
- set `shuffle=False`;
- assert batch shape and dtype;
- print sample and batch counts;
- handle empty test data explicitly.

Detection may use single-image iteration when required by the verified predictor API. Volumetric pipelines must preserve original shape metadata for output restoration.

### Section 5: Inference loop

Run under `torch.no_grad()` and choose output activation/decoding from the grounded task:

- binary classification: sigmoid only when the model returns logits;
- multiclass: softmax/argmax according to submission requirements;
- regression: retain scalar/vector predictions without classification transforms;
- detection: apply verified confidence filtering and class-aware NMS when the model does not already do so;
- segmentation: restore predictions to grounded original dimensions with appropriate interpolation.

Collect predictions and IDs, then assert count, shape, dtype, finiteness, and expected range before post-processing. CPU transfer is allowed here, outside the training loop.

### Section 6: Post-processing

Apply only post-processing justified by the metric and output contract:

- threshold calibration for classification;
- box rescaling and formatting for detection;
- morphology/component filtering for segmentation;
- inverse transforms for normalized regression targets.

Keep parameters in a just-in-time config cell. Define helpers immediately before use.

## Inference lens

Show a small prediction sample, print prediction/ID counts and shapes, check NaN/Inf and range constraints, and compare the attached in-memory versus reloaded path when both exist.

<!-- file: modality-dispatch.md — consumers: eda.md, training.md, inference.md -->

## Modality-specific sample display — dispatch by `input_modality`

Use this dispatch wherever samples need showing (EDA Section 3, training sanity check, inference spot check). Pick matching branch; each is self-contained set of cells.

## Contents

- `image`: 2D image grids and dimension checks
- `image-3d`: volumetric loading, three-plane viewer, and statistics
- `tabular`: descriptive statistics, correlation, and target plots
- `point-cloud`: bounded 3D point display

### `image` — 2D images (default)

Setup: no extra installs needed.

```python
# %%
# show_images: grid of N samples with label overlay
def show_images(df, n=9, img_dir=PATH_DATASET):
    n_cols = 3; n_rows = (n + n_cols - 1) // n_cols
    fig, axes = plt.subplots(n_rows, n_cols, figsize=(16, 4 * n_rows))
    for ax, (_, row) in zip(axes.flat, df.sample(n).iterrows()):
        img = plt.imread(os.path.join(img_dir, row["image_id"]))  # adapt column name
        ax.imshow(img); ax.set_title(str(row.get("label", "")), fontsize=8); ax.axis("off")
    plt.tight_layout(); plt.show()

_= show_images(df_train)
```

Optional dimension scatter (when image sizes vary):

```python
# %%
from PIL import Image
df_train["w"], df_train["h"] = zip(
    *[Image.open(os.path.join(PATH_DATASET, r["image_id"])).size
      for _, r in df_train.iterrows()]
)
_= df_train.plot.scatter("w", "h", alpha=0.3, title="Image dimensions")
```

### `image-3d` — volumetric / TIFF stacks / medical imaging

Setup cell (add to notebook setup `# %%`):

```python
# %%
# ! pip install -q tifffile imagecodecs ipywidgets
# ! pip list | grep -E 'tifffile|ipywidgets'
```

Imports (add to imports `# %%`):

```python
import tifffile
import ipywidgets as widgets
from ipywidgets import interact, IntSlider, FloatSlider
from matplotlib.colors import ListedColormap, BoundaryNorm
```

Load one volume:

```python
# %%
def load_volume(sample_id):
    img  = tifffile.imread(os.path.join(PATH_DATASET, "train_images", f"{sample_id}.tif"))
    mask = tifffile.imread(os.path.join(PATH_DATASET, "train_labels", f"{sample_id}.tif"))
    return img, mask

sample_id = df_train.iloc[0]["id"]   # adapt column name
vol, mask_vol = load_volume(sample_id)
print(f"Volume: {vol.shape} {vol.dtype}  |  Mask: {mask_vol.shape}  labels: {np.unique(mask_vol)}")
```

Interactive 3-plane viewer + mask overlay:

```python
# %%
_MASK_COLORS = ["lightgray", "yellow", "cyan", "red"]
_MASK_CMAP   = ListedColormap(_MASK_COLORS[:len(np.unique(mask_vol))])
_MASK_NORM   = BoundaryNorm(np.arange(-0.5, len(np.unique(mask_vol))), _MASK_CMAP.N)

def show_volume(vol, mask_vol, z, y, x, mask_alpha):
    vZ, vY, vX = vol.shape[:3]
    fig = plt.figure(figsize=(14, 14))
    ax_xy = fig.add_subplot(2, 2, 1)
    ax_yz = fig.add_subplot(2, 2, 2)
    ax_xz = fig.add_subplot(2, 2, 3)
    ax_3d = fig.add_subplot(2, 2, 4, projection="3d")

    for ax, sl, msl, title in [
        (ax_xy, vol[z, :, :], mask_vol[z, :, :], f"Axial z={z}"),
        (ax_yz, vol[:, :, x], mask_vol[:, :, x], f"Sagittal x={x}"),
        (ax_xz, vol[:, y, :], mask_vol[:, y, :], f"Coronal y={y}"),
    ]:
        ax.imshow(sl, cmap="gray")
        ax.imshow(msl, cmap=_MASK_CMAP, norm=_MASK_NORM, alpha=mask_alpha, interpolation="nearest")
        ax.set_title(title); ax.axis("off")

    Yg, Xg = np.meshgrid(np.arange(vY), np.arange(vX), indexing="ij")
    Ys, Zs = np.meshgrid(np.arange(vY), np.arange(vZ), indexing="ij")
    Xc, Zc = np.meshgrid(np.arange(vX), np.arange(vZ), indexing="ij")
    ax_3d.plot_surface(Xg, Yg, np.full_like(Xg, z),   color="r", alpha=0.15)
    ax_3d.plot_surface(np.full_like(Ys, x), Ys, Zs,   color="b", alpha=0.10)
    ax_3d.plot_surface(Xc, np.full_like(Xc, y), Zc,   color="g", alpha=0.10)
    ax_3d.set(xlabel="X", ylabel="Y", zlabel="Z"); ax_3d.set_title("Slice planes")
    plt.tight_layout(); plt.show()

interact(
    lambda z, y, x, mask_alpha: show_volume(vol, mask_vol, z, y, x, mask_alpha),
    z=IntSlider(min=0, max=vol.shape[0]-1, step=1, value=vol.shape[0]//2, description="Z-slice"),
    y=IntSlider(min=0, max=vol.shape[1]-1, step=1, value=vol.shape[1]//2, description="Y-slice"),
    x=IntSlider(min=0, max=vol.shape[2]-1, step=1, value=vol.shape[2]//2, description="X-slice"),
    mask_alpha=FloatSlider(min=0.0, max=1.0, step=0.05, value=0.3, description="Mask α"),
)
```

Volume statistics:

```python
# %%
print(f"Volume: min={vol.min()}, max={vol.max()}, mean={vol.mean():.2f}")
label_counts = dict(zip(*np.unique(mask_vol, return_counts=True)))
print(f"Mask labels: {label_counts}")
_= pd.Series(label_counts).plot(kind="bar", title="Label voxel counts")
```

### `tabular` — structured / CSV only

Setup: no extra installs beyond pandas.

```python
# %%
# shape, dtypes, missing values
print(df_train.shape)
display(df_train.describe())
print(df_train.isnull().sum().sort_values(ascending=False).head(20))
```

```python
# %%
import seaborn as sns
num_cols = df_train.select_dtypes("number").columns.tolist()
if len(num_cols) <= 20:
    fig, ax = plt.subplots(figsize=(len(num_cols), len(num_cols)))
    sns.heatmap(df_train[num_cols].corr(), annot=True, fmt=".2f", ax=ax)
    ax.set_title("Feature correlation"); plt.tight_layout(); plt.show()
```

```python
# %%
# Target distribution
_= df_train[TARGET_COL].value_counts().plot(kind="bar", title="Target distribution")
```

### `point-cloud` — 3D point sets

Setup cell:

```python
# %%
# ! pip install -q open3d
# ! python -c "import open3d; print(open3d.__version__)"
```

```python
# %%
import open3d as o3d

def show_pcd(path, n_points=50_000):
    pcd = o3d.io.read_point_cloud(str(path))
    pts = np.asarray(pcd.points)[:n_points]
    fig = plt.figure(figsize=(10, 7))
    ax  = fig.add_subplot(111, projection="3d")
    ax.scatter(pts[:, 0], pts[:, 1], pts[:, 2], s=0.1, alpha=0.5)
    ax.set_title(str(path)); plt.tight_layout(); plt.show()

matches = sorted(Path(PATH_DATASET).glob("**/*.pcd"))
if not matches:
    raise FileNotFoundError(f"No .pcd files under {PATH_DATASET}")
sample_path = matches[0]
show_pcd(sample_path)
```

<!-- file: style-rules.md — applied by composition.md -->

Apply ALL of these in generated script:

01. **Every section `# %% [markdown]` header: extensive narrative, not caption**: after `## Section Name`, write a full explanation, not a 2-sentence blurb — cover (a) what this stage does, (b) why this approach over named alternatives (trade-offs stated), (c) how it advances the competition objective (metric, leaderboard placement, submission quality), (d) how it builds on the previous section's finding and sets up the next. Read the whole notebook top to bottom like a university/seminar lecture on solving this competition — reader new to it follows the full reasoning chain, not just code output. Applies to every heading level generated (`##`, `###`, `####`), not only top-level section openers: a subsection heading (e.g. `### Dataset overview`) still needs at least one full sentence beneath it before any list, table, blockquote, or code cell follows. Bare heading with nothing but a list/table/code underneath — no sentence at all — forbidden at any level.
02. **Structured markdown over prose blocks**: whenever a section's content has more than one comparable item — options considered, config values, metrics, schema fields, decisions — render it as a bulleted/numbered list or a markdown table, not a wall of paragraph text. Plain unbroken paragraphs are boring, easy to skim past; structure (tables, lists, short bolded lead-ins, blockquote takeaways) is scannable, keeps the reader oriented. Prose paragraphs still carry the connecting narrative between structured blocks — this rule trims filler paragraphs, doesn't replace narrative with bullet fragments. Separate stacked blocks (table→list, list→table) with a truly blank line only — never a bare `#` spacer line (rule 13): a lone `#` renders as an empty H1, not whitespace.
03. **Markdown ↔ plot cells flow together**: the markdown cell immediately before a plotting cell states what the plot will show and the question it answers — never drop a chart on the reader cold. The markdown (or short comment) immediately after states the observed pattern and its design implication (what changes because of what was just seen). Plot → interpretation → decision is one continuous beat, never an orphaned chart with no before/after framing.
04. **Small, single-purpose code cells**: one action per `# %%` cell — one load, one transform, one plot, one check, one train call. Never bundle load+display+validate, or setup+run+verify, into a single cell to save cell count; split immediately when a cell does more than one of those. Exception: a just-in-time config cell (constants only) may precede its action cell without being split further.
05. **Every code cell carries a why, not a what**: one short line — inline `#` comment (procedural cells) or the preceding markdown sentence (section-opening cells) — states the *specific reason* this step happens here: a metric choice, a leakage risk avoided, a memory limit, a competition-specific quirk. Never restate what the code already shows (`# load the data` forbidden); no why → no cell.
06. **Long comments move to a markdown cell, never sit in code**: a comment longer than one short why-line — a multi-line `#` block, or a paragraph explaining rationale/trade-offs — does not belong inside a code cell. Extract it into its own `# %% [markdown]` cell (using rule 2's structured formatting, not a wall of `#`-prefixed lines). If the long comment sits mid-cell **at top level, between statements**, split there: `code cell` (up to the comment) → `# %% [markdown]` cell (the extracted explanation) → `code cell` (continuing after it). **Not feasible mid-function or mid-class body**: a markdown cell cannot interrupt a `def`/`class` block — the code before it would be incomplete, unparsable on its own. When the long comment sits inside a function/class, either move the whole rationale into the markdown cell immediately *before* the `def`/`class` (function-level docstring covers the *what*; the preceding markdown covers the *why* at length) or compress it to a single why-line (rule 5) that stays inline — never split the function/class itself.
07. **Shell commands — `# ! cmd` inline, `%%bash` for multi-line blocks**: single shell command mixed into a Python cell → `# ! cmd` (valid Python syntax, visible as comment in Jupyter). Cell of 2+ consecutive shell commands (installs, `nvidia-smi` + `ls -lh` + `df -h` chains) → dedicate the whole cell to `%%bash` cell magic as its first line, no Python statements in that cell — simpler than repeating `# !` per line. `%matplotlib inline` verbatim — never `get_ipython().run_line_magic(...)`. Linter rejects `%`/`%%` magic → allowlist `%%bash` and `%matplotlib` in linter config, don't avoid the magic
08. `# ==============================` between logical blocks within cell (not every line — only major breaks)
09. `_=` to suppress matplotlib/pandas return values: `_= df["col"].plot(...)`
10. **Every plot: axis labels + grid + legend when multiple series**: always call `plt.xlabel("...")`, `plt.ylabel("...")`, `plt.grid(True)` after any plot; chart with multiple lines/bars/hues → add `plt.legend()` or pass `legend=True`; seaborn facets use `g.set_axis_labels("x label", "y label")`
11. No `if __name__ == '__main__':` guards
12. No argparse, no dataclasses for config
13. **Markdown blank lines — empty lines only, never bare `#`**: inside `# %% [markdown]` cells, use a truly empty line to separate paragraphs, list, table, and blockquote blocks. Never write a bare `#` or `# ` line as a spacer — Jupyter/Kaggle renders it as an empty level-1 heading, not whitespace: source looks like a harmless blank line but output shows a stray heading plus its outsized margin, blowing a large empty gap into the rendered cell. Risk rises with rule 2's heavier list/table/blockquote stacking — check every inter-block gap is an actual empty line before finishing a markdown cell.
14. **`display()` over `print()` for pandas objects**: use `display(df.head())`, `display(df.dtypes)`, `display(metrics.dropna(axis=1, how="all").head())`; use `print()` only for scalars and status strings
15. **No doctests in ipy scripts**: doctests belong in package modules, not notebook scripts — `# %% [markdown]` cell above function cell IS explanation; don't duplicate as doctest
16. **Compact docstrings — never omit**: always include one-line docstring; never omit — narrative lives in `# %% [markdown]` cell immediately above function cell; full Google-style docstrings with `Args:`, `Returns:`, `Example:` blocks apply only after distillation to `src/` utils package
17. **No forward references in headers**: describe only what the cell contains now; keep future refactoring or package-distillation plans out of notebook headings

<!-- file: submission.md — selected by composition.md -->

# Submission section contract

Generate final section from grounded sample-submission or competition output evidence.

## CSV classification/regression

- Read grounded sample submission.
- Join predictions by stable ID if ID column exists; never rely on incidental row order.
- Assign exact grounded target column(s).
- Preserve required column order and row count.
- Write `submission.csv` without an index.

## Detection

- Use grounded coordinate order, scale, class mapping, score precision, empty-detection representation.
- Format one prediction record per required sample ID.
- Validate boxes finite, ordered, within expected image bounds.

## Segmentation or file outputs

- Restore original spatial shape.
- Use grounded file format, dtype, naming, compression, directory structure.
- Validate written file count against expected sample IDs.

## Submission lens

Verify before reporting completion:

- row/file count equals expected test count;
- columns/schema and order match grounded evidence;
- IDs unique, cover expected set;
- predictions contain no unintended NaN/Inf;
- values, labels, boxes, shapes satisfy grounded constraints.

End CSV workflows with `# ! head submission.csv`; equivalent listing/schema check for non-CSV. Display small sample, print final path.

<!-- file: training.md — selected by composition.md -->

# Training section contract

Generate data/features, model, and training. Derive configuration from EDA evidence.

## Section 4: Dataset/DataModule or feature engineering

Open with markdown header and just-in-time config cell for batch size, input size, validation fraction, workers.

For neural pipelines:

- Make Dataset accept pre-split table/index plus transforms; do not hide splitting in mode argument.
- Return tensors and stable identifiers needed by inference.
- Assert tensor shape and dtype at Dataset/DataLoader boundary.
- Make LightningDataModule sole owner of seeded, leakage-aware split.
- Use grouped or stratified splitting when grounded EDA requires it; do not default blindly to row shuffling.
- Define train/validation/test loaders with explicit shuffle and worker behavior.

For tabular/non-neural pipelines:

- Build reproducible sklearn/XGBoost pipeline for missing values, categoricals, feature transforms.
- Use seeded split appropriate to target and group structure.
- Keep preprocessing fitted on training data only.

Add lens cell that creates pipeline, reads one batch/sample, prints shapes and dtypes, visualizes representative batch through selected modality helper when meaningful.

## Section 5: Model

Open with markdown header and just-in-time model constants such as `MODEL_NAME`, `MAX_EPOCHS`, `LEARNING_RATE`.

For neural training:

- Use LightningModule and current verified package APIs.
- Save hyperparameters needed for checkpoint restoration.
- Keep `forward` shallow; validate model input/output shape and dtype.
- Choose loss and TorchMetrics from grounded competition metric and EDA findings.
- Log separate train/validation metrics without manual epoch-end metric misuse.
- Use AdamW plus justified scheduler; commented alternatives optional, not mandatory clutter.
- Write reusable model definition with `%%writefile <competition>_model.py` when companion inference notebook must import it, then import it in training notebook.

For pure tabular baselines, use verified sklearn/XGBoost API and fixed random seed; do not wrap it in Lightning.

## Section 6: Training

For Lightning training:

- Use `CSVLogger`, `ModelCheckpoint`, `LearningRateMonitor`, justified early stopping.
- Set `accelerator="auto"`, `devices="auto"`, supported mixed-precision setting.
- Match checkpoint/early-stopping direction to grounded metric.
- Keep visible `fast_dev_run` option for debugging without claiming it was executed.
- Call `trainer.fit` after training configuration and lens checks are ready.
- Print actual best checkpoint path after training.

For non-neural training, fit pipeline and evaluate grounded metric on validation data with correct direction.

### Training lens

Read `metrics.csv` for neural runs, display metric columns, and plot train/validation curves. For non-neural runs, display validation metrics and relevant diagnostics.