Last verified: 2026-07-13 — refresh when a new
size-class anchor is validated or optimizer/dtype
defaults change.

# Memory Math

A worksheet for estimating whether a model
size-class, method, and batch/pack combination
fits available memory before a run. This is
planning math, not a guarantee — leave headroom
rather than sizing to the byte. Base models are
never named here; every example is labeled by
size class only (for example, "8B-class LoRA
bf16"). See `model-catalog.md` for which actual
model to use at a given size class.

## The Four Terms

Total footprint ≈ **weights + optimizer states +
gradients + activations**, plus a near-zero term
for LoRA/QLoRA adapters. Work each term from
parameter count and dtype, then sum.

### 1. Weights

`params × bytes/param`, by dtype:

| dtype | bytes/param |
|---|---|
| fp32 | 4 |
| bf16 / fp16 | 2 |
| int8 | 1 |
| int4 (QLoRA NF4) | 0.5 |

This term dominates for full fine-tuning, and the
calculation (`params × bytes/param`) is the same
formula regardless of method — but the dtype, and
so the result, is not: bf16 LoRA loads weights at
2 bytes/param while int4 QLoRA loads the same
parameter count at 0.5 bytes/param, a 4x gap.
Reuse the formula across methods; never reuse the
resulting weight-memory number from one method's
dtype for another's.

### 2. Optimizer states

Full fine-tuning carries optimizer state for
every trainable parameter; LoRA and QLoRA carry
it only for the adapter parameters, which is why
this term is negligible for them regardless of
base model size.

| Optimizer | bytes/param (trainable only) |
|---|---|
| AdamW, fp32 states | 8 (4B momentum + 4B variance) |
| AdamW 8-bit | ≈2 (quantized momentum + variance) |

8-bit AdamW roughly quarters this term versus the
fp32 variant for any run that isn't LoRA/QLoRA-
adapter-only, where it's already negligible.

### 3. Gradients

Same dtype as compute precision — typically bf16,
so 2 bytes/param — and, like optimizer state,
only for trainable parameters. Full fine-tuning
pays this for every weight; LoRA and QLoRA pay it
only for the adapter, since frozen base weights
never accumulate a gradient.

### 4. Activations

The hardest term to pin to a single number — it
scales with batch size, sequence/packing length,
and architecture, not just parameter count. Two
levers matter more than exact estimation:

- **Gradient checkpointing** trades recompute for
  memory: expect roughly **30% savings** on this
  term versus no checkpointing, at the cost of a
  recompute pass per checkpointed segment.
- Packing/sequence length is a more direct lever
  than batch size for this term.

### 5. LoRA/QLoRA adapter overhead

A rank-`r` adapter on a linear layer adds
`r × (in + out)` parameters — `A` is `r×in` and `B`
is `out×r`, so together they contribute
`r·in + r·out`. At normal rank sizes (1–32 for RL,
up to ~256 for SFT-at-scale),
this is a small fraction of a percent of base
model size — round it to zero in the worksheet
unless an unusually high rank is in play.

## Worked Examples

### 8B-class LoRA, bf16

Weights dominate; optimizer state and gradients
are adapter-only and small.

```python
params = 8e9
weights_gb = params * 2 / 1e9   # bf16, step 1
adapter_gb = 0.2                # step 5, negligible
total_gb = weights_gb + adapter_gb  # + activations
print(f"{total_gb:.0f}GB before activations")
```

Weights alone land around 16GB — the reference
point for "an 8B-class model fits comfortably on
a single high-memory GPU in bf16 LoRA."

### 8B-class QLoRA

Same parameter count, quantized weights:

```python
params = 8e9
weights_gb = params * 0.5 / 1e9  # int4 NF4, step 1
adapter_gb = 0.2                 # step 5, negligible
total_gb = weights_gb + adapter_gb  # + activations
print(f"{total_gb:.0f}GB before activations")
```

Weights land around 4GB — roughly a 4x reduction
versus bf16 LoRA, which is why QLoRA is the
method that buys headroom for larger batch size
or longer packing at the same size class, not
just a way to fit bigger models.

### 70B-class QLoRA (≈40GB anchor)

```python
params = 70e9
weights_gb = params * 0.5 / 1e9  # int4 NF4, step 1
adapter_gb = 0.5                 # step 5, negligible
total_gb = weights_gb + adapter_gb  # + activations
print(f"{total_gb:.0f}GB before activations")
```

The idealized formula lands weights at **≈35GB**
(decimal GB, weights only); treat **≈40GB** as the
real-world anchor once quantization metadata
(NF4 double-quant constants) and runtime overhead
are included — the reference point for "a 70B-class
model is reachable via QLoRA, not bf16," where
bf16 weights alone (≈140GB) would already exceed
most single-device budgets before optimizer state,
gradients, or activations are added. A plan
estimating far above the ≈40GB anchor for the same
size class is a signal to recheck dtype and
method, not just add headroom.

## Using These Numbers

1. Pick the size class and method from
   `model-catalog.md`.
2. Sum weights + optimizer + gradients from the
   tables above for that combination.
3. Add activations, applying the ~30% gradient-
   checkpointing saving if it's enabled.
4. Compare against the closest worked example or
   anchor above rather than trusting the estimate
   in isolation — a plan far off an anchor for the
   same size class and method is a signal to
   recheck inputs before assuming the hardware
   won't work.

# Model Catalog

Last verified: 2026-07-14
Refresh checklist: (1) check Unsloth supported-models page, (2) check the current open-weights leaderboards for each size class, (3) update rows + bump this date. Refresh at least quarterly; this file is the ONLY place base models are named in the llm-finetuning and dgx-spark-ops plugins.

## How to Read This Table

Pick the row matching the target parameter count,
then read across: a text recommendation, a vision
(VLM) recommendation for the same size class, what
that class can do on a single DGX Spark, and any
notes that change the recommendation. Cross-check
the "last verified" date above before trusting a
row — if it's stale, work the refresh checklist
first.

## Catalog (2026-07)

| Size class | Text recommendation | Vision recommendation | Spark feasibility | Notes |
|---|---|---|---|---|
| ≤4B | Qwen3 4B class | SmolVLM / Gemma 3 4B | Full fine-tune feasible | Smallest class where full FT is still *feasible* by default — a hardware/size-class note, not a method recommendation. Method choice (LoRA vs. full FT) is `lora-qlora-recipes`'s LoRA vs QLoRA vs Full FT table, routed by task shape (demonstrations vs. dense knowledge injection); that table governs over this feasibility note whenever the two appear to disagree. |
| 7–9B | Qwen3 8B, Llama-class 8B | Qwen2.5-VL-7B | Full fine-tune ceiling | Above this class, full FT stops being the default on Spark — see 12–32B row. |
| 12–32B | Qwen3 14B/32B, Gemma 3 27B | Qwen2.5-VL-32B | LoRA-only; 27B is the LoRA ceiling at pack≤1024 | 27B is the largest dense model that fits a LoRA run on a single Spark in practice. |
| 70B+ | Llama 3.3 70B class | Use the 12–32B vision class instead — no 70B+ VLM recommendation at this size | QLoRA-only, ≈40GB, 30–48h for 3 epochs | bf16 is not feasible at this class on a single Spark; QLoRA is the only path in. |
| 100B+ MoE | gpt-oss-120b class | Use the 12–32B vision class instead — no 100B+ MoE VLM recommendation at this size | NVFP4-native LoRA via community recipe (`nvfp4-lora-spark`), experimental | Not the default assumption for other 100B+ MoE models — verify per-model before relying on this row. |

## Vision Model Notes

- **LLaVA is legacy.** Do not recommend it for new
  work; it is listed here only so a stale
  recommendation can be recognized as such.
- **InternVL3.5 MoE variants are the MoE VLM
  alternative** to the dense Qwen2.5-VL / Qwen3-VL
  and Gemma 3 vision models above, for cases that
  specifically call for a mixture-of-experts
  vision-language architecture — InternVL3.5 also
  ships dense checkpoints, so pick the MoE variant
  explicitly rather than assuming every InternVL3.5
  release is MoE.