Claude Skill

alterlab-transformers

Loads, runs, and fine-tunes pretrained models with Hugging Face Transformers v5 (PyTorch-only) — pipeline() inference for chat-model text generation, text classification, NER, zero-shot, speech recognition, image classification, object detection, and image-text-to-text VLMs; Auto

LLM Mart · 0 points · 2 views 0 listing impressions 0 install-command copies
Download AlterLab-IEU-AlterLab-Academic-Skills-skills_data-science_alterlab-transformers-3ab6d03.zip · 28 KB

Install

skills CLI npx skills add https://github.com/AlterLab-IEU/AlterLab-Academic-Skills/tree/main/skills/data-science/alterlab-transformers
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install alterlab-ieu-alterlab-academic-skills@llmmart
Git git clone https://github.com/AlterLab-IEU/AlterLab-Academic-Skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole alterlab-ieu/alterlab-academic-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Transformers

Overview

Hugging Face Transformers loads pretrained checkpoints from the Hub for NLP, vision, audio, and multimodal tasks, runs inference through pipeline() or the Auto* classes, and fine-tunes them with Trainer. This skill targets transformers v5 (≥ 5.0; current 5.17.0 as of 2026-09). v5 is PyTorch-only and changes several v4 idioms, so check "v5 changes that break v4 code" below before reusing older snippets from papers, blogs, or model cards.

When to Use This Skill

Use this skill when the user wants to:

  • Run quick inference with a Hub checkpoint via pipeline() — text classification, NER, zero-shot classification, text generation with a chat model, speech recognition, image classification/detection, or image-text-to-text with a VLM.
  • Load a model plus tokenizer/processor with explicit dtype, device_map, attention backend, or 4/8-bit quantization.
  • Control decoding in model.generate() (greedy, sampling, beam search, streaming, chat templates).
  • Fine-tune an encoder or decoder on a custom labelled dataset with Trainer, optionally with LoRA via peft.
  • Port v4-era code (torch_dtype=, load_in_8bit=, removed pipelines, TF/Flax models) to v5.

Does NOT Trigger

Scenario Use Instead
Classical ML on tabular features (random forests, preprocessing pipelines, CV grid search) alterlab-scikit-learn
Structuring a custom PyTorch architecture's training loop, multi-GPU strategy, and checkpointing with a LightningModule alterlab-pytorch-lightning
Topic models, embeddings, or text classifiers as a social-science measurement design (validity, BERTopic, dictionaries) alterlab-text-as-data
Protein language models (ESM3 / ESM C embeddings, protein design) alterlab-esm
Zero-shot time-series forecasting with a pretrained foundation model alterlab-timesfm

Hand-off rule. When the request matches a row above, open your answer by naming that skill by its exact id (for example alterlab-scikit-learn) and saying in one line why it fits better, then stop applying this skill's procedure. Name it even when you also need to ask for missing input. If only part of the request is out of scope, name the skill for that part and do only the in-scope part. A brief general pointer is fine; the named skill does the work.

Installation

uv run --python ">=3.12" --with "transformers>=5" --with torch --with accelerate --with datasets python infer.py   # no install step
# torch + transformers already importable (`python -c "import torch, transformers"`)? run with plain `python`. CPU-only machine: add --index https://download.pytorch.org/whl/cpu
  • Vision: add pillow and torchvision (the default image-processor backend; timm only for timm-backed models).
  • Audio: add librosa soundfile.
  • LoRA / quantization: peft, bitsandbytes (CUDA). Metrics: evaluate or plain scikit-learn metrics.

Authentication

Public checkpoints (everything in this skill's examples) download with no token or account. A Hub token only adds access to gated or private models and higher download rate limits. Either log in once:

from huggingface_hub import login
login()  # or run `hf auth login` in a shell (huggingface-cli is deprecated)

or export the variable the Hub client reads:

export HF_TOKEN="your_token_here"

Tokens: https://huggingface.co/settings/tokens. Pass token= (not the removed use_auth_token=) when you need it explicitly.

Quick Start

from transformers import pipeline

# Text generation with a small open chat model (CPU-friendly; add device_map="auto" on GPU)
generator = pipeline("text-generation", model="Qwen/Qwen3-0.6B")
messages = [{"role": "user", "content": "Explain p-hacking in two sentences."}]
out = generator(
    messages,
    max_new_tokens=128,
    tokenizer_encode_kwargs={"enable_thinking": False},  # Qwen3 chat-template switch; omit for other models
)
print(out[0]["generated_text"][-1]["content"])

# Text classification — pin the checkpoint; task defaults change between releases
classifier = pipeline("text-classification", model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")
classifier(["This movie was excellent!", "Terrible pacing."])

# Zero-shot classification
zero_shot = pipeline("zero-shot-classification", model="facebook/bart-large-mnli")
zero_shot("The grant covers two postdoc salaries.", candidate_labels=["funding", "teaching", "ethics"])

Use max_new_tokens (tokens to generate) rather than max_length (prompt + output, and in the text-generation pipeline it also feeds tokenizer truncation). Summarization, translation, and question answering no longer have dedicated pipelines in v5 — prompt a chat model as above.

v5 Changes That Break v4 Code

v4 idiom v5 replacement
torch_dtype=torch.float16 dtype=torch.float16 (torch_dtype only warns). Default is now dtype="auto" — the checkpoint's dtype, often bf16 — not float32
TFAutoModel… / FlaxAutoModel…, framework="tf", return_tensors="tf" Removed; PyTorch only (return_tensors accepts "pt", "np", "mlx")
load_in_8bit=True / load_in_4bit=True quantization_config=BitsAndBytesConfig(...)
pipeline("summarization" / "translation_xx_to_yy" / "text2text-generation" / "question-answering") Removed — pipeline("text-generation") with a chat model and an instruction prompt
pipeline("image-to-text" / "visual-question-answering") pipeline("image-text-to-text") with a VLM, e.g. Qwen/Qwen3-VL-2B-Instruct
AutoModelForVision2Seq, AutoModelWithLMHead AutoModelForImageTextToText; AutoModelForCausalLM / AutoModelForMaskedLM / AutoModelForSeq2SeqLM
apply_chat_template(..., tokenize=True) returned a tensor of ids Returns a BatchEncoding (input_ids, attention_mask) → model.generate(**inputs)
Slow vs fast tokenizers, use_fast= One tokenizer per model on the 🤗 tokenizers backend; AutoTokenizer ignores use_fast
batch_decode, encode_plus, additional_special_tokens decode handles batches, tokenizer(...), extra_special_tokens (old names kept for backward compatibility)
penalty_alpha (contrastive search), force_words_ids / constraints, group beam search, DoLa Moved to Hub custom_generate repos — e.g. custom_generate="transformers-community/contrastive-search", trust_remote_code=True
TrainingArguments(logging_dir=..., warmup_ratio=...) Both removed: set the TENSORBOARD_LOGGING_DIR env var; warmup_steps=0.1 (a float < 1 is a ratio)
Trainer(tokenizer=...) Trainer(processing_class=...); report_to now defaults to "none"
save_pretrained(safe_serialization=False), use_auth_token= Always safetensors; token=
TRANSFORMERS_CACHE, low_cpu_mem_usage=True HF_HOME / HF_HUB_CACHE; low-memory loading is always on (flag ignored)

Source: the official v5 migration guide, cross-checked against the 5.17.0 release.

Core Capabilities

1. Pipelines for Quick Inference

One call covers tokenization, the forward pass, and post-processing for text classification, NER, zero-shot, fill-mask, text generation, ASR, audio classification, image classification/segmentation, object detection, depth estimation, and image-text-to-text. Use for prototyping and batch inference without custom preprocessing. See references/pipelines.md.

2. Model Loading and Management

from_pretrained with dtype, device_map, attn_implementation (SDPA by default), and quantization_config; saving, Hub upload, and ONNX export options. See references/models.md.

3. Text Generation

generate() with greedy, sampling (temperature/top-k/top-p), and beam search; chat templates, streaming, and static caches. See references/generation.md.

4. Training and Fine-Tuning

Trainer + TrainingArguments with mixed precision, gradient accumulation/checkpointing, callbacks, hyperparameter search, and LoRA via peft. See references/training.md.

5. Tokenization

Padding, truncation, special tokens, offsets, and chat templates. See references/tokenizers.md.

Common Patterns

Pattern 1: Simple Inference

pipe = pipeline("task-name", model="org/model-id")
output = pipe(input_data)

Pattern 2: Explicit Model + Tokenizer (chat model)

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Qwen/Qwen3-0.6B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")

messages = [{"role": "user", "content": "State the central limit theorem in one sentence."}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    enable_thinking=False,  # Qwen3-specific template variable
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=128)
new_tokens = outputs[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))

Pattern 3: Fine-Tuning a Classifier

from datasets import load_dataset
from transformers import (AutoModelForSequenceClassification, AutoTokenizer,
                          DataCollatorWithPadding, Trainer, TrainingArguments)

model_id = "google-bert/bert-base-uncased"  # or answerdotai/ModernBERT-base
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id, num_labels=5)

ds = load_dataset("Yelp/yelp_review_full")
ds = ds.map(lambda b: tokenizer(b["text"], truncation=True), batched=True)

args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=16,
    eval_strategy="epoch",
    warmup_steps=0.1,          # ratio of total steps
    report_to="tensorboard",   # default is "none" in v5
)
trainer = Trainer(
    model=model,
    args=args,
    train_dataset=ds["train"],
    eval_dataset=ds["test"],
    processing_class=tokenizer,
    data_collator=DataCollatorWithPadding(tokenizer),
)
trainer.train()

Practical Notes

  • Pin model IDs (and ideally revision=) in research code: pipeline defaults and Hub repos change, which silently changes results.
  • Report the exact checkpoint, transformers version, decoding parameters, and seed (transformers.set_seed) when generated text or fine-tuned metrics appear in a paper.
  • Check the model card's license and gating terms before redistributing weights or outputs.

Reference Documentation

  • references/pipelines.md — supported v5 tasks, parameters, batching, and removed pipelines
  • references/models.md — loading, dtype/device/attention/quantization, saving, export
  • references/generation.md — decoding strategies, chat templates, streaming, caches
  • references/training.md — Trainer workflow, TrainingArguments, PEFT, tuning
  • references/tokenizers.md — tokenization, special tokens, chat templates

Part of the AlterLab Academic Skills suite.

Files (alterlab-academic-skills)
  • evals
    • evals.json 7.6 KB
      Loading…
  • references
    • generation.md 10.2 KB
      Loading…
    • models.md 11.1 KB
      Loading…
    • pipelines.md 10.9 KB
      Loading…
    • tokenizers.md 11.3 KB
      Loading…
    • training.md 11.4 KB
      Loading…
  • SKILL.md 12.5 KB
    ---
    name: alterlab-transformers
    description: Loads, runs, and fine-tunes pretrained models with Hugging Face Transformers v5 (PyTorch-only) — pipeline() inference for chat-model text generation, text classification, NER, zero-shot, speech recognition, image classification, object detection, and image-text-to-text VLMs; AutoModel/AutoTokenizer loading with dtype, device_map and bitsandbytes quantization; generate() decoding control; and Trainer fine-tuning with optional PEFT/LoRA. Use when running inference with a Hugging Face Hub checkpoint, fine-tuning BERT/ModernBERT/Qwen-style models on a custom labelled dataset, controlling generation (sampling, beam search, streaming, chat templates), or porting v4 code (torch_dtype, removed summarization/translation/QA pipelines, TF/Flax) to v5. Part of the AlterLab Academic Skills suite.
    license: Apache-2.0
    allowed-tools: Read Write Edit Bash(python:*) Bash(uv:*)
    compatibility: No API key required for public models. Runs locally via `uv run --python ">=3.12" --with "transformers>=5" --with torch python script.py` (no install step; the interpreter pin matters because an older default Python resolves older releases); requires transformers >= 5.0 (current 5.17 as of 2026-09, Python >= 3.10) and PyTorch — v5 removed TensorFlow and Flax support. An optional Hugging Face token (HF_TOKEN) adds access to gated or private models.
    metadata:
        skill-author: AlterLab
        version: "1.1.1"
        last_updated: "2026-10-22"
        credentials: "optional: HF_TOKEN (access to gated or private Hub models and higher Hub download rate limits; public checkpoints download without it)"
    ---
    
    # Transformers
    
    ## Overview
    
    Hugging Face Transformers loads pretrained checkpoints from the Hub for NLP, vision, audio, and multimodal tasks, runs inference through `pipeline()` or the `Auto*` classes, and fine-tunes them with `Trainer`. This skill targets **transformers v5** (≥ 5.0; current 5.17.0 as of 2026-09). v5 is PyTorch-only and changes several v4 idioms, so check "v5 changes that break v4 code" below before reusing older snippets from papers, blogs, or model cards.
    
    ## When to Use This Skill
    
    Use this skill when the user wants to:
    - Run quick inference with a Hub checkpoint via `pipeline()` — text classification, NER, zero-shot classification, text generation with a chat model, speech recognition, image classification/detection, or image-text-to-text with a VLM.
    - Load a model plus tokenizer/processor with explicit `dtype`, `device_map`, attention backend, or 4/8-bit quantization.
    - Control decoding in `model.generate()` (greedy, sampling, beam search, streaming, chat templates).
    - Fine-tune an encoder or decoder on a custom labelled dataset with `Trainer`, optionally with LoRA via `peft`.
    - Port v4-era code (`torch_dtype=`, `load_in_8bit=`, removed pipelines, TF/Flax models) to v5.
    
    ### Does NOT Trigger
    
    | Scenario | Use Instead |
    |----------|-------------|
    | Classical ML on tabular features (random forests, preprocessing pipelines, CV grid search) | `alterlab-scikit-learn` |
    | Structuring a custom PyTorch architecture's training loop, multi-GPU strategy, and checkpointing with a LightningModule | `alterlab-pytorch-lightning` |
    | Topic models, embeddings, or text classifiers as a social-science measurement design (validity, BERTopic, dictionaries) | `alterlab-text-as-data` |
    | Protein language models (ESM3 / ESM C embeddings, protein design) | `alterlab-esm` |
    | Zero-shot time-series forecasting with a pretrained foundation model | `alterlab-timesfm` |
    
    **Hand-off rule.** When the request matches a row above, open your answer by naming that skill by its exact id (for example `alterlab-scikit-learn`) and saying in one line why it fits better, then stop applying this skill's procedure. Name it even when you also need to ask for missing input. If only part of the request is out of scope, name the skill for that part and do only the in-scope part. A brief general pointer is fine; the named skill does the work.
    
    ## Installation
    
    ```bash
    uv run --python ">=3.12" --with "transformers>=5" --with torch --with accelerate --with datasets python infer.py   # no install step
    # torch + transformers already importable (`python -c "import torch, transformers"`)? run with plain `python`. CPU-only machine: add --index https://download.pytorch.org/whl/cpu
    ```
    
    - Vision: add `pillow` and `torchvision` (the default image-processor backend; `timm` only for timm-backed models).
    - Audio: add `librosa soundfile`.
    - LoRA / quantization: `peft`, `bitsandbytes` (CUDA). Metrics: `evaluate` or plain scikit-learn metrics.
    
    ## Authentication
    
    Public checkpoints (everything in this skill's examples) download with no token or account. A Hub
    token only adds access to gated or private models and higher download rate limits. Either log in once:
    
    ```python
    from huggingface_hub import login
    login()  # or run `hf auth login` in a shell (huggingface-cli is deprecated)
    ```
    
    or export the variable the Hub client reads:
    
    ```bash
    export HF_TOKEN="your_token_here"
    ```
    
    Tokens: https://huggingface.co/settings/tokens. Pass `token=` (not the removed `use_auth_token=`) when you need it explicitly.
    
    ## Quick Start
    
    ```python
    from transformers import pipeline
    
    # Text generation with a small open chat model (CPU-friendly; add device_map="auto" on GPU)
    generator = pipeline("text-generation", model="Qwen/Qwen3-0.6B")
    messages = [{"role": "user", "content": "Explain p-hacking in two sentences."}]
    out = generator(
        messages,
        max_new_tokens=128,
        tokenizer_encode_kwargs={"enable_thinking": False},  # Qwen3 chat-template switch; omit for other models
    )
    print(out[0]["generated_text"][-1]["content"])
    
    # Text classification — pin the checkpoint; task defaults change between releases
    classifier = pipeline("text-classification", model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")
    classifier(["This movie was excellent!", "Terrible pacing."])
    
    # Zero-shot classification
    zero_shot = pipeline("zero-shot-classification", model="facebook/bart-large-mnli")
    zero_shot("The grant covers two postdoc salaries.", candidate_labels=["funding", "teaching", "ethics"])
    ```
    
    Use `max_new_tokens` (tokens to generate) rather than `max_length` (prompt + output, and in the text-generation pipeline it also feeds tokenizer truncation). Summarization, translation, and question answering no longer have dedicated pipelines in v5 — prompt a chat model as above.
    
    ## v5 Changes That Break v4 Code
    
    | v4 idiom | v5 replacement |
    |----------|----------------|
    | `torch_dtype=torch.float16` | `dtype=torch.float16` (`torch_dtype` only warns). Default is now `dtype="auto"` — the checkpoint's dtype, often bf16 — not float32 |
    | `TFAutoModel…` / `FlaxAutoModel…`, `framework="tf"`, `return_tensors="tf"` | Removed; PyTorch only (`return_tensors` accepts `"pt"`, `"np"`, `"mlx"`) |
    | `load_in_8bit=True` / `load_in_4bit=True` | `quantization_config=BitsAndBytesConfig(...)` |
    | `pipeline("summarization" / "translation_xx_to_yy" / "text2text-generation" / "question-answering")` | Removed — `pipeline("text-generation")` with a chat model and an instruction prompt |
    | `pipeline("image-to-text" / "visual-question-answering")` | `pipeline("image-text-to-text")` with a VLM, e.g. `Qwen/Qwen3-VL-2B-Instruct` |
    | `AutoModelForVision2Seq`, `AutoModelWithLMHead` | `AutoModelForImageTextToText`; `AutoModelForCausalLM` / `AutoModelForMaskedLM` / `AutoModelForSeq2SeqLM` |
    | `apply_chat_template(..., tokenize=True)` returned a tensor of ids | Returns a `BatchEncoding` (`input_ids`, `attention_mask`) → `model.generate(**inputs)` |
    | Slow vs fast tokenizers, `use_fast=` | One tokenizer per model on the 🤗 tokenizers backend; `AutoTokenizer` ignores `use_fast` |
    | `batch_decode`, `encode_plus`, `additional_special_tokens` | `decode` handles batches, `tokenizer(...)`, `extra_special_tokens` (old names kept for backward compatibility) |
    | `penalty_alpha` (contrastive search), `force_words_ids` / `constraints`, group beam search, DoLa | Moved to Hub `custom_generate` repos — e.g. `custom_generate="transformers-community/contrastive-search", trust_remote_code=True` |
    | `TrainingArguments(logging_dir=..., warmup_ratio=...)` | Both removed: set the `TENSORBOARD_LOGGING_DIR` env var; `warmup_steps=0.1` (a float < 1 is a ratio) |
    | `Trainer(tokenizer=...)` | `Trainer(processing_class=...)`; `report_to` now defaults to `"none"` |
    | `save_pretrained(safe_serialization=False)`, `use_auth_token=` | Always safetensors; `token=` |
    | `TRANSFORMERS_CACHE`, `low_cpu_mem_usage=True` | `HF_HOME` / `HF_HUB_CACHE`; low-memory loading is always on (flag ignored) |
    
    Source: the official [v5 migration guide](https://github.com/huggingface/transformers/blob/main/MIGRATION_GUIDE_V5.md), cross-checked against the 5.17.0 release.
    
    ## Core Capabilities
    
    ### 1. Pipelines for Quick Inference
    
    One call covers tokenization, the forward pass, and post-processing for text classification, NER, zero-shot, fill-mask, text generation, ASR, audio classification, image classification/segmentation, object detection, depth estimation, and image-text-to-text. Use for prototyping and batch inference without custom preprocessing. See `references/pipelines.md`.
    
    ### 2. Model Loading and Management
    
    `from_pretrained` with `dtype`, `device_map`, `attn_implementation` (SDPA by default), and `quantization_config`; saving, Hub upload, and ONNX export options. See `references/models.md`.
    
    ### 3. Text Generation
    
    `generate()` with greedy, sampling (temperature/top-k/top-p), and beam search; chat templates, streaming, and static caches. See `references/generation.md`.
    
    ### 4. Training and Fine-Tuning
    
    `Trainer` + `TrainingArguments` with mixed precision, gradient accumulation/checkpointing, callbacks, hyperparameter search, and LoRA via `peft`. See `references/training.md`.
    
    ### 5. Tokenization
    
    Padding, truncation, special tokens, offsets, and chat templates. See `references/tokenizers.md`.
    
    ## Common Patterns
    
    ### Pattern 1: Simple Inference
    
    ```python
    pipe = pipeline("task-name", model="org/model-id")
    output = pipe(input_data)
    ```
    
    ### Pattern 2: Explicit Model + Tokenizer (chat model)
    
    ```python
    from transformers import AutoModelForCausalLM, AutoTokenizer
    
    model_id = "Qwen/Qwen3-0.6B"
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")
    
    messages = [{"role": "user", "content": "State the central limit theorem in one sentence."}]
    inputs = tokenizer.apply_chat_template(
        messages,
        add_generation_prompt=True,
        return_tensors="pt",
        enable_thinking=False,  # Qwen3-specific template variable
    ).to(model.device)
    
    outputs = model.generate(**inputs, max_new_tokens=128)
    new_tokens = outputs[0, inputs["input_ids"].shape[1]:]
    print(tokenizer.decode(new_tokens, skip_special_tokens=True))
    ```
    
    ### Pattern 3: Fine-Tuning a Classifier
    
    ```python
    from datasets import load_dataset
    from transformers import (AutoModelForSequenceClassification, AutoTokenizer,
                              DataCollatorWithPadding, Trainer, TrainingArguments)
    
    model_id = "google-bert/bert-base-uncased"  # or answerdotai/ModernBERT-base
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForSequenceClassification.from_pretrained(model_id, num_labels=5)
    
    ds = load_dataset("Yelp/yelp_review_full")
    ds = ds.map(lambda b: tokenizer(b["text"], truncation=True), batched=True)
    
    args = TrainingArguments(
        output_dir="./results",
        num_train_epochs=3,
        per_device_train_batch_size=16,
        eval_strategy="epoch",
        warmup_steps=0.1,          # ratio of total steps
        report_to="tensorboard",   # default is "none" in v5
    )
    trainer = Trainer(
        model=model,
        args=args,
        train_dataset=ds["train"],
        eval_dataset=ds["test"],
        processing_class=tokenizer,
        data_collator=DataCollatorWithPadding(tokenizer),
    )
    trainer.train()
    ```
    
    ## Practical Notes
    
    - Pin model IDs (and ideally `revision=`) in research code: pipeline defaults and Hub repos change, which silently changes results.
    - Report the exact checkpoint, transformers version, decoding parameters, and seed (`transformers.set_seed`) when generated text or fine-tuned metrics appear in a paper.
    - Check the model card's license and gating terms before redistributing weights or outputs.
    
    ## Reference Documentation
    
    - `references/pipelines.md` — supported v5 tasks, parameters, batching, and removed pipelines
    - `references/models.md` — loading, dtype/device/attention/quantization, saving, export
    - `references/generation.md` — decoding strategies, chat templates, streaming, caches
    - `references/training.md` — `Trainer` workflow, `TrainingArguments`, PEFT, tuning
    - `references/tokenizers.md` — tokenization, special tokens, chat templates
    
    Part of the AlterLab Academic Skills suite.
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.