alterlab-transformers
Loads, runs, and fine-tunes pretrained models with Hugging Face Transformers v5 (PyTorch-only) — pipeline() inference for chat-model text generation, text classification, NER, zero-shot, speech recognition, image classification, object detection, and image-text-to-text VLMs; Auto
Install
npx skills add https://github.com/AlterLab-IEU/AlterLab-Academic-Skills/tree/main/skills/data-science/alterlab-transformers
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install alterlab-ieu-alterlab-academic-skills@llmmart
git clone https://github.com/AlterLab-IEU/AlterLab-Academic-Skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole alterlab-ieu/alterlab-academic-skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Transformers
Overview
Hugging Face Transformers loads pretrained checkpoints from the Hub for NLP, vision, audio, and multimodal tasks, runs inference through pipeline() or the Auto* classes, and fine-tunes them with Trainer. This skill targets transformers v5 (≥ 5.0; current 5.17.0 as of 2026-09). v5 is PyTorch-only and changes several v4 idioms, so check "v5 changes that break v4 code" below before reusing older snippets from papers, blogs, or model cards.
When to Use This Skill
Use this skill when the user wants to:
- Run quick inference with a Hub checkpoint via
pipeline()— text classification, NER, zero-shot classification, text generation with a chat model, speech recognition, image classification/detection, or image-text-to-text with a VLM. - Load a model plus tokenizer/processor with explicit
dtype,device_map, attention backend, or 4/8-bit quantization. - Control decoding in
model.generate()(greedy, sampling, beam search, streaming, chat templates). - Fine-tune an encoder or decoder on a custom labelled dataset with
Trainer, optionally with LoRA viapeft. - Port v4-era code (
torch_dtype=,load_in_8bit=, removed pipelines, TF/Flax models) to v5.
Does NOT Trigger
| Scenario | Use Instead |
|---|---|
| Classical ML on tabular features (random forests, preprocessing pipelines, CV grid search) | alterlab-scikit-learn |
| Structuring a custom PyTorch architecture's training loop, multi-GPU strategy, and checkpointing with a LightningModule | alterlab-pytorch-lightning |
| Topic models, embeddings, or text classifiers as a social-science measurement design (validity, BERTopic, dictionaries) | alterlab-text-as-data |
| Protein language models (ESM3 / ESM C embeddings, protein design) | alterlab-esm |
| Zero-shot time-series forecasting with a pretrained foundation model | alterlab-timesfm |
Hand-off rule. When the request matches a row above, open your answer by naming that skill by its exact id (for example alterlab-scikit-learn) and saying in one line why it fits better, then stop applying this skill's procedure. Name it even when you also need to ask for missing input. If only part of the request is out of scope, name the skill for that part and do only the in-scope part. A brief general pointer is fine; the named skill does the work.
Installation
uv run --python ">=3.12" --with "transformers>=5" --with torch --with accelerate --with datasets python infer.py # no install step
# torch + transformers already importable (`python -c "import torch, transformers"`)? run with plain `python`. CPU-only machine: add --index https://download.pytorch.org/whl/cpu
- Vision: add
pillowandtorchvision(the default image-processor backend;timmonly for timm-backed models). - Audio: add
librosa soundfile. - LoRA / quantization:
peft,bitsandbytes(CUDA). Metrics:evaluateor plain scikit-learn metrics.
Authentication
Public checkpoints (everything in this skill's examples) download with no token or account. A Hub token only adds access to gated or private models and higher download rate limits. Either log in once:
from huggingface_hub import login
login() # or run `hf auth login` in a shell (huggingface-cli is deprecated)
or export the variable the Hub client reads:
export HF_TOKEN="your_token_here"
Tokens: https://huggingface.co/settings/tokens. Pass token= (not the removed use_auth_token=) when you need it explicitly.
Quick Start
from transformers import pipeline
# Text generation with a small open chat model (CPU-friendly; add device_map="auto" on GPU)
generator = pipeline("text-generation", model="Qwen/Qwen3-0.6B")
messages = [{"role": "user", "content": "Explain p-hacking in two sentences."}]
out = generator(
messages,
max_new_tokens=128,
tokenizer_encode_kwargs={"enable_thinking": False}, # Qwen3 chat-template switch; omit for other models
)
print(out[0]["generated_text"][-1]["content"])
# Text classification — pin the checkpoint; task defaults change between releases
classifier = pipeline("text-classification", model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")
classifier(["This movie was excellent!", "Terrible pacing."])
# Zero-shot classification
zero_shot = pipeline("zero-shot-classification", model="facebook/bart-large-mnli")
zero_shot("The grant covers two postdoc salaries.", candidate_labels=["funding", "teaching", "ethics"])
Use max_new_tokens (tokens to generate) rather than max_length (prompt + output, and in the text-generation pipeline it also feeds tokenizer truncation). Summarization, translation, and question answering no longer have dedicated pipelines in v5 — prompt a chat model as above.
v5 Changes That Break v4 Code
| v4 idiom | v5 replacement |
|---|---|
torch_dtype=torch.float16 |
dtype=torch.float16 (torch_dtype only warns). Default is now dtype="auto" — the checkpoint's dtype, often bf16 — not float32 |
TFAutoModel… / FlaxAutoModel…, framework="tf", return_tensors="tf" |
Removed; PyTorch only (return_tensors accepts "pt", "np", "mlx") |
load_in_8bit=True / load_in_4bit=True |
quantization_config=BitsAndBytesConfig(...) |
pipeline("summarization" / "translation_xx_to_yy" / "text2text-generation" / "question-answering") |
Removed — pipeline("text-generation") with a chat model and an instruction prompt |
pipeline("image-to-text" / "visual-question-answering") |
pipeline("image-text-to-text") with a VLM, e.g. Qwen/Qwen3-VL-2B-Instruct |
AutoModelForVision2Seq, AutoModelWithLMHead |
AutoModelForImageTextToText; AutoModelForCausalLM / AutoModelForMaskedLM / AutoModelForSeq2SeqLM |
apply_chat_template(..., tokenize=True) returned a tensor of ids |
Returns a BatchEncoding (input_ids, attention_mask) → model.generate(**inputs) |
Slow vs fast tokenizers, use_fast= |
One tokenizer per model on the 🤗 tokenizers backend; AutoTokenizer ignores use_fast |
batch_decode, encode_plus, additional_special_tokens |
decode handles batches, tokenizer(...), extra_special_tokens (old names kept for backward compatibility) |
penalty_alpha (contrastive search), force_words_ids / constraints, group beam search, DoLa |
Moved to Hub custom_generate repos — e.g. custom_generate="transformers-community/contrastive-search", trust_remote_code=True |
TrainingArguments(logging_dir=..., warmup_ratio=...) |
Both removed: set the TENSORBOARD_LOGGING_DIR env var; warmup_steps=0.1 (a float < 1 is a ratio) |
Trainer(tokenizer=...) |
Trainer(processing_class=...); report_to now defaults to "none" |
save_pretrained(safe_serialization=False), use_auth_token= |
Always safetensors; token= |
TRANSFORMERS_CACHE, low_cpu_mem_usage=True |
HF_HOME / HF_HUB_CACHE; low-memory loading is always on (flag ignored) |
Source: the official v5 migration guide, cross-checked against the 5.17.0 release.
Core Capabilities
1. Pipelines for Quick Inference
One call covers tokenization, the forward pass, and post-processing for text classification, NER, zero-shot, fill-mask, text generation, ASR, audio classification, image classification/segmentation, object detection, depth estimation, and image-text-to-text. Use for prototyping and batch inference without custom preprocessing. See references/pipelines.md.
2. Model Loading and Management
from_pretrained with dtype, device_map, attn_implementation (SDPA by default), and quantization_config; saving, Hub upload, and ONNX export options. See references/models.md.
3. Text Generation
generate() with greedy, sampling (temperature/top-k/top-p), and beam search; chat templates, streaming, and static caches. See references/generation.md.
4. Training and Fine-Tuning
Trainer + TrainingArguments with mixed precision, gradient accumulation/checkpointing, callbacks, hyperparameter search, and LoRA via peft. See references/training.md.
5. Tokenization
Padding, truncation, special tokens, offsets, and chat templates. See references/tokenizers.md.
Common Patterns
Pattern 1: Simple Inference
pipe = pipeline("task-name", model="org/model-id")
output = pipe(input_data)
Pattern 2: Explicit Model + Tokenizer (chat model)
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Qwen/Qwen3-0.6B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "State the central limit theorem in one sentence."}]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt",
enable_thinking=False, # Qwen3-specific template variable
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128)
new_tokens = outputs[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))
Pattern 3: Fine-Tuning a Classifier
from datasets import load_dataset
from transformers import (AutoModelForSequenceClassification, AutoTokenizer,
DataCollatorWithPadding, Trainer, TrainingArguments)
model_id = "google-bert/bert-base-uncased" # or answerdotai/ModernBERT-base
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id, num_labels=5)
ds = load_dataset("Yelp/yelp_review_full")
ds = ds.map(lambda b: tokenizer(b["text"], truncation=True), batched=True)
args = TrainingArguments(
output_dir="./results",
num_train_epochs=3,
per_device_train_batch_size=16,
eval_strategy="epoch",
warmup_steps=0.1, # ratio of total steps
report_to="tensorboard", # default is "none" in v5
)
trainer = Trainer(
model=model,
args=args,
train_dataset=ds["train"],
eval_dataset=ds["test"],
processing_class=tokenizer,
data_collator=DataCollatorWithPadding(tokenizer),
)
trainer.train()
Practical Notes
- Pin model IDs (and ideally
revision=) in research code: pipeline defaults and Hub repos change, which silently changes results. - Report the exact checkpoint, transformers version, decoding parameters, and seed (
transformers.set_seed) when generated text or fine-tuned metrics appear in a paper. - Check the model card's license and gating terms before redistributing weights or outputs.
Reference Documentation
references/pipelines.md— supported v5 tasks, parameters, batching, and removed pipelinesreferences/models.md— loading, dtype/device/attention/quantization, saving, exportreferences/generation.md— decoding strategies, chat templates, streaming, cachesreferences/training.md—Trainerworkflow,TrainingArguments, PEFT, tuningreferences/tokenizers.md— tokenization, special tokens, chat templates
Part of the AlterLab Academic Skills suite.
Files (alterlab-academic-skills)
-
evals
-
evals.json 7.6 KB
Loading…
-
-
references
-
generation.md 10.2 KB
Loading… -
models.md 11.1 KB
Loading… -
pipelines.md 10.9 KB
Loading… -
tokenizers.md 11.3 KB
Loading… -
training.md 11.4 KB
Loading…
-
-
SKILL.md 12.5 KB
--- name: alterlab-transformers description: Loads, runs, and fine-tunes pretrained models with Hugging Face Transformers v5 (PyTorch-only) — pipeline() inference for chat-model text generation, text classification, NER, zero-shot, speech recognition, image classification, object detection, and image-text-to-text VLMs; AutoModel/AutoTokenizer loading with dtype, device_map and bitsandbytes quantization; generate() decoding control; and Trainer fine-tuning with optional PEFT/LoRA. Use when running inference with a Hugging Face Hub checkpoint, fine-tuning BERT/ModernBERT/Qwen-style models on a custom labelled dataset, controlling generation (sampling, beam search, streaming, chat templates), or porting v4 code (torch_dtype, removed summarization/translation/QA pipelines, TF/Flax) to v5. Part of the AlterLab Academic Skills suite. license: Apache-2.0 allowed-tools: Read Write Edit Bash(python:*) Bash(uv:*) compatibility: No API key required for public models. Runs locally via `uv run --python ">=3.12" --with "transformers>=5" --with torch python script.py` (no install step; the interpreter pin matters because an older default Python resolves older releases); requires transformers >= 5.0 (current 5.17 as of 2026-09, Python >= 3.10) and PyTorch — v5 removed TensorFlow and Flax support. An optional Hugging Face token (HF_TOKEN) adds access to gated or private models. metadata: skill-author: AlterLab version: "1.1.1" last_updated: "2026-10-22" credentials: "optional: HF_TOKEN (access to gated or private Hub models and higher Hub download rate limits; public checkpoints download without it)" --- # Transformers ## Overview Hugging Face Transformers loads pretrained checkpoints from the Hub for NLP, vision, audio, and multimodal tasks, runs inference through `pipeline()` or the `Auto*` classes, and fine-tunes them with `Trainer`. This skill targets **transformers v5** (≥ 5.0; current 5.17.0 as of 2026-09). v5 is PyTorch-only and changes several v4 idioms, so check "v5 changes that break v4 code" below before reusing older snippets from papers, blogs, or model cards. ## When to Use This Skill Use this skill when the user wants to: - Run quick inference with a Hub checkpoint via `pipeline()` — text classification, NER, zero-shot classification, text generation with a chat model, speech recognition, image classification/detection, or image-text-to-text with a VLM. - Load a model plus tokenizer/processor with explicit `dtype`, `device_map`, attention backend, or 4/8-bit quantization. - Control decoding in `model.generate()` (greedy, sampling, beam search, streaming, chat templates). - Fine-tune an encoder or decoder on a custom labelled dataset with `Trainer`, optionally with LoRA via `peft`. - Port v4-era code (`torch_dtype=`, `load_in_8bit=`, removed pipelines, TF/Flax models) to v5. ### Does NOT Trigger | Scenario | Use Instead | |----------|-------------| | Classical ML on tabular features (random forests, preprocessing pipelines, CV grid search) | `alterlab-scikit-learn` | | Structuring a custom PyTorch architecture's training loop, multi-GPU strategy, and checkpointing with a LightningModule | `alterlab-pytorch-lightning` | | Topic models, embeddings, or text classifiers as a social-science measurement design (validity, BERTopic, dictionaries) | `alterlab-text-as-data` | | Protein language models (ESM3 / ESM C embeddings, protein design) | `alterlab-esm` | | Zero-shot time-series forecasting with a pretrained foundation model | `alterlab-timesfm` | **Hand-off rule.** When the request matches a row above, open your answer by naming that skill by its exact id (for example `alterlab-scikit-learn`) and saying in one line why it fits better, then stop applying this skill's procedure. Name it even when you also need to ask for missing input. If only part of the request is out of scope, name the skill for that part and do only the in-scope part. A brief general pointer is fine; the named skill does the work. ## Installation ```bash uv run --python ">=3.12" --with "transformers>=5" --with torch --with accelerate --with datasets python infer.py # no install step # torch + transformers already importable (`python -c "import torch, transformers"`)? run with plain `python`. CPU-only machine: add --index https://download.pytorch.org/whl/cpu ``` - Vision: add `pillow` and `torchvision` (the default image-processor backend; `timm` only for timm-backed models). - Audio: add `librosa soundfile`. - LoRA / quantization: `peft`, `bitsandbytes` (CUDA). Metrics: `evaluate` or plain scikit-learn metrics. ## Authentication Public checkpoints (everything in this skill's examples) download with no token or account. A Hub token only adds access to gated or private models and higher download rate limits. Either log in once: ```python from huggingface_hub import login login() # or run `hf auth login` in a shell (huggingface-cli is deprecated) ``` or export the variable the Hub client reads: ```bash export HF_TOKEN="your_token_here" ``` Tokens: https://huggingface.co/settings/tokens. Pass `token=` (not the removed `use_auth_token=`) when you need it explicitly. ## Quick Start ```python from transformers import pipeline # Text generation with a small open chat model (CPU-friendly; add device_map="auto" on GPU) generator = pipeline("text-generation", model="Qwen/Qwen3-0.6B") messages = [{"role": "user", "content": "Explain p-hacking in two sentences."}] out = generator( messages, max_new_tokens=128, tokenizer_encode_kwargs={"enable_thinking": False}, # Qwen3 chat-template switch; omit for other models ) print(out[0]["generated_text"][-1]["content"]) # Text classification — pin the checkpoint; task defaults change between releases classifier = pipeline("text-classification", model="distilbert/distilbert-base-uncased-finetuned-sst-2-english") classifier(["This movie was excellent!", "Terrible pacing."]) # Zero-shot classification zero_shot = pipeline("zero-shot-classification", model="facebook/bart-large-mnli") zero_shot("The grant covers two postdoc salaries.", candidate_labels=["funding", "teaching", "ethics"]) ``` Use `max_new_tokens` (tokens to generate) rather than `max_length` (prompt + output, and in the text-generation pipeline it also feeds tokenizer truncation). Summarization, translation, and question answering no longer have dedicated pipelines in v5 — prompt a chat model as above. ## v5 Changes That Break v4 Code | v4 idiom | v5 replacement | |----------|----------------| | `torch_dtype=torch.float16` | `dtype=torch.float16` (`torch_dtype` only warns). Default is now `dtype="auto"` — the checkpoint's dtype, often bf16 — not float32 | | `TFAutoModel…` / `FlaxAutoModel…`, `framework="tf"`, `return_tensors="tf"` | Removed; PyTorch only (`return_tensors` accepts `"pt"`, `"np"`, `"mlx"`) | | `load_in_8bit=True` / `load_in_4bit=True` | `quantization_config=BitsAndBytesConfig(...)` | | `pipeline("summarization" / "translation_xx_to_yy" / "text2text-generation" / "question-answering")` | Removed — `pipeline("text-generation")` with a chat model and an instruction prompt | | `pipeline("image-to-text" / "visual-question-answering")` | `pipeline("image-text-to-text")` with a VLM, e.g. `Qwen/Qwen3-VL-2B-Instruct` | | `AutoModelForVision2Seq`, `AutoModelWithLMHead` | `AutoModelForImageTextToText`; `AutoModelForCausalLM` / `AutoModelForMaskedLM` / `AutoModelForSeq2SeqLM` | | `apply_chat_template(..., tokenize=True)` returned a tensor of ids | Returns a `BatchEncoding` (`input_ids`, `attention_mask`) → `model.generate(**inputs)` | | Slow vs fast tokenizers, `use_fast=` | One tokenizer per model on the 🤗 tokenizers backend; `AutoTokenizer` ignores `use_fast` | | `batch_decode`, `encode_plus`, `additional_special_tokens` | `decode` handles batches, `tokenizer(...)`, `extra_special_tokens` (old names kept for backward compatibility) | | `penalty_alpha` (contrastive search), `force_words_ids` / `constraints`, group beam search, DoLa | Moved to Hub `custom_generate` repos — e.g. `custom_generate="transformers-community/contrastive-search", trust_remote_code=True` | | `TrainingArguments(logging_dir=..., warmup_ratio=...)` | Both removed: set the `TENSORBOARD_LOGGING_DIR` env var; `warmup_steps=0.1` (a float < 1 is a ratio) | | `Trainer(tokenizer=...)` | `Trainer(processing_class=...)`; `report_to` now defaults to `"none"` | | `save_pretrained(safe_serialization=False)`, `use_auth_token=` | Always safetensors; `token=` | | `TRANSFORMERS_CACHE`, `low_cpu_mem_usage=True` | `HF_HOME` / `HF_HUB_CACHE`; low-memory loading is always on (flag ignored) | Source: the official [v5 migration guide](https://github.com/huggingface/transformers/blob/main/MIGRATION_GUIDE_V5.md), cross-checked against the 5.17.0 release. ## Core Capabilities ### 1. Pipelines for Quick Inference One call covers tokenization, the forward pass, and post-processing for text classification, NER, zero-shot, fill-mask, text generation, ASR, audio classification, image classification/segmentation, object detection, depth estimation, and image-text-to-text. Use for prototyping and batch inference without custom preprocessing. See `references/pipelines.md`. ### 2. Model Loading and Management `from_pretrained` with `dtype`, `device_map`, `attn_implementation` (SDPA by default), and `quantization_config`; saving, Hub upload, and ONNX export options. See `references/models.md`. ### 3. Text Generation `generate()` with greedy, sampling (temperature/top-k/top-p), and beam search; chat templates, streaming, and static caches. See `references/generation.md`. ### 4. Training and Fine-Tuning `Trainer` + `TrainingArguments` with mixed precision, gradient accumulation/checkpointing, callbacks, hyperparameter search, and LoRA via `peft`. See `references/training.md`. ### 5. Tokenization Padding, truncation, special tokens, offsets, and chat templates. See `references/tokenizers.md`. ## Common Patterns ### Pattern 1: Simple Inference ```python pipe = pipeline("task-name", model="org/model-id") output = pipe(input_data) ``` ### Pattern 2: Explicit Model + Tokenizer (chat model) ```python from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Qwen/Qwen3-0.6B" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto") messages = [{"role": "user", "content": "State the central limit theorem in one sentence."}] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_tensors="pt", enable_thinking=False, # Qwen3-specific template variable ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=128) new_tokens = outputs[0, inputs["input_ids"].shape[1]:] print(tokenizer.decode(new_tokens, skip_special_tokens=True)) ``` ### Pattern 3: Fine-Tuning a Classifier ```python from datasets import load_dataset from transformers import (AutoModelForSequenceClassification, AutoTokenizer, DataCollatorWithPadding, Trainer, TrainingArguments) model_id = "google-bert/bert-base-uncased" # or answerdotai/ModernBERT-base tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForSequenceClassification.from_pretrained(model_id, num_labels=5) ds = load_dataset("Yelp/yelp_review_full") ds = ds.map(lambda b: tokenizer(b["text"], truncation=True), batched=True) args = TrainingArguments( output_dir="./results", num_train_epochs=3, per_device_train_batch_size=16, eval_strategy="epoch", warmup_steps=0.1, # ratio of total steps report_to="tensorboard", # default is "none" in v5 ) trainer = Trainer( model=model, args=args, train_dataset=ds["train"], eval_dataset=ds["test"], processing_class=tokenizer, data_collator=DataCollatorWithPadding(tokenizer), ) trainer.train() ``` ## Practical Notes - Pin model IDs (and ideally `revision=`) in research code: pipeline defaults and Hub repos change, which silently changes results. - Report the exact checkpoint, transformers version, decoding parameters, and seed (`transformers.set_seed`) when generated text or fine-tuned metrics appear in a paper. - Check the model card's license and gating terms before redistributing weights or outputs. ## Reference Documentation - `references/pipelines.md` — supported v5 tasks, parameters, batching, and removed pipelines - `references/models.md` — loading, dtype/device/attention/quantization, saving, export - `references/generation.md` — decoding strategies, chat templates, streaming, caches - `references/training.md` — `Trainer` workflow, `TrainingArguments`, PEFT, tuning - `references/tokenizers.md` — tokenization, special tokens, chat templates Part of the AlterLab Academic Skills suite.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.