infra-ci-cd-turborepo-ci
Turborepo CI pipelines with remote caching and affected detection
Install
npx skills add https://github.com/agents-inc/skills/tree/main/dist/plugins/infra-ci-cd-turborepo-ci/skills/infra-ci-cd-turborepo-ci
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install agents-inc-skills@llmmart
git clone https://github.com/agents-inc/skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole agents-inc/skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Turborepo CI Patterns
Quick Guide: Use
--affectedfor PR builds (auto-detects CI environment, falls back to full suite on shallow clones). Enable Remote Cache withTURBO_TOKEN+TURBO_TEAMenv vars. Declareoutputsfor every cacheable task or cached results will be incomplete. UseenvandglobalEnvin turbo.json to include environment variables in cache hashes -- missing entries cause cross-environment cache collisions. Pinturboversion in CI. Useturbo query affectedto conditionally skip entire CI steps.
<critical_requirements>
CRITICAL: Before Using This Skill
All code must follow project conventions in CLAUDE.md (kebab-case, named exports, import ordering,
import type, named constants)
(You MUST declare outputs for every cacheable task in turbo.json -- missing outputs means cached results restore without build artifacts)
(You MUST list environment variables that affect task output in env (task-level) or globalEnv (all tasks) -- omitting them causes cross-environment cache hits with wrong values)
(You MUST use --affected for PR builds -- running the full task graph on PRs wastes CI time on unchanged packages)
(You MUST pin the turbo CLI version in CI -- latest can introduce breaking changes mid-pipeline)
</critical_requirements>
Detailed Resources:
- examples/core.md - turbo.json task config, outputs, env, caching control, filter syntax
- examples/remote-cache.md - Remote Cache setup, self-hosted options, signature verification
- examples/affected-detection.md - --affected flag, turbo query affected, conditional CI steps
- examples/docker.md - turbo prune --docker, multi-stage Dockerfile, layer caching
- reference.md - Decision frameworks, CLI flags, turbo.json quick reference
Auto-detection: Turborepo CI, turbo.json, turbo run, turbo prune, --affected, --filter, TURBO_TOKEN, TURBO_TEAM, Remote Cache, turbo query affected, outputs, dependsOn, globalEnv, envMode, concurrency, turbo login, turbo link, cache artifacts, monorepo CI
When to use:
- Configuring CI pipelines for a Turborepo monorepo
- Setting up Remote Cache for shared build artifacts across CI and local
- Using
--affectedto run only changed-package tasks on PRs - Optimizing Docker builds with
turbo prune --docker - Debugging cache misses with
--summarizeor--dry - Skipping CI steps conditionally with
turbo query affected
When NOT to use:
- General Turborepo workspace setup (package structure, task graph design) -- that belongs in a monorepo/workspace skill
- CI provider-specific workflow syntax (use your CI provider's skill)
- Application build configuration (bundler, compiler settings)
Key patterns covered:
- turbo.json task configuration (
outputs,env,dependsOn,cache,inputs) - Remote Cache authentication and setup (
TURBO_TOKEN,TURBO_TEAM, signature verification) - Affected detection (
--affected,--filter=...[origin/main],turbo query affected) - Docker optimization with
turbo prune --dockerand multi-stage builds - Cache debugging (
--summarize,--dry,--force) - Environment variable modes (
strictvsloose) andpassThroughEnv - Concurrency control and output log filtering
<decision_framework>
Decision Framework
When to use --affected vs --filter?
Running a PR build?
|-- YES --> Use --affected (auto-detects CI env, graceful shallow clone fallback)
+-- NO --> Need specific package selection?
|-- YES --> Use --filter (explicit package/directory/git targeting)
+-- NO --> Run full task graph (main branch, release builds)
When to enable Remote Cache?
Team > 1 person OR using CI?
|-- YES --> Enable Remote Cache (shared artifacts save significant time)
| |-- Using Vercel for hosting?
| | |-- YES --> Free, auto-configured on Vercel deployments
| | +-- NO --> Set TURBO_TOKEN + TURBO_TEAM as CI secrets
| +-- Need artifact signing?
| |-- YES --> Enable signature verification in turbo.json
| +-- NO --> Default (unsigned) is fine for trusted environments
+-- NO --> Local cache sufficient (solo developer, single machine)
env vs globalEnv vs passThroughEnv?
Does the variable affect task output (build artifacts, test results)?
|-- YES --> Does it affect ALL tasks or just one?
| |-- ALL tasks --> globalEnv (changes bust cache for everything)
| +-- One task --> env on that specific task
+-- NO --> Does the task need it at runtime?
|-- YES --> passThroughEnv (available but not in hash)
+-- NO --> Don't list it (strict mode blocks it)
When to use turbo prune --docker?
Building Docker images from monorepo?
|-- YES --> Does your Dockerfile install from root lockfile?
| |-- YES --> Use turbo prune --docker (pruned lockfile = better layer caching)
| +-- NO --> Standard turbo prune (no --docker flag needed)
+-- NO --> Not applicable
</decision_framework>
<red_flags>
RED FLAGS
High Priority:
- Missing
outputson cacheable tasks -- cached runs restore nothing, tasks appear to succeed but produce no artifacts - Missing
envfor environment-dependent tasks -- build withAPI_URL=staginghits cache fromAPI_URL=production, serving wrong config - Using
latestturbo version in CI -- non-reproducible builds, potential breaking changes mid-pipeline - Shallow clones without fallback --
--filter=...[origin/main]fails when git history is insufficient;--affectedhandles this gracefully by falling back to full suite
Medium Priority:
- All environment variables in
globalEnv-- every change busts cache for ALL tasks; use task-levelenvfor task-specific variables - No
--affectedon PR builds -- full task graph on PRs wastes CI time rebuilding unchanged packages - Missing signature verification on shared Remote Cache -- unsigned artifacts can be tampered with in shared environments
- Not using
turbo runexplicitly in CI -- bareturbo buildmay conflict with future CLI subcommands
Common Mistakes:
- Forgetting
dependsOn: ["^build"]for tasks that consume upstream package outputs (test/lint fail because dependency not built) - Using
looseenv mode and wondering why cache hits serve wrong environment config - Not including
globalDependenciesfor files like.envortsconfig.base.jsonthat affect all packages - Setting
cache: falseon tasks that should be cached (e.g., test, lint) because of past debugging and forgetting to re-enable
Gotchas & Edge Cases:
--affectedauto-detects GitHub Actions viaGITHUB_BASE_REF-- other CI providers may need manual--filterinsteadturbo query affectedreturns JSON -- parse withjqfor conditional CI stepsoutputsglobs are relative to the package directory, not the repo root$TURBO_DEFAULT$ininputsrestores the default input behavior when you only want to add inputs, not replace them- Remote Cache
signature: truerequiresTURBO_REMOTE_CACHE_SIGNATURE_KEY-- missing key means cache reads fail silently (treated as misses) --summarizeoutput goes to.turbo/runs/-- useful for diffing hashes between two runs to find what changed- Circular package dependencies are allowed since v2.9 (validated at task graph level, not package graph level)
--parallelflag is deprecated -- usepersistent: truein turbo.json for long-running tasks insteadturbo-ignoreis deprecated -- useturbo query affectedinstead for conditional CI stepsremoteCache.timeoutdefault is 30s,uploadTimeoutis 60s -- increase for large monorepo artifacts
</red_flags>
<critical_reminders>
CRITICAL REMINDERS
All code must follow project conventions in CLAUDE.md
(You MUST declare outputs for every cacheable task in turbo.json -- missing outputs means cached results restore without build artifacts)
(You MUST list environment variables that affect task output in env (task-level) or globalEnv (all tasks) -- omitting them causes cross-environment cache hits with wrong values)
(You MUST use --affected for PR builds -- running the full task graph on PRs wastes CI time on unchanged packages)
(You MUST pin the turbo CLI version in CI -- latest can introduce breaking changes mid-pipeline)
Failure to follow these rules will cause incorrect cache hits (wrong build artifacts served), slow CI (full rebuilds on every PR), and non-reproducible pipelines.
</critical_reminders>
Files (skills)
-
examples
-
affected-detection.md 5 KB
# Turborepo CI - Affected Detection Examples > --affected flag, turbo query affected for conditional CI steps, and PR vs main branch patterns. See [SKILL.md](../SKILL.md) for core concepts and [reference.md](../reference.md) for filter syntax cheat sheet. --- ## Pattern 1: --affected for PR Builds ### Basic Usage ```bash # Auto-detects CI environment (GitHub Actions, GitLab CI, etc.) # Compares current branch changes against default branch turbo run build test lint --affected ``` **How --affected works in CI:** 1. Detects CI provider via environment variables (e.g., `GITHUB_BASE_REF` for GitHub Actions) 2. Determines comparison base (PR base branch or push event's before SHA) 3. Identifies packages with file changes between base and HEAD 4. Runs tasks only in those packages (and their dependents if tasks have `dependsOn: ["^..."]`) **Shallow clone handling:** If git history is insufficient for comparison, `--affected` gracefully falls back to running all tasks. This is safer than `--filter=...[origin/main]` which may fail or produce incorrect results on shallow clones. ### Manual Comparison with --filter ```bash # Explicit git range comparison (requires sufficient clone depth) turbo run test --filter=...[origin/main] # Changed packages in a specific directory turbo run test --filter={./apps/*}[origin/main] # Changed packages since last commit turbo run lint --filter=...[HEAD^1] ``` --- ## Pattern 2: PR vs Main Branch Strategy ### PR Builds: Fast Feedback ```bash # Only changed packages -- target: < 3 minutes turbo run lint type-check test --affected ``` ### Main Branch: Full Validation ```bash # Full task graph -- catches integration issues turbo run lint type-check test build ``` ### CI Script Pattern ```bash #!/usr/bin/env bash set -euo pipefail if [ "${GITHUB_EVENT_NAME:-}" = "pull_request" ]; then echo "PR build: running affected tasks only" turbo run lint type-check test --affected else echo "Main branch: running full task graph" turbo run lint type-check test build fi ``` **Why separate strategies:** PRs need fast feedback (developer is waiting). Main branch needs comprehensive validation (catching cross-package integration issues that affected detection might miss). --- ## Pattern 3: turbo query affected for Conditional Steps `turbo query affected` outputs structured JSON, enabling conditional CI steps that skip expensive operations when packages aren't affected. ### Check if Specific Package is Affected ```bash # Check if 'web' app is affected (any task) AFFECTED_COUNT=$(turbo query affected --packages web \ | jq '.data.affectedPackages.length') if [ "$AFFECTED_COUNT" -gt 0 ]; then echo "web app affected -- running deployment" # Run expensive deployment steps else echo "web app not affected -- skipping deployment" fi ``` ### Check if Specific Task is Affected ```bash # Check if any build tasks are affected turbo query affected --tasks build # Check if any test tasks are affected turbo query affected --tasks test ``` ### Skip Dependency Installation ```bash # If no packages are affected, skip the entire CI pipeline TOTAL_AFFECTED=$(turbo query affected --packages \ | jq '.data.affectedPackages.length') if [ "$TOTAL_AFFECTED" -eq 0 ]; then echo "No packages affected -- skipping CI" exit 0 fi # Proceed with install and task execution npm install --frozen-lockfile turbo run build test lint --affected ``` **Why useful:** Dependency installation itself takes 30-90 seconds. If `turbo query affected` shows zero affected packages, you can skip even the install step, reducing CI time to seconds. --- ## Pattern 4: Clone Depth for Affected Detection ### Sufficient History for --affected ```bash # Fetch enough history for accurate comparison # Most CI providers default to shallow clone (depth=1) # Option 1: Full history (most accurate, slowest) git clone --depth=0 <repo> # or in CI: fetch-depth: 0 # Option 2: Reasonable depth (good balance) git clone --depth=50 <repo> # Option 3: Rely on --affected fallback # Shallow clone + --affected falls back to full task graph # Acceptable if Remote Cache makes full runs fast anyway ``` **Trade-off:** Full clone is most accurate but adds clone time (10-30s for large repos). `--affected` with shallow clone falls back gracefully, so if Remote Cache hit rate is high, the fallback penalty is minimal. ### Fetch Base Branch for PR Comparison ```bash # Some CI environments only clone the PR branch # Fetch the base branch for accurate comparison git fetch origin main --depth=1 turbo run test --filter=...[origin/main] ``` --- ## Pattern 5: Combining --affected with --filter ```bash # Affected packages, but only in the apps directory turbo run build --affected --filter=./apps/* # Affected packages, excluding admin app turbo run test --affected --filter=!admin # Affected packages that are dependencies of web turbo run build --affected --filter=web... ``` **Note:** When combining `--affected` with `--filter`, the result is the intersection -- only packages that are both affected AND match the filter. -
core.md 7.5 KB
# Turborepo CI - Core Examples > turbo.json task configuration, outputs, environment variables, caching control, and filter syntax. See [SKILL.md](../SKILL.md) for core concepts and [reference.md](../reference.md) for decision frameworks. **Additional Examples:** - [remote-cache.md](remote-cache.md) - Remote Cache setup, self-hosted, signature verification - [affected-detection.md](affected-detection.md) - --affected, turbo query affected, conditional CI steps - [docker.md](docker.md) - turbo prune --docker, multi-stage Dockerfile --- ## Pattern 1: Complete turbo.json for CI ### Task Configuration with Outputs and Env ```jsonc // turbo.json - Root configuration { "$schema": "https://turborepo.dev/schema.json", "globalDependencies": ["tsconfig.base.json", ".env"], "globalEnv": ["CI", "NODE_ENV"], "tasks": { "build": { "dependsOn": ["^build"], "outputs": ["dist/**", "build/**"], "env": ["API_URL", "SENTRY_DSN", "PUBLIC_ASSET_PREFIX"], "outputLogs": "new-only", }, "test": { "dependsOn": ["^build"], "outputs": ["coverage/**"], "env": ["DATABASE_URL", "TEST_API_URL"], "outputLogs": "new-only", }, "lint": { "dependsOn": [], "cache": true, "outputLogs": "errors-only", }, "type-check": { "dependsOn": ["^build"], "cache": true, "outputLogs": "errors-only", }, "dev": { "dependsOn": ["^build"], "cache": false, "persistent": true, }, }, } ``` **Why good:** Every cacheable task declares `outputs` so artifacts restore correctly on cache hit. Environment variables that affect output are in `env`. `dev` has `cache: false` and `persistent: true` because it's a long-running process. `outputLogs` reduces CI noise -- `errors-only` for lint/type-check, `new-only` for build/test. ### Bad Example: Missing Outputs and Env ```jsonc // ❌ Bad - Missing outputs and env declarations { "tasks": { "build": { "dependsOn": ["^build"], // No outputs: cache hit restores nothing // No env: API_URL change doesn't bust cache }, "test": { "dependsOn": ["^build"], // No outputs: coverage reports lost on cache hit }, }, } ``` **Why bad:** Cache hits restore logs but no artifacts (missing `outputs`). Environment variable changes don't invalidate cache (missing `env`), so `build` with `API_URL=staging` serves cached result from `API_URL=production`. --- ## Pattern 2: Package-Level turbo.json Overrides Individual packages can extend root turbo.json to add package-specific configuration. ```jsonc // apps/web/turbo.json - Package-level override { "$schema": "https://turborepo.dev/schema.json", "extends": ["//"], "tasks": { "build": { "env": ["PUBLIC_API_URL", "PUBLIC_ANALYTICS_ID"], "outputs": ["dist/**", "build/**"], }, "test": { "env": ["PLAYWRIGHT_BASE_URL"], "inputs": ["src/**", "tests/**", "playwright.config.ts"], }, }, } ``` **Why good:** `extends: ["//"]` inherits from root config. Package-specific env vars and outputs are declared where they're relevant. `inputs` on test narrows what invalidates the cache -- changes to `README.md` won't trigger test re-runs. --- ## Pattern 3: Environment Variable Modes ### Strict Mode (Default in v2) ```jsonc { "envMode": "strict", "globalEnv": ["CI", "NODE_ENV"], "globalPassThroughEnv": ["npm_config_registry", "HTTP_PROXY"], "tasks": { "build": { "env": ["API_URL"], "passThroughEnv": ["DEPLOY_REGION"], }, }, } ``` **Strict mode behavior:** - Only variables listed in `env`/`globalEnv`/`passThroughEnv`/`globalPassThroughEnv` are available - Unlisted variables are invisible to tasks - Prevents accidental dependency on undeclared variables ### Loose Mode (Migration Helper) ```jsonc { "envMode": "loose", } ``` **Loose mode behavior:** - All process environment variables available to all tasks - Only `env`/`globalEnv` variables affect the cache hash - Useful during migration to strict mode -- everything works, but cache correctness depends on manual `env` declarations **Migration strategy:** Start with `loose`, run `turbo run build --summarize` to identify which variables are actually used, add them to `env`/`globalEnv`, then switch to `strict`. --- ## Pattern 4: Cache Control in CI ### Selective Cache Sources ```bash # Use only local cache (debugging remote cache issues) turbo run build --cache=local:rw,remote:off # Read from remote but don't upload (preserve known-good cache) turbo run build --cache=local:rw,remote:r # No caching at all (full clean build) turbo run build --force # Disable caching for a specific run turbo run build --cache=off ``` ### Debugging Cache Misses with --summarize ```bash # Generate run summary turbo run build --summarize # Output: .turbo/runs/<run-id>.json # Contains: task hashes, input file hashes, env var hashes, timing data # Compare two runs to find what caused a cache miss diff <(jq '.tasks[0].hash' .turbo/runs/run1.json) \ <(jq '.tasks[0].hash' .turbo/runs/run2.json) ``` ### Dry Run for Execution Plan ```bash # See what would execute without running turbo run build test lint --dry # JSON output for scripting turbo run build --dry --json ``` **Why useful in CI:** Verify affected detection is selecting the right packages before committing to a full run. Debug why a task is being re-executed when you expect a cache hit. --- ## Pattern 5: Concurrency and Parallel Execution ```bash # Default: 10 parallel tasks turbo run build test lint # Increase for CI runners with many cores turbo run build test lint --concurrency=20 # Percentage of available CPUs turbo run build test lint --concurrency=50% # Serial execution for debugging ordering issues turbo run build --concurrency=1 ``` ### Task Dependency Patterns ```jsonc { "tasks": { // Lint has no dependencies -- runs immediately "lint": { "dependsOn": [] }, // Type-check needs upstream builds "type-check": { "dependsOn": ["^build"] }, // Test needs upstream builds "test": { "dependsOn": ["^build"] }, // Build needs upstream builds "build": { "dependsOn": ["^build"] }, // Deploy runs after build in the SAME package (no ^ prefix) "deploy": { "dependsOn": ["build"] }, // E2E tests run after build in the same package "test:e2e": { "dependsOn": ["build"], "cache": false }, }, } ``` **Key distinction:** `^build` means "build in dependency packages must complete first." `build` (no `^`) means "build in THIS package must complete first." `[]` means "no dependencies, run immediately." --- ## Pattern 6: Input Customization ### Narrowing Inputs for Better Cache Hit Rate ```jsonc { "tasks": { "lint": { "inputs": [ "src/**/*.ts", "src/**/*.tsx", ".eslintrc.*", "eslint.config.*", ], "dependsOn": [], }, "test": { "inputs": ["src/**", "tests/**", "vitest.config.*", "$TURBO_DEFAULT$"], }, }, } ``` **Why good:** Lint only re-runs when source files or eslint config change -- README edits don't trigger it. `$TURBO_DEFAULT$` in test inputs adds custom globs to the default inputs rather than replacing them. ### Using $TURBO_ROOT$ for Root-Level Files ```jsonc { "tasks": { "build": { "inputs": [ "src/**", "$TURBO_ROOT$/tsconfig.base.json", "$TURBO_DEFAULT$", ], }, }, } ``` **Why useful:** `$TURBO_ROOT$` makes the glob relative to the repository root instead of the package directory. Useful for shared configs that live at the root. -
docker.md 3.7 KB
# Turborepo CI - Docker Examples > turbo prune --docker for optimized Docker builds, multi-stage Dockerfile patterns, and layer caching. See [SKILL.md](../SKILL.md) for core concepts and [core.md](core.md) for turbo.json task configuration. --- ## Pattern 1: turbo prune --docker ### Generate Pruned Workspace ```bash # Prune to only packages needed for 'web' app turbo prune web --docker # Output structure: # out/ # json/ # package.json files only (for dependency install layer) # full/ # Full source code (for build layer) # package-lock.json # Pruned lockfile (subset of root lockfile) ``` **Why --docker flag:** Splits output into `json/` (package manifests) and `full/` (source). This enables Docker layer caching: the dependency install layer only invalidates when `package.json` files change, not when source code changes. **Without --docker:** All files land in a single directory, meaning any source change invalidates the dependency install layer. --- ## Pattern 2: Multi-Stage Dockerfile ```dockerfile # Stage 0: Generate pruned monorepo FROM node:20-alpine AS pruner RUN npm install -g turbo WORKDIR /app COPY . . RUN turbo prune web --docker # Stage 1: Install dependencies (cached unless package.json/lockfile changes) FROM node:20-alpine AS deps WORKDIR /app # Copy only package manifests and pruned lockfile COPY --from=pruner /app/out/json/ . COPY --from=pruner /app/out/package-lock.json ./package-lock.json RUN npm install --frozen-lockfile # Stage 2: Build (cached unless source changes) FROM node:20-alpine AS builder WORKDIR /app COPY --from=deps /app . COPY --from=pruner /app/out/full/ . RUN npx turbo run build --filter=web # Stage 3: Production image (minimal) FROM node:20-alpine AS runner WORKDIR /app # Copy only the built output (adjust paths to your framework's output) COPY --from=builder /app/apps/web/dist ./dist COPY --from=builder /app/apps/web/public ./public CMD ["node", "dist/server.js"] ``` **Why good:** Four distinct layers with clear cache invalidation boundaries. Source changes only invalidate stage 2+. Dependency changes invalidate stage 1+. The production image contains only runtime artifacts. ### Bad Example: No Pruning ```dockerfile # ❌ Bad - Copies entire monorepo, no layer separation FROM node:20-alpine WORKDIR /app COPY . . RUN npm install RUN npx turbo run build --filter=web CMD ["node", "apps/web/server.js"] ``` **Why bad:** Any file change in any package invalidates ALL layers. Docker rebuilds everything from `COPY . .` forward. No lockfile pruning means unrelated package additions bust the install cache. Production image includes all source, devDependencies, and build tools. --- ## Pattern 3: Custom Output Directory ```bash # Change output directory (default: ./out) turbo prune web --docker --out=./docker-context # Use in Dockerfile COPY --from=pruner /app/docker-context/json/ . ``` --- ## Pattern 4: Respect .gitignore ```bash # Exclude gitignored files from pruned output turbo prune web --docker --respect-gitignore ``` **When useful:** Prevents copying generated files, caches, or local env files into the Docker context. Reduces context size sent to Docker daemon. --- ## Pattern 5: Combining prune with Remote Cache in CI ```bash #!/usr/bin/env bash set -euo pipefail # Step 1: Check if web app is affected AFFECTED=$(turbo query affected --packages web \ | jq '.data.affectedPackages.length') if [ "$AFFECTED" -eq 0 ]; then echo "web not affected -- skipping Docker build" exit 0 fi # Step 2: Prune and build Docker image turbo prune web --docker docker build -t web:latest -f apps/web/Dockerfile . ``` **Why good:** Skips the entire Docker build if the web package isn't affected. Combines `turbo query affected` (skip decision) with `turbo prune` (optimized build context). -
remote-cache.md 3.6 KB
# Turborepo CI - Remote Cache Examples > Remote Cache setup, self-hosted options, signature verification, and cache permission tuning. See [SKILL.md](../SKILL.md) for core concepts and [core.md](core.md) for turbo.json task configuration. --- ## Pattern 1: Vercel Remote Cache Setup ### Local Development ```bash # Authenticate with Vercel account turbo login # For SSO-enabled teams turbo login --sso-team=my-team # Link local repo to Remote Cache turbo link # Verify: delete local cache and rebuild rm -rf .turbo/cache turbo run build # Should download from Remote Cache instead of rebuilding ``` ### CI Environment ```bash # Set as CI secrets (not in turbo.json) TURBO_TOKEN=<vercel-token> TURBO_TEAM=<team-slug> # That's it -- turbo auto-detects and uses Remote Cache turbo run build test lint ``` **Why good:** Two environment variables enable Remote Cache across all CI runs. No turbo.json changes needed. Vercel-hosted projects get this automatically. --- ## Pattern 2: Self-Hosted Remote Cache ### Environment Variables ```bash # Point to custom Remote Cache server TURBO_API=https://cache.internal.example.com TURBO_TOKEN=<server-auth-token> TURBO_TEAM=my-team ``` ### Manual Login for Self-Hosted ```bash # Authenticate with custom server turbo login --manual # Enter the URL and token when prompted ``` **Community implementations:** - `ducktors/turborepo-remote-cache` -- Node.js, supports S3/GCS/Azure Blob storage - `brunojppb/turbo-cache-server` -- Lightweight Rust implementation --- ## Pattern 3: Signature Verification ### Enable in turbo.json ```jsonc { "remoteCache": { "signature": true, }, } ``` ### Set Signing Key ```bash # Set as CI secret -- HMAC-SHA256 key TURBO_REMOTE_CACHE_SIGNATURE_KEY=<your-secret-key> ``` **Behavior:** - Artifacts are signed on upload with HMAC-SHA256 - Signature verified on download - Failed verification = treated as cache miss (task re-executes) - Missing key = all Remote Cache reads fail silently (treated as misses) **When to enable:** Shared caches where multiple teams or CI systems push artifacts. Prevents tampered artifacts from being served to other consumers. --- ## Pattern 4: Cache Permission Tuning ### Read-Only Remote Cache in CI ```bash # CI reads from Remote Cache but doesn't push (prevents CI from overwriting known-good cache) turbo run build --cache=local:rw,remote:r ``` ### Local-Only for Debugging ```bash # Disable Remote Cache for this run (debug local behavior) turbo run build --cache=local:rw,remote:off ``` ### Full Control ```bash # Default behavior (read + write to both) turbo run build --cache=local:rw,remote:rw # No caching at all turbo run build --cache=off # Remote read-only, no local cache turbo run build --cache=local:off,remote:r ``` --- ## Pattern 5: Remote Cache Configuration in turbo.json ```jsonc { "remoteCache": { "enabled": true, "signature": false, "preflight": false, "timeout": 30, "uploadTimeout": 60, }, } ``` | Key | Default | Purpose | | --------------- | ------- | --------------------------------------- | | `enabled` | `true` | Toggle Remote Cache | | `signature` | `false` | HMAC-SHA256 artifact signing | | `preflight` | `false` | Send preflight request before cache ops | | `timeout` | `30` | Download timeout in seconds | | `uploadTimeout` | `60` | Upload timeout in seconds | **When to increase timeouts:** Large monorepos with many packages produce large cache artifacts. If CI logs show timeout errors during cache upload/download, increase these values.
-
-
reference.md 7.3 KB
# Turborepo CI Quick Reference Decision frameworks, CLI flags, and turbo.json quick reference for CI pipelines. --- ## CLI Flags Reference ### turbo run | Flag | Default | Purpose | | --------------------- | -------------------- | --------------------------------------------------------------------- | | `--affected` | - | Run tasks only in changed packages (auto-detects CI env) | | `--filter` / `-F` | - | Target specific packages, directories, or git ranges | | `--cache` | `local:rw,remote:rw` | Cache source and permission control | | `--concurrency` | `10` | Max parallel task execution (integer or percentage) | | `--dry` / `--dry-run` | - | Show execution plan without running | | `--force` | - | Bypass cache, re-execute all tasks | | `--summarize` | - | Generate run summary in `.turbo/runs/` | | `--output-logs` | `full` | Log verbosity: `full`, `hash-only`, `new-only`, `errors-only`, `none` | | `--env-mode` | `strict` | `strict` (only listed vars) or `loose` (all vars available) | | `--graph` | - | Generate task graph (svg, html, mermaid, dot) | | `--only` | - | Run specified tasks without their dependencies | | `--continue` | `never` | Error handling: `never`, `dependencies-successful`, `always` | | `--json` | - | Stream NDJSON output to stdout | | `--log-file` | - | Write structured logs to file | | `--team` | - | Remote Cache team slug | | `--token` | - | Remote Cache auth token | ### turbo prune | Flag | Default | Purpose | | --------------------- | ------- | ---------------------------------------------------------- | | `--docker` | - | Split output for Docker layer caching (json/ + full/ dirs) | | `--out` | `./out` | Output directory for pruned workspace | | `--respect-gitignore` | - | Honor .gitignore when copying files | ### turbo query | Subcommand | Purpose | | ------------------------------------ | ------------------------------------------------- | | `turbo query affected` | List affected packages/tasks as JSON | | `turbo query affected --packages` | List only affected package names | | `turbo query affected --tasks build` | List affected tasks matching a specific task name | | `turbo query ls` | List all packages in the workspace | --- ## --filter Syntax Cheat Sheet | Pattern | Meaning | | ----------------------------- | ------------------------------------------------------ | | `--filter=web` | Package named `web` | | `--filter=web...` | `web` and all its dependencies | | `--filter=...web` | `web` and all its dependents | | `--filter=...^web` | All dependents of `web` (excluding `web` itself) | | `--filter=./apps/*` | All packages in `apps/` directory | | `--filter=...[origin/main]` | Packages changed since `origin/main` | | `--filter={./apps/*}[HEAD^1]` | Packages in `apps/` changed since last commit | | `--filter=web --filter=api` | Union of `web` and `api` | | `--filter=!admin` | Exclude `admin` from results | | `web#build` | Run `build` task for `web` only (no `--filter` needed) | --- ## turbo.json Task Keys | Key | Affects Hash? | Purpose | | ---------------- | ------------- | -------------------------------------------------------- | | `dependsOn` | No | Tasks/packages that must complete first (`^` = upstream) | | `outputs` | No | File globs to cache after task completion | | `cache` | No | Enable/disable caching (default: `true`) | | `env` | Yes | Environment variables included in task hash | | `passThroughEnv` | No | Variables available at runtime but not in hash | | `inputs` | Yes | File globs determining task invalidation | | `outputLogs` | No | Log verbosity for cached task replay | | `persistent` | No | Mark long-running processes (e.g., dev servers) | | `interactive` | No | Allow stdin input during execution | | `description` | No | Human-readable task documentation | ## turbo.json Global Keys | Key | Affects Hash? | Purpose | | ---------------------- | --------------- | --------------------------------------------------- | | `globalEnv` | Yes (all tasks) | Environment variables affecting all task hashes | | `globalDependencies` | Yes (all tasks) | File globs affecting all task hashes | | `globalPassThroughEnv` | No | Variables available to all tasks, not in hash | | `envMode` | No | `strict` (default) or `loose` variable filtering | | `concurrency` | No | Default max parallel tasks | | `cacheDir` | No | Filesystem cache location (default: `.turbo/cache`) | --- ## Environment Variables for CI | Variable | Purpose | | ---------------------------------- | --------------------------------------------------- | | `TURBO_TOKEN` | Bearer token for Remote Cache authentication | | `TURBO_TEAM` | Team/account slug for Remote Cache | | `TURBO_API` | Custom Remote Cache server URL (self-hosted) | | `TURBO_REMOTE_CACHE_SIGNATURE_KEY` | HMAC-SHA256 key for artifact signing | | `TURBO_FORCE` | Force re-execution (equivalent to `--force`) | | `CI` | Auto-detected by Turborepo for CI-specific behavior | --- ## Gotchas & Edge Cases > See [SKILL.md](SKILL.md) RED FLAGS section for the complete list of gotchas, edge cases, and anti-patterns. -
SKILL.md 17.4 KB
--- name: infra-ci-cd-turborepo-ci description: Turborepo CI pipelines with remote caching and affected detection --- # Turborepo CI Patterns > **Quick Guide:** Use `--affected` for PR builds (auto-detects CI environment, falls back to full suite on shallow clones). Enable Remote Cache with `TURBO_TOKEN` + `TURBO_TEAM` env vars. Declare `outputs` for every cacheable task or cached results will be incomplete. Use `env` and `globalEnv` in turbo.json to include environment variables in cache hashes -- missing entries cause cross-environment cache collisions. Pin `turbo` version in CI. Use `turbo query affected` to conditionally skip entire CI steps. --- <critical_requirements> ## CRITICAL: Before Using This Skill > **All code must follow project conventions in CLAUDE.md** (kebab-case, named exports, import ordering, `import type`, named constants) **(You MUST declare `outputs` for every cacheable task in turbo.json -- missing outputs means cached results restore without build artifacts)** **(You MUST list environment variables that affect task output in `env` (task-level) or `globalEnv` (all tasks) -- omitting them causes cross-environment cache hits with wrong values)** **(You MUST use `--affected` for PR builds -- running the full task graph on PRs wastes CI time on unchanged packages)** **(You MUST pin the `turbo` CLI version in CI -- `latest` can introduce breaking changes mid-pipeline)** </critical_requirements> --- **Detailed Resources:** - [examples/core.md](examples/core.md) - turbo.json task config, outputs, env, caching control, filter syntax - [examples/remote-cache.md](examples/remote-cache.md) - Remote Cache setup, self-hosted options, signature verification - [examples/affected-detection.md](examples/affected-detection.md) - --affected flag, turbo query affected, conditional CI steps - [examples/docker.md](examples/docker.md) - turbo prune --docker, multi-stage Dockerfile, layer caching - [reference.md](reference.md) - Decision frameworks, CLI flags, turbo.json quick reference --- **Auto-detection:** Turborepo CI, turbo.json, turbo run, turbo prune, --affected, --filter, TURBO_TOKEN, TURBO_TEAM, Remote Cache, turbo query affected, outputs, dependsOn, globalEnv, envMode, concurrency, turbo login, turbo link, cache artifacts, monorepo CI **When to use:** - Configuring CI pipelines for a Turborepo monorepo - Setting up Remote Cache for shared build artifacts across CI and local - Using `--affected` to run only changed-package tasks on PRs - Optimizing Docker builds with `turbo prune --docker` - Debugging cache misses with `--summarize` or `--dry` - Skipping CI steps conditionally with `turbo query affected` **When NOT to use:** - General Turborepo workspace setup (package structure, task graph design) -- that belongs in a monorepo/workspace skill - CI provider-specific workflow syntax (use your CI provider's skill) - Application build configuration (bundler, compiler settings) **Key patterns covered:** - turbo.json task configuration (`outputs`, `env`, `dependsOn`, `cache`, `inputs`) - Remote Cache authentication and setup (`TURBO_TOKEN`, `TURBO_TEAM`, signature verification) - Affected detection (`--affected`, `--filter=...[origin/main]`, `turbo query affected`) - Docker optimization with `turbo prune --docker` and multi-stage builds - Cache debugging (`--summarize`, `--dry`, `--force`) - Environment variable modes (`strict` vs `loose`) and `passThroughEnv` - Concurrency control and output log filtering --- <philosophy> ## Philosophy Turborepo's CI value comes from two things: **caching** (never redo work whose inputs haven't changed) and **affected detection** (never start work that can't have changed). The combination turns a 15-minute full monorepo build into a sub-minute cache restore for unchanged packages. **Core CI principles:** - **Cache correctness over speed:** A wrong cache hit is worse than a cache miss. Declare all `outputs` and all `env` variables that affect task results. - **Affected detection for PRs, full suite for main:** PRs get fast feedback via `--affected`. Main branch runs the full task graph to catch integration issues. - **Remote Cache for team-wide sharing:** Local cache is per-machine. Remote Cache shares artifacts across CI runners and developer machines, eliminating redundant work organization-wide. - **Pin versions in CI:** Turborepo follows semver, but `latest` in CI means non-reproducible builds. Pin to the major version at minimum. **When to use Turborepo in CI:** - Monorepo with 2+ packages where cross-package caching saves meaningful time - Teams where multiple developers and CI runners rebuild the same packages - Projects where Docker builds benefit from pruned lockfiles **When NOT to use:** - Single-package repos (no cross-package caching benefit) - Repos where every PR touches every package (affected detection provides no speedup) </philosophy> --- <patterns> ## Core Patterns ### Pattern 1: Task Configuration in turbo.json Every task that produces files must declare `outputs`. Every task affected by environment variables must declare `env`. Missing either causes cache correctness issues. ```jsonc { "$schema": "https://turborepo.dev/schema.json", "tasks": { "build": { "dependsOn": ["^build"], "outputs": ["dist/**", "build/**"], "env": ["NODE_ENV", "API_URL"], }, "test": { "dependsOn": ["^build"], "outputs": ["coverage/**"], "env": ["DATABASE_URL"], }, "lint": { "dependsOn": [], "cache": true, }, "type-check": { "dependsOn": ["^build"], "cache": true, }, }, } ``` **Key decisions:** - `dependsOn: ["^build"]` means "run build in all dependencies first" -- the `^` prefix means upstream packages - `outputs` defines what gets cached and restored -- omit it and cached runs produce empty results - `env` includes variables in the cache hash -- change the value, bust the cache - `cache: false` disables caching for tasks like `dev` or deployment scripts See [examples/core.md](examples/core.md) for complete task config with `inputs`, `passThroughEnv`, `outputLogs`, and package-level overrides. --- ### Pattern 2: Remote Cache Setup Remote Cache shares build artifacts across CI runners and developer machines. Two environment variables enable it. ```bash # Required for Remote Cache in CI TURBO_TOKEN=<bearer-token> # Auth token (Vercel or self-hosted) TURBO_TEAM=<team-slug> # Team/account identifier ``` **Vercel Remote Cache:** Free, automatic on Vercel deployments. For other CI providers, set `TURBO_TOKEN` and `TURBO_TEAM` as CI secrets. **Self-hosted:** Use `TURBO_API` to point to a custom Remote Cache server implementing the Turborepo Remote Cache API. Community options: `ducktors/turborepo-remote-cache`, `brunojppb/turbo-cache-server`. **Signature verification** (recommended for shared caches): ```jsonc { "remoteCache": { "signature": true, }, } ``` Set `TURBO_REMOTE_CACHE_SIGNATURE_KEY` with an HMAC-SHA256 secret. Failed verification is treated as a cache miss. See [examples/remote-cache.md](examples/remote-cache.md) for complete setup including self-hosted config and cache permission tuning. --- ### Pattern 3: Affected Detection for PR Builds `--affected` runs tasks only in packages with code changes. Auto-detects CI environment variables (`GITHUB_BASE_REF`, etc.) to determine the comparison base. ```bash # PR builds: only changed packages turbo run build test lint --affected # Manual comparison base turbo run test --filter=...[origin/main] ``` **Gotcha:** Shallow clones break affected detection because git diff needs history. `--affected` gracefully falls back to running all tasks, but `--filter=...[origin/main]` may fail silently. Ensure sufficient clone depth in CI. **Advanced: Conditional CI steps** with `turbo query affected`: ```bash # Check if a specific package is affected before running expensive steps AFFECTED=$(turbo query affected --packages web \ | jq '.data.affectedPackages.length') if [ "$AFFECTED" -gt 0 ]; then # Run expensive deployment steps fi ``` See [examples/affected-detection.md](examples/affected-detection.md) for PR vs main branch patterns and `turbo query affected` examples. --- ### Pattern 4: Docker Optimization with turbo prune `turbo prune` generates a sparse monorepo with only the packages needed to build a target. The `--docker` flag splits output for Docker layer caching. ```dockerfile # Stage 1: Install dependencies (cached unless lockfile changes) FROM node:20-alpine AS deps WORKDIR /app COPY out/json/ . RUN npm install --frozen-lockfile # Stage 2: Build (cached unless source changes) FROM node:20-alpine AS builder WORKDIR /app COPY --from=deps /app . COPY out/full/ . RUN npx turbo run build --filter=web ``` **Key benefit:** Changes to source code in one package don't invalidate the dependency install layer for other packages. Without pruning, any lockfile change (even in unrelated packages) busts the Docker cache for all images. See [examples/docker.md](examples/docker.md) for complete multi-stage Dockerfile with `turbo prune --docker`. --- ### Pattern 5: Cache Debugging When tasks produce unexpected results after cache hits, use these tools to diagnose. ```bash # See what would run without executing (dry run) turbo run build --dry # Generate detailed run summary with hashes and timing turbo run build --summarize # Output in .turbo/runs/<id>.json # Force re-execution, ignoring cache turbo run build --force # Control cache sources (disable remote, keep local) turbo run build --cache=local:rw,remote:off ``` **Common cache miss causes:** - Changed environment variable not listed in `env` or `globalEnv` - Modified file not captured by default inputs (use `inputs` key to customize) - Different `turbo` version between CI and local (different hashing algorithm) - Missing `outputs` declaration (task runs but restores nothing from cache) See [examples/core.md](examples/core.md) for `--summarize` output analysis and cache troubleshooting. --- ### Pattern 6: Environment Variable Strategy Turborepo's `strict` env mode (default in v2) only makes variables listed in `env`, `globalEnv`, or `passThroughEnv` available to tasks. This prevents accidental cache collisions but requires explicit configuration. ```jsonc { "globalEnv": ["CI", "NODE_ENV"], "tasks": { "build": { "env": ["API_URL", "SENTRY_DSN"], "passThroughEnv": ["npm_config_registry"], }, }, } ``` **Decision: `env` vs `globalEnv` vs `passThroughEnv`:** | Key | Affects hash? | Available to task? | Scope | | ---------------------- | --------------- | ------------------ | ----------- | | `env` | Yes | Yes | Single task | | `globalEnv` | Yes (all tasks) | Yes | All tasks | | `passThroughEnv` | No | Yes | Single task | | `globalPassThroughEnv` | No | Yes | All tasks | **When to use `passThroughEnv`:** Variables needed at runtime but that don't affect build output (e.g., `npm_config_registry`, `HTTP_PROXY`). Changing them should not bust the cache. See [examples/core.md](examples/core.md) for strict vs loose mode examples and environment variable debugging. </patterns> --- <performance> ## Performance Optimization **Goal: PR builds < 3 minutes with affected detection + Remote Cache** **Cache hit rate optimization:** - Declare all `outputs` and `env` variables to prevent false misses - Use `--summarize` to compare hashes between runs and identify unexpected invalidations - Keep `globalEnv` minimal -- variables there bust cache for ALL tasks - Use task-level `env` for variables that only affect specific tasks **Concurrency tuning:** ```bash # Default concurrency is 10 parallel tasks turbo run build test lint --concurrency=20 # Use percentage of available CPUs turbo run build --concurrency=50% # Serial execution (debugging) turbo run build --concurrency=1 ``` **Output log filtering for CI readability:** ```jsonc { "tasks": { "build": { "outputLogs": "new-only", }, "lint": { "outputLogs": "errors-only", }, }, } ``` Options: `full` (default), `hash-only`, `new-only`, `errors-only`, `none`. **Monitoring targets:** - **PR build:** < 3 min (with affected + Remote Cache) - **Main build:** < 10 min (full suite) - **Cache hit rate:** > 80% on Remote Cache - **Docker build:** < 5 min with pruned lockfile </performance> --- <decision_framework> ## Decision Framework ### When to use --affected vs --filter? ``` Running a PR build? |-- YES --> Use --affected (auto-detects CI env, graceful shallow clone fallback) +-- NO --> Need specific package selection? |-- YES --> Use --filter (explicit package/directory/git targeting) +-- NO --> Run full task graph (main branch, release builds) ``` ### When to enable Remote Cache? ``` Team > 1 person OR using CI? |-- YES --> Enable Remote Cache (shared artifacts save significant time) | |-- Using Vercel for hosting? | | |-- YES --> Free, auto-configured on Vercel deployments | | +-- NO --> Set TURBO_TOKEN + TURBO_TEAM as CI secrets | +-- Need artifact signing? | |-- YES --> Enable signature verification in turbo.json | +-- NO --> Default (unsigned) is fine for trusted environments +-- NO --> Local cache sufficient (solo developer, single machine) ``` ### env vs globalEnv vs passThroughEnv? ``` Does the variable affect task output (build artifacts, test results)? |-- YES --> Does it affect ALL tasks or just one? | |-- ALL tasks --> globalEnv (changes bust cache for everything) | +-- One task --> env on that specific task +-- NO --> Does the task need it at runtime? |-- YES --> passThroughEnv (available but not in hash) +-- NO --> Don't list it (strict mode blocks it) ``` ### When to use turbo prune --docker? ``` Building Docker images from monorepo? |-- YES --> Does your Dockerfile install from root lockfile? | |-- YES --> Use turbo prune --docker (pruned lockfile = better layer caching) | +-- NO --> Standard turbo prune (no --docker flag needed) +-- NO --> Not applicable ``` </decision_framework> --- <red_flags> ## RED FLAGS **High Priority:** - **Missing `outputs` on cacheable tasks** -- cached runs restore nothing, tasks appear to succeed but produce no artifacts - **Missing `env` for environment-dependent tasks** -- build with `API_URL=staging` hits cache from `API_URL=production`, serving wrong config - **Using `latest` turbo version in CI** -- non-reproducible builds, potential breaking changes mid-pipeline - **Shallow clones without fallback** -- `--filter=...[origin/main]` fails when git history is insufficient; `--affected` handles this gracefully by falling back to full suite **Medium Priority:** - **All environment variables in `globalEnv`** -- every change busts cache for ALL tasks; use task-level `env` for task-specific variables - **No `--affected` on PR builds** -- full task graph on PRs wastes CI time rebuilding unchanged packages - **Missing signature verification on shared Remote Cache** -- unsigned artifacts can be tampered with in shared environments - **Not using `turbo run` explicitly in CI** -- bare `turbo build` may conflict with future CLI subcommands **Common Mistakes:** - Forgetting `dependsOn: ["^build"]` for tasks that consume upstream package outputs (test/lint fail because dependency not built) - Using `loose` env mode and wondering why cache hits serve wrong environment config - Not including `globalDependencies` for files like `.env` or `tsconfig.base.json` that affect all packages - Setting `cache: false` on tasks that should be cached (e.g., test, lint) because of past debugging and forgetting to re-enable **Gotchas & Edge Cases:** - `--affected` auto-detects GitHub Actions via `GITHUB_BASE_REF` -- other CI providers may need manual `--filter` instead - `turbo query affected` returns JSON -- parse with `jq` for conditional CI steps - `outputs` globs are relative to the package directory, not the repo root - `$TURBO_DEFAULT$` in `inputs` restores the default input behavior when you only want to add inputs, not replace them - Remote Cache `signature: true` requires `TURBO_REMOTE_CACHE_SIGNATURE_KEY` -- missing key means cache reads fail silently (treated as misses) - `--summarize` output goes to `.turbo/runs/` -- useful for diffing hashes between two runs to find what changed - Circular package dependencies are allowed since v2.9 (validated at task graph level, not package graph level) - `--parallel` flag is deprecated -- use `persistent: true` in turbo.json for long-running tasks instead - `turbo-ignore` is deprecated -- use `turbo query affected` instead for conditional CI steps - `remoteCache.timeout` default is 30s, `uploadTimeout` is 60s -- increase for large monorepo artifacts </red_flags> --- <critical_reminders> ## CRITICAL REMINDERS > **All code must follow project conventions in CLAUDE.md** **(You MUST declare `outputs` for every cacheable task in turbo.json -- missing outputs means cached results restore without build artifacts)** **(You MUST list environment variables that affect task output in `env` (task-level) or `globalEnv` (all tasks) -- omitting them causes cross-environment cache hits with wrong values)** **(You MUST use `--affected` for PR builds -- running the full task graph on PRs wastes CI time on unchanged packages)** **(You MUST pin the `turbo` CLI version in CI -- `latest` can introduce breaking changes mid-pipeline)** **Failure to follow these rules will cause incorrect cache hits (wrong build artifacts served), slow CI (full rebuilds on every PR), and non-reproducible pipelines.** </critical_reminders>
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.