{
  "schema_version": 1,
  "skill_name": "traefik",
  "evals": [
    {
      "id": "http-routing-config",
      "prompt": "I am setting up Traefik v3 as a reverse proxy in front of two services: a web app on port 3000 and an API on port 8080. Requests to app.example.com should go to the web app and api.example.com to the API. How do I configure the routers and services?",
      "expected_output": "A Traefik configuration with the routing model explained: the response defines HTTP routers keyed on host rules (Host(`app.example.com`) and Host(`api.example.com`)), each router with a rule, a priority understanding for overlapping rules (Traefik v3 uses rule specificity, with explicit priorities only when needed), and each router pointing to a service defined with its load balancer and the correct backend port. The response explains the provider model: the same routing can be declared in a static config with a file provider or via the Docker provider's labels, and it shows a concrete YAML example for the file provider covering entryPoints, routers, and services, plus the corresponding Docker labels. It covers the operational details: which entrypoint the routers attach to, how to verify with a healthcheck or curl through the proxy, and the common mistake of a router rule that matches too broadly.",
      "assertions": [
        "Routers are keyed on host rules with correct rule syntax for the two domains",
        "Services are defined with the right load balancer backend ports",
        "The response shows the file-provider YAML and the Docker-label equivalent",
        "Entrypoints and rule-priority behavior are explained",
        "Verification via curl or healthcheck is included"
      ]
    },
    {
      "id": "tls-acme",
      "prompt": "My Traefik proxy is up and routing works over HTTP, but I need HTTPS with automatic certificates for two domains. I have a DNS provider with an API for verification. How do I configure TLS with ACME, and how do I know the certificates are being issued correctly?",
      "expected_output": "An ACME configuration for Traefik: the response prescribes the certificatesResolver in the static config with the ACME storage path, the challenge type chosen for the setup (DNS challenge via the DNS provider's API when HTTP challenge is not viable, e.g., for wildcards or restricted inbound ports), and the provider credentials wired securely via environment or secrets rather than plaintext in the config. It explains how certificates attach to routers: routers with TLS enabled automatically request certificates for their Host rules through the resolver, and it covers the operational verification: checking the ACME storage for the issued certificates, confirming the certificate's SANs and expiry, testing the HTTPS handshake with curl, and monitoring renewal behavior. It flags the common failures: a resolver with no storage file permissions, wrong challenge provider credentials, and the ACME staging versus production endpoint confusion.",
      "assertions": [
        "The certificatesResolver is configured with storage and the DNS challenge using the provider API",
        "Provider credentials are wired via environment or secrets, not plaintext",
        "TLS-enabled routers automatically obtain certificates for their host rules",
        "Verification covers the ACME store, SANs, expiry, and an HTTPS handshake test",
        "Common ACME failures and staging-versus-production confusion are addressed"
      ]
    },
    {
      "id": "middleware-chain",
      "prompt": "I need to protect my API with rate limiting, add security headers to responses, and require a client certificate or basic auth for an admin path. How do I use Traefik middlewares for this, and how do they chain together?",
      "expected_output": "A middleware design showing how middlewares compose: the response defines each middleware (rateLimit with a source criterion and burst, headers middleware adding the security headers, basicAuth with a hashed credentials file or forwardAuth for a client certificate requirement) and chains them on the routers that need them, explaining the ordering semantics — middleware chains apply in declared order and the response explains where auth sits relative to rate limiting so unauthenticated floods are rejected before hitting the backend. It covers the provider-agnostic mechanics: middlewares are defined once and referenced by name from routers (file provider or Docker labels), and the response flags the traps: middleware definitions that are declared but never referenced, rate limit source criteria that are too coarse, and basicAuth credentials stored as plaintext instead of hashed. It includes a concrete chain example with verification.",
      "assertions": [
        "Each middleware is defined with correct parameters: rateLimit, headers, basicAuth or forwardAuth",
        "Middleware chains are ordered with rationale, including auth before backend access",
        "Middlewares are defined once and referenced by routers",
        "Common traps are flagged: unreferenced middlewares, plaintext credentials, coarse rate-limit keys",
        "A concrete chained example with verification is included"
      ]
    },
    {
      "id": "docker-provider-labels",
      "prompt": "I want Traefik to discover my containers automatically: every container with a label should get a route without me editing a central config file. How do I use the Docker provider labels, and what labels do I need for a container to be routed?",
      "expected_output": "A Docker-provider setup with the discovery model explained: the response prescribes enabling the docker provider in the static config (with the docker socket mounted and the provider enabled, noting the socket permission model), then shows the container labels that define routing: traefik.enable=true, the router rule (Host(...)), the entrypoint, the service port, and optionally middlewares and TLS settings. It explains the label conventions: the double-label form for router and service definitions, how labels map to the same router-service model as the file provider, and the provider's default behaviors such as network selection and when a container is excluded. The response flags the security and operational gotchas: the docker socket access grants proxy privileges, containers with multiple networks need the right network selected, and label typos silently produce no route. Verification is prescribed by hitting the generated route and inspecting the Traefik dashboard or API.",
      "assertions": [
        "The Docker provider is enabled with the socket mount and its permission model explained",
        "The required labels for a routed container are listed: enable, rule, entrypoint, port",
        "Label-to-router-service mapping and double-label conventions are explained",
        "Socket security and multi-network selection gotchas are flagged",
        "Verification via the generated route and dashboard or API is prescribed"
      ]
    },
    {
      "id": "bad-gateway-troubleshoot",
      "prompt": "My Traefik setup was working and now one route returns 502 Bad Gateway while others work. The backend container is running and the app inside responds on localhost. Where do I look?",
      "expected_output": "A 502 diagnosis that walks the proxy-to-backend path: the response explains that 502 means Traefik could not reach the backend and directs the checks in order — is the backend actually listening on the address Traefik uses (containers on the same Docker network resolve and connect by service name, so localhost-only listeners fail), does the service definition use the correct port and scheme (http versus https against the backend), is the container on the network Traefik is configured to watch, and did the backend restart and change IP with the service pointing at a stale address. It also covers the health-check angle: if a healthcheck is defined, Traefik only routes to healthy backends, so an unhealthy backend yields a 502 even though the container runs. The response prescribes testing connectivity from inside the proxy's context and checking the Traefik logs for the specific dial error, which usually names the failing address.",
      "assertions": [
        "The 502 cause is explained as backend unreachability, not a routing-rule failure",
        "The localhost-listener pitfall is called out: Traefik connects by service name over the container network",
        "Service port, scheme, and network membership are checked",
        "Healthcheck routing to healthy-backends-only is covered as a 502 source",
        "Traefik logs are prescribed as the source of the specific dial error"
      ]
    }
  ]
}

# API & Dashboard Reference

Traefik exposes a REST API for querying configuration, health, and routing state. The dashboard is a web UI built on top of the API.

## API Configuration

```yaml
# Static config
api:
  insecure: false              # Expose on Traefik's default entryPoint (not recommended)
  dashboard: true              # Enable the dashboard UI
  debug: false                 # Enable /debug/pprof endpoints
  disableDashboardAd: false    # Remove the "Powered by Traefik" ad
```

## Securing the API/Dashboard

**NEVER use `api.insecure: true` in production.** Instead, create a router with authentication:

```yaml
# Method 1: File provider
http:
  routers:
    dashboard:
      rule: "Host(`traefik.example.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
      service: "api@internal"                    # Built-in API service
      middlewares:
        - "dashboard-auth"
        - "dashboard-headers"
      tls:
        certResolver: "letsencrypt"

  middlewares:
    dashboard-auth:
      basicAuth:
        users:
          - "admin:$2y$05$..."                   # bcrypt hash
    dashboard-headers:
      headers:
        customResponseHeaders:
          X-Robots-Tag: "noindex,nofollow,nocache"
```

Or with Docker labels:

```yaml
services:
  traefik:
    image: traefik:v3.7
    labels:
      - "traefik.http.routers.dashboard.rule=Host(`traefik.example.com`)"
      - "traefik.http.routers.dashboard.service=api@internal"
      - "traefik.http.routers.dashboard.tls=true"
      - "traefik.http.routers.dashboard.tls.certresolver=letsencrypt"
      - "traefik.http.routers.dashboard.middlewares=dashboard-auth"
      - "traefik.http.middlewares.dashboard-auth.basicauth.users=admin:$$2y$$05$$..."
```

## API Endpoints

All API endpoints are available under the configured base path (default `/`). Requires `api.dashboard: true` in static config.

### Core Endpoints

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/http/routers` | GET | List all HTTP routers |
| `/api/http/routers/{name}` | GET | Get router details |
| `/api/http/services` | GET | List all HTTP services |
| `/api/http/services/{name}` | GET | Get service details |
| `/api/http/middlewares` | GET | List all HTTP middlewares |
| `/api/http/middlewares/{name}` | GET | Get middleware details |
| `/api/http/rules` | GET | List all HTTP rules |
| `/api/tcp/routers` | GET | List all TCP routers |
| `/api/tcp/services` | GET | List all TCP services |
| `/api/tcp/middlewares` | GET | List all TCP middlewares |
| `/api/tcp/routers/{name}` | GET | Get TCP router details |
| `/api/udp/routers` | GET | List all UDP routers |
| `/api/udp/services` | GET | List all UDP services |
| `/api/version` | GET | Traefik version info |
| `/api/overview` | GET | Aggregated overview counts |

### Debug Endpoints (require `api.debug: true`)

| Endpoint | Description |
|----------|-------------|
| `/api/providers/{provider}` | View provider-specific configuration |
| `/debug/pprof/` | Go pprof profiling data |
| `/debug/pprof/cmdline` | Command line |
| `/debug/pprof/profile` | CPU profile |
| `/debug/pprof/trace` | Execution trace |
| `/debug/pprof/heap` | Heap profile |
| `/debug/pprof/goroutine` | Goroutine dump |
| `/debug/pprof/threadcreate` | Thread creation profile |
| `/debug/pprof/block` | Blocking profile |
| `/debug/pprof/mutex` | Mutex profile |

### Raw Configuration

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/rawconfig` | GET | Full current configuration (static + dynamic) |
| `/api/rawconfig/http/services` | GET | HTTP services config |
| `/api/rawconfig/http/routers` | GET | HTTP routers config |
| `/api/rawconfig/tcp/routers` | GET | TCP routers config |

### Health Check

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/ping` | GET, HEAD | Health check (requires `ping` config) |
| `/api/http/health` | GET | API health check |

## Ping Configuration

Separate from the API, the ping endpoint provides a simple health check:

```yaml
# Static config
ping:
  entryPoint: "web"              # EntryPoint for ping
  manualRouting: false           # Set true if creating your own router for ping
  terminatingStatusCode: 503     # Status code to return during shutdown
```

With `ping.entryPoint` set, Traefik auto-creates a router for `/ping`. Set `manualRouting: true` to create your own:

```yaml
http:
  routers:
    ping:
      rule: "Path(`/ping`)"
      entryPoints: ["web"]
      service: "ping@internal"
```

## API Usage Examples

```bash
# Get all HTTP routers
curl -s https://traefik.example.com/api/http/routers | jq .

# Get specific router
curl -s https://traefik.example.com/api/http/routers/my-router | jq .

# Check version
curl -s https://traefik.example.com/api/version

# Get overview counts
curl -s https://traefik.example.com/api/overview | jq .

# Health check
curl -s -o /dev/null -w "%{http_code}" https://traefik.example.com/ping

# Get raw configuration (debug mode required)
curl -s https://traefik.example.com/api/rawconfig | jq '.http.routers'
```

## Dashboard Security Checklist

1. **Never** use `api.insecure: true` in production
2. Always put authentication (BasicAuth, ForwardAuth, OIDC) in front of the dashboard
3. Restrict access by IP when possible (IPAllowList middleware)
4. Use a dedicated hostname for the dashboard (separate from your app domains)
5. Enable TLS on the dashboard route
6. Set the `X-Robots-Tag: noindex` header to prevent search indexing
7. Consider putting the dashboard on a private network/vpn-only entryPoint
8. Monitor dashboard access logs

# Community Patterns & Production Wisdom

Real-world patterns, best practices, and troubleshooting knowledge gathered from community blogs, forums, and production deployments — supplementing the official Traefik documentation.

## Middleware Execution Order

The order of middlewares in a chain is the single most impactful configuration decision. Incorrect ordering can bypass security controls.

**Proven production chain order:**

```
IP Whitelist → Rate Limit → Authentication → Header Injection → Path Rewriting
```

```yaml
http:
  middlewares:
    prod-security-chain:
      chain:
        middlewares:
          - ip-whitelist
          - rate-limit
          - auth-middleware
          - secure-headers
          - strip-api-prefix

    ip-whitelist:
      ipAllowList:
        sourceRange:
          - "10.0.0.0/8"
          - "172.16.0.0/12"

    rate-limit:
      rateLimit:
        average: 100
        burst: 200
        sourceCriterion:
          requestHost: true

    auth-middleware:
      basicAuth:
        users:
          - "admin:$2y$10$..."
        realm: "Protected Service"

    secure-headers:
      headers:
        customResponseHeaders:
          X-Content-Type-Options: "nosniff"
          Strict-Transport-Security: "max-age=31536000; includeSubDomains; preload"

    strip-api-prefix:
      stripPrefix:
        prefixes:
          - "/api/v1"
```

**Why this order matters:** A misordered chain where rate limit comes before IP whitelist allows a blocked subnet to send 200 burst requests that hit the auth middleware unnecessarily. Testing showed 80% wasted auth invocations from the wrong order.

## Rate Limiting Per-Service vs Global

Traefik's rate limiting is per-router, not global. A single misbehaving client on one route should not degrade others.

```yaml
# Per-service rate limiting — isolates noisy neighbors
http:
  routers:
    api-v2:
      rule: "Host(`api.example.com`) && PathPrefix(`/v2`)"
      middlewares:
        - strict-rate-limit
      service: api-backend

  middlewares:
    strict-rate-limit:
      rateLimit:
        average: 100
        burst: 50
        sourceCriterion:
          ipStrategy:
            depth: 1          # Use X-Forwarded-For to get real client IP
```

**Trade-off:** Per-service limits prevent a noisy client from degrading all routes. A single compromised client can still exhaust its own backend's connection pool. Combine with circuit breakers for full protection.

## Structured Logging & Metrics — Do This First

Enable before serving production traffic. Without structured logs, debugging takes 4x longer.

```yaml
# Static config
log:
  level: INFO
  format: json                  # Essential for log aggregation
  filePath: "/var/log/traefik/traefik.log"

accessLog:
  format: json
  filePath: "/var/log/traefik/access.log"
  filters:
    statusCodes:
      - "200-499"               # Don't log successful responses
    minDuration: "500ms"        # Only log slow requests
  fields:
    headers:
      defaultMode: "drop"
      names:
        User-Agent: "keep"      # But keep user-agent for analysis

metrics:
  prometheus:
    addEntryPointsLabels: true
    addServicesLabels: true
    buckets:
      - 0.005
      - 0.01
      - 0.025
      - 0.05
      - 0.1
      - 0.25
      - 0.5
      - 1.0

ping:
  entryPoint: "web"
```

**Real-world impact:** Teams with structured logs and Grafana dashboards triaged incidents in 11 minutes vs 45 minutes without — a 75% reduction in mean-time-to-resolution.

## Performance Tuning

### Connection Timeouts

```yaml
entryPoints:
  websecure:
    address: ":443"
    transport:
      respondingTimeouts:
        readTimeout: 30s
        writeTimeout: 30s
        idleTimeout: 180s
      keepAliveMaxRequests: 1000          # Max requests per keep-alive connection
      keepAliveMaxTime: 5m                # Max keep-alive connection lifetime

serversTransport:
  maxIdleConnsPerHost: 200               # Connection pool size
  forwardingTimeouts:
    dialTimeout: 30s
    responseHeaderTimeout: 15s
    idleConnTimeout: 90s
```

### TLS Performance

```yaml
# Dynamic config
tls:
  options:
    performance:
      minVersion: VersionTLS12
      cipherSuites:
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256    # Fast, hardware-accelerated
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
      preferServerCipherSuites: true
      curvePreferences:
        - CurveP256                    # Fastest P-256 curve, widely supported
```

### HTTP/2 and HTTP/3

```yaml
entryPoints:
  websecure:
    address: ":443"
    http2:
      maxConcurrentStreams: 250         # Default is fine for most workloads
    http3: {}                           # Enable HTTP/3 (UDP) — zero-config
```

**Note:** HTTP/3 requires a TCP entryPoint (starts as TCP then upgrades to UDP). Port 443 must be open for both TCP and UDP.

### Request Buffering

```yaml
http:
  middlewares:
    buf:
      buffering:
        maxRequestBodyBytes: 4194304         # 4MB — protects memory
        memRequestBodyBytes: 1048576          # 1MB in memory before disk spill
        maxResponseBodyBytes: 4194304
        memResponseBodyBytes: 1048576
```

### Load Balancing Strategies

| Strategy | Best For | Notes |
|----------|----------|-------|
| `wrr` (default) | Equal-capacity backends | Simple round-robin with optional weights |
| `p2c` (Power of Two Choices) | Variable request durations | Picks 2 random servers, routes to the one with fewer active connections |
| `hrw` (Highest Random Weight) | Session affinity without cookies | Consistent hashing by client IP |
| `leasttime` | Latency-sensitive services | Routes to the server with lowest response time + fewest connections |

```yaml
http:
  services:
    latency-sensitive-svc:
      loadBalancer:
        strategy: "leasttime"
        servers:
          - url: "http://10.0.0.1:3000"
          - url: "http://10.0.0.2:3000"
```

## Health Checks & Circuit Breakers

Always configure passive health checks — they catch latency spikes before they cascade into outages.

```yaml
http:
  services:
    api-backend:
      loadBalancer:
        servers:
          - url: "http://10.0.0.1:8080"
          - url: "http://10.0.0.2:8080"
          - url: "http://10.0.0.3:8080"
        healthCheck:
          path: "/health"
          interval: "10s"
          timeout: "3s"
          followRedirects: false
        passiveHealthCheck:
          maxFailedAttempts: 3
          failureWindow: "60s"

  middlewares:
    circuit-breaker:
      circuitBreaker:
        expression: "LatencyAtQuantileMS(50.0) > 5000 || NetworkErrorRatio() > 0.1"
        checkPeriod: "500ms"
        fallbackDuration: "30s"
        recoveryDuration: "10s"
```

**The circuit breaker expression catches two failure modes:**
- `LatencyAtQuantileMS(50.0) > 5000` — median latency over 5 seconds
- `NetworkErrorRatio() > 0.1` — 10%+ of requests producing network errors

## TLS Automation with Let's Encrypt

Traefik's built-in ACME is superior to cert-manager for Docker deployments — no external dependency.

```yaml
certificatesResolvers:
  letsencrypt:
    acme:
      email: "admin@example.com"
      storage: "/letsencrypt/acme.json"
      # HTTP-01 — simplest, no DNS provider needed
      httpChallenge:
        entryPoint: "web"
```

**When to use HTTP-01 vs DNS-01 vs TLS-ALPN-01:**

| Challenge | Wildcard | Port Needed | Complexity |
|-----------|----------|-------------|------------|
| HTTP-01 | No | 80 | Simple |
| TLS-ALPN-01 | No | 443 | Simple |
| DNS-01 | Yes | None | Complex (DNS provider API) |

**Recommendation:** Start with HTTP-01. Only switch to DNS-01 if you need wildcard certificates. HTTP-01 can coexist with HTTPS redirections — Traefik handles the challenge at the entryPoint level before the redirect.

## Traefik Behind Cloudflare / CDN

When Traefik sits behind Cloudflare (or any CDN), the client IP must be correctly identified.

### Step 1: Trust Cloudflare IPs at EntryPoint Level

This is the **only** correct way to get real client IPs in access logs. Middleware plugins cannot fix access log IPs.

```yaml
entryPoints:
  http:
    address: :80
    forwardedHeaders:
      trustedIPs: &cloudflareIPs
        - 103.21.244.0/22
        - 103.22.200.0/22
        - 103.31.4.0/22
        - 104.16.0.0/13
        - 104.24.0.0/14
        - 108.162.192.0/18
        - 131.0.72.0/22
        - 141.101.64.0/18
        - 162.158.0.0/15
        - 172.64.0.0/13
        - 173.245.48.0/20
        - 188.114.96.0/20
        - 190.93.240.0/20
        - 197.234.240.0/22
        - 198.41.128.0/17
        - 2400:cb00::/32
        - 2606:4700::/32
        - 2803:f800::/32
        - 2405:b500::/32
        - 2405:8100::/32
        - 2a06:98c0::/29
        - 2c0f:f248::/32
    http:
      redirections:
        entryPoint:
          to: https
          scheme: https

  https:
    address: :443
    forwardedHeaders:
      trustedIPs: *cloudflareIPs        # YAML anchor reuses the list
```

**Get the current Cloudflare IPs:** https://www.cloudflare.com/ips/ — update these periodically as Cloudflare's ranges change.

### Step 2: Use Plugin for Real-IP in Backend Headers (Optional)

The `forwardedHeaders.trustedIPs` setting correctly populates `X-Forwarded-For` and `X-Real-IP` headers. For additional control, use the Cloudflare plugin:

```yaml
experimental:
  plugins:
    cloudflare:
      moduleName: "github.com/BetterCorp/cloudflarewarp"
      version: "v1.0.0"
```

### Step 3: PROXY Protocol (Alternative to forwardedHeaders)

If your CDN supports PROXY protocol (Cloudflare does for some plans), use it instead:

```yaml
entryPoints:
  https:
    address: :443
    proxyProtocol:
      trustedIPs:
        - 10.0.0.0/8          # Your CDN's egress IPs
    # No forwardedHeaders needed with PROXY protocol
```

**Don't use both PROXY protocol and forwardedHeaders on the same entryPoint — they conflict.**

## CrowdSec Integration for Threat Intelligence

CrowdSec provides community-powered IP reputation filtering. Integrate as a ForwardAuth middleware.

### Architecture

```
Request → Traefik → CrowdSec Bouncer (ForwardAuth) → Backend Service
                          ↓
                    CrowdSec Agent analyzes Traefik access logs
                          ↓
                    Block decision via LAPI
```

### Docker Compose Setup

```yaml
services:
  traefik:
    image: traefik:v3.7
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./logs:/var/log/traefik
    labels:
      - "traefik.http.middlewares.crowdsec-bouncer.forwardAuth.address=http://crowdsec-bouncer:8080/api/v1/forward-auth"
      - "traefik.http.middlewares.crowdsec-bouncer.forwardAuth.trustForwardHeader=true"
      # Apply middleware to routers
      - "traefik.http.routers.secured-app.middlewares=crowdsec-bouncer"

  crowdsec:
    image: crowdsecurity/crowdsec:latest
    environment:
      - COLLECTIONS=crowdsecurity/traefik crowdsecurity/http-cve crowdsecurity/linux
    volumes:
      - ./crowdsec:/etc/crowdsec
      - ./logs:/var/log/traefik:ro        # Read Traefik access logs
    restart: unless-stopped

  crowdsec-bouncer:
    image: crowdsecurity/crowdsec-traefik-bouncer:latest
    environment:
      - CROWDSEC_BOUNCER_API_URL=http://crowdsec:8080
      - CROWDSEC_BOUNCER_API_KEY=your-api-key
    restart: unless-stopped
```

**Key points:**
- CrowdSec agent reads Traefik's access logs (must be file-based, not stdout)
- The bouncer runs as a separate container implementing ForwardAuth
- Bouncer returns 403 for blocked IPs before requests reach the backend
- CrowdSec agent uses community blocklists for known malicious IPs

## Authelia Integration (Single Sign-On)

Authelia provides SSO with 2FA, integrating via Traefik's ForwardAuth middleware.

```yaml
services:
  authelia:
    image: authelia/authelia:4.38
    labels:
      # Expose Authelia itself through Traefik
      - "traefik.enable=true"
      - "traefik.http.routers.authelia.rule=Host(`auth.example.com`)"
      - "traefik.http.routers.authelia.entrypoints=websecure"
      - "traefik.http.routers.authelia.tls=true"
      - "traefik.http.routers.authelia.tls.certresolver=letsencrypt"
      # Define ForwardAuth middleware
      - "traefik.http.middlewares.authelia.forwardAuth.address=http://authelia:9091/api/authz/forward-auth"
      - "traefik.http.middlewares.authelia.forwardAuth.trustForwardHeader=true"
      - "traefik.http.middlewares.authelia.forwardAuth.authResponseHeaders=X-Forwarded-User"
      # Trusted proxies configuration for Authelia
      - "traefik.http.middlewares.authelia.forwardAuth.tls.insecureSkipVerify=true"

  # Protected service
  protected-app:
    image: nginx:alpine
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.protected-app.rule=Host(`app.example.com`)"
      - "traefik.http.routers.protected-app.entrypoints=websecure"
      - "traefik.http.routers.protected-app.tls=true"
      - "traefik.http.routers.protected-app.middlewares=authelia"
      - "traefik.http.services.protected-app.loadbalancer.server.port=80"
```

**Authelia configuration considerations:**
- The ForwardAuth endpoint is `http://authelia:9091/api/authz/forward-auth`
- `trustForwardHeader` must be `true` so Authelia sees the original request URL
- `authResponseHeaders` passes the authenticated user to the backend
- Authelia must be on the same Docker network as Traefik
- For production, configure TLS between Traefik and Authelia (serversTransport with client certificates)

## Authentik Integration

Similar to Authelia but with more SSO/OAuth provider features:

```yaml
labels:
  # Authentik ForwardAuth middleware
  - "traefik.http.middlewares.authentik.forwardAuth.address=http://authentik-proxy:9000/outpost.goauthentik.io/auth/traefik"
  - "traefik.http.middlewares.authentik.forwardAuth.trustForwardHeader=true"
  - "traefik.http.middlewares.authentik.forwardAuth.authResponseHeaders=X-authentik-username,X-authentik-groups,X-authentik-email,X-authentik-name,X-authentik-uid"
  - "traefik.http.middlewares.authentik.forwardAuth.authResponseHeadersRegex=X-authentik-.*"

  # Router for Authentik's embedded outpost
  - "traefik.http.routers.authentik.rule=Host(`auth.example.com`)"
  - "traefik.http.routers.authentik.service=authentik-proxy"
  - "traefik.http.services.authentik-proxy.loadbalancer.server.port=9000"
```

Authentik uses an "outpost" model — the proxy component runs alongside Authentik and handles ForwardAuth. The outpost URL differs based on whether you use the embedded or standalone outpost.

## Known Limitations & Workarounds

### 1. Built-in Rate Limiting is Per-Instance

Traefik OSS rate limiting operates independently on each instance. With 3 Traefik replicas and a limit of 100 req/s, each instance allows 100 req/s = 300 total.

**Workaround:** Use Redis-backed distributed rate limiting (Traefik Hub/Enterprise feature). Or configure per-instance limits assuming worst-case single-instance load.

### 2. No Native Global Rate Limiting

Unlike NGINX's `limit_req_zone`, there's no built-in global rate limit store.

**Workaround similar to above:** Use the Redis rate limit plugin from the plugin catalog.

### 3. TCP Router Precedence Over HTTP

On shared entryPoints, TCP routers take precedence over HTTP routers. If a TCP router matches, HTTP routers never get to evaluate the request.

**Fix:** Use separate entryPoints for TCP and HTTP traffic wherever possible.

### 4. No URL Rewrite / Map (NGINX Equivalent)

Traefik lacks NGINX's `rewrite ... break` capability. Path manipulation is limited to prefix stripping, prefix adding, and regex replacement.

**Workaround:** Use `replacePathRegex` for most rewrite needs, or combine `stripPrefix` with `addPrefix` for mapping patterns.

### 5. Connection Draining

Traefik doesn't support connection draining on shutdown as gracefully as NGINX/HAPROXY. The `lifeCycle.graceTimeOut` helps but active connections may be dropped during rapid restarts.

**Mitigation:** Set `lifeCycle.requestAcceptGraceTimeout` to a reasonable value (5-10s) and use `lifeCycle.graceTimeOut` of at least 30s. Use `reusePort: true` for zero-downtime deployments.

### 6. Large Configuration = Higher Memory

Traefik's dynamic configuration scales with the number of routes. With 1000+ routes in Kubernetes, memory usage can exceed 2GB.

**Mitigation:** Use provider constraints, namespaces, and label selectors to limit the configuration scope.

### 7. No Native Fail2Ban Integration

Traefik doesn't have built-in fail2ban style rate-limiting-by-log-analysis.

**Workaround:** Use CrowdSec (recommended), or parse Traefik access logs with external fail2ban that updates iptables.

## Troubleshooting Quick Reference

### 502 Bad Gateway

| Likely Cause | Check | Fix |
|-------------|-------|-----|
| Backend container not running | `docker ps` | Start the container |
| Wrong port | `traefik.http.services.X.loadbalancer.server.port` | Set the correct exposed port |
| Backend on different network | Docker network connectivity | Ensure same network or external routing |
| Backend health check failing | `traefik.http.services.X.loadbalancer.healthCheck` | Fix backend health endpoint or remove health check |

### 503 Service Unavailable

| Likely Cause | Check | Fix |
|-------------|-------|-----|
| Circuit breaker open | Metrics show circuit breaker tripped | Check backend health, wait for recovery |
| All backends unhealthy | Passive health check counts | Fix backend or increase maxFailedAttempts |
| No servers in service | Provider didn't discover backends | Check labels/tags are correct |

### TLS / ACME Issues

| Symptom | Likely Cause | Fix |
|---------|-------------|-----|
| Certificate not issued | ACME challenge can't reach Traefik | Check port 80/443 accessibility |
| Certificate expiry warning | DNS-01 propagation delay | Increase `delayBeforeCheck` |
| "acme.json" permission denied | File permissions | `chmod 600 acme.json` |
| Rate limited by Let's Encrypt | Too many cert requests | Use staging CA for testing, reduce cert churn |

### Real-IP Problems

| Symptom | Likely Cause | Fix |
|---------|-------------|-----|
| Logs show Docker bridge IP | No forwardedHeaders configured | Set `forwardedHeaders.trustedIPs` on entryPoint |
| Backend sees only CDN IP | Cloudflare IPs not trusted | Add Cloudflare ranges to `trustedIPs` |
| Double IP in X-Forwarded-For | Normal with CDN — correct | Backend should use the first IP in the chain |

### Config Validation

```bash
# Check Traefik config (if using file provider)
traefik healthcheck --conf=/etc/traefik/traefik.yml

# Check dynamic config syntax with Traefik itself
docker exec traefik traefik healthcheck

# Verify routing is working
curl -v -H "Host: app.example.com" http://localhost/
curl -v -H "Host: app.example.com" https://localhost/ -k

# Check ACME certificate status
docker exec traefik sh -c 'cat /letsencrypt/acme.json' | python3 -m json.tool
```

# CSP / SPA Debugging — Entrypoint Header Overwrite

Debugging guide for the failure mode where an entrypoint-level `headers` middleware
silently breaks cross-origin SPAs by overwriting router-level
Content-Security-Policy headers.

## The Mechanism

Traefik middleware execution order on the response path is the **reverse** of the
request path:

```
Request:  entrypoint middlewares → router middlewares → service middlewares → backend
Response: backend → service middlewares → router middlewares → entrypoint middlewares
```

Entrypoint-level middlewares run **last on the response path**. The `headers`
middleware documentation states:

> "Custom headers will overwrite existing headers if they have identical names."
> — [Traefik Headers middleware docs](https://doc.traefik.io/traefik/reference/routing-configuration/http/middlewares/headers/)

Therefore, when an entrypoint applies a chain containing `contentSecurityPolicy`
(e.g., a `default@file` security-headers chain on `websecure`), it overwrites any
router-level CSP on the response. **Router-level CSP overrides are impossible**
when the entrypoint also sets CSP.

## The Failure

If the entrypoint CSP is generic — typically `default-src 'self'` with no
`connect-src` directive — the browser enforces it and blocks every cross-origin
`fetch`/`XHR` the SPA makes. The SPA's own page and static assets load normally
(they are same-origin), but all API calls to a different origin silently fail.

**The backend logs show zero requests from the SPA.** This is the key tell that
distinguishes this failure from backend issues, network problems, or CORS
misconfiguration on the backend.

## Diagnostic Flow

```
SPA page loads (200) but API calls fail silently
│
├─ 1. Check CSP on the SPA page:
│     curl -D- -o /dev/null https://your-spa.example.com/ | grep -i content-security-policy
│
│   If you see `default-src 'self'` with no `connect-src` → this is the problem.
│   The browser blocks all cross-origin requests.
│
├─ 2. Confirm zero requests reach the backend:
│     Check backend logs for the complete absence of requests from the SPA.
│     (Not 403s, not CORS errors — nothing at all.)
│
├─ 3. Identify the source of the CSP:
│     Check whether the CSP comes from the backend or from Traefik.
│     curl -D- -o /dev/null https://your-backend-api.example.com/ | grep -i content-security-policy
│
│   If the backend emits its own (different) CSP but the SPA page shows a
│   generic one, the entrypoint middleware is overwriting it.
│
└─ 4. Check the entrypoint middleware chain:
      Look at the entrypoint's `http.middlewares` list in static config,
      then trace the chain to find `contentSecurityPolicy` in a headers middleware.
```

## The Fix

Remove `contentSecurityPolicy` from the entrypoint-level default middleware chain.
Let each service emit its own tailored CSP. Many applications (GoToSocial,
Mastodon, Nextcloud, etc.) ship their own CSP headers that are specific to their
needs.

A proxy-wide `default-src 'self'` is actively harmful for any SPA that
communicates with a different origin. The proxy should not impose a CSP that
overrides what the application itself intends.

Other security headers (HSTS, X-Content-Type-Options, X-Frame-Options,
Referrer-Policy) are safe to keep in the entrypoint chain — they do not interfere
with cross-origin API calls.

## CORS Preflight Interception

The `headers` middleware also intercepts CORS preflight requests when CORS headers
are configured:

> "If CORS headers are set, then the middleware does not pass preflight requests
> to any service, instead the response will be generated and sent back to the
> client directly."
> — [Traefik Headers middleware docs](https://doc.traefik.io/traefik/reference/routing-configuration/http/middlewares/headers/)

This is useful when a backend does not handle `OPTIONS` preflight requests (returns
405). Adding a `headers` middleware with `accessControlAllowMethods` and
`accessControlAllowOriginList` to the router handles preflights at the proxy level.

Example:

```yaml
http:
  middlewares:
    cors-preflight:
      headers:
        accessControlAllowMethods:
          - GET
          - POST
          - PUT
          - DELETE
          - PATCH
          - OPTIONS
        accessControlAllowOriginList:
          - "*"
        accessControlAllowHeaders:
          - "*"
        accessControlMaxAge: 120
        addVaryHeader: true
```

## Related but Distinct Failures

| Failure | Symptom | Cause |
|---------|---------|-------|
| **CSP overwrite (this guide)** | SPA loads, zero API requests reach backend | Entrypoint `headers` middleware overwrites router CSP |
| **Rate limiting** | SPA loads, 429 errors on asset/API requests | Entrypoint rate limit too low for SPA burst |
| **Double middleware execution** | Doubled rate-limit counts, duplicate compression | Same middleware declared at both entrypoint and router level |

## Sources

- [Traefik Headers middleware](https://doc.traefik.io/traefik/reference/routing-configuration/http/middlewares/headers/) — header overwrite behavior, CORS preflight interception
- [Traefik Middleware overview](https://doc.traefik.io/traefik/reference/routing-configuration/http/middlewares/overview/) — router vs. service middleware execution order
- [Traefik Entrypoints](https://doc.traefik.io/traefik/reference/install-configuration/entrypoints/) — entrypoint-level `http.middlewares` configuration
- [MDN Content-Security-Policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy) — `default-src`, `connect-src`, browser enforcement
- [MDN CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) — preflight request mechanics
- [unrolled/secure](https://github.com/unrolled/secure#available-options) — the library Traefik uses for security headers

# Docker Provider Reference

Traefik's Docker provider automatically discovers containers and generates routing configuration from container labels. This is the most common way to use Traefik in development and production.

## Enabling the Docker Provider

```yaml
# Static config (traefik.yml)
providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false    # Secure: only expose labeled containers
    network: "proxy"           # Default network
    defaultRule: "Host(`{{ normalize .Name }}`)"
    watch: true
```

## Docker Compose — Mounting the Socket

```yaml
services:
  traefik:
    image: traefik:v3.7
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock   # Required
      - ./traefik.yml:/etc/traefik/traefik.yml       # Static config
      - ./letsencrypt:/letsencrypt                    # ACME storage
    networks:
      - proxy
```

## All Docker Labels Reference

Labels follow the pattern: `traefik.<protocol>.<resource-type>.<name>.<option>=<value>`

### General Labels

| Label | Description |
|-------|-------------|
| `traefik.enable=true` | Explicitly enable this container (required when `exposedByDefault=false`) |
| `traefik.docker.network=mynetwork` | Override the default Docker network for this container |
| `traefik.docker.allowNonRunning=true` | Enable discovery of non-running containers |

### HTTP Router Labels

Prefix: `traefik.http.routers.<name>.`

| Label | Description |
|-------|-------------|
| `rule=Host(\`example.com\`)` | Router matching rule (REQUIRED — or use `defaultRule`) |
| `entrypoints=web,websecure` | EntryPoints to attach this router to |
| `middlewares=auth,ratelimit` | Comma-separated list of middleware names |
| `service=my-service` | Target service for this router (auto-assigned if omitted) |
| `tls=true` | Enable TLS |
| `tls.certresolver=letsencrypt` | Certificate resolver for automatic cert |
| `tls.domains[0].main=example.com` | Define domain for TLS certificate |
| `tls.domains[0].sans=www.example.com` | SANs for the certificate |
| `tls.options=default@file` | TLS options to apply |
| `priority=42` | Router priority (higher = matched first) |
| `ruleSyntax=v3` | Rule syntax version (v3 or v2, default: v3) |
| `observability.metrics=true` | Enable metrics for this router |
| `observability.accesslogs=true` | Enable access logs for this router |
| `observability.tracing=true` | Enable tracing for this router |

### HTTP Service Labels

Prefix: `traefik.http.services.<name>.`

| Label | Description |
|-------|-------------|
| `loadbalancer.server.port=8080` | Backend port (REQUIRED when container exposes multiple ports) |
| `loadbalancer.server.scheme=https` | Backend scheme (default: http) |
| `loadbalancer.passhostheader=true` | Forward Host header to backend |
| `loadbalancer.strategy=wrr` | Strategy: wrr, p2c, hrw, leasttime |
| `loadbalancer.sticky.cookie.name=sticky` | Enable sticky sessions via cookie |
| `loadbalancer.sticky.cookie.httponly=true` | Set HttpOnly flag on sticky cookie |
| `loadbalancer.sticky.cookie.secure=true` | Set Secure flag on sticky cookie |
| `loadbalancer.sticky.cookie.samesite=none` | SameSite attribute: none/lax/strict |
| `loadbalancer.sticky.cookie.maxage=86400` | Cookie max age in seconds |
| `loadbalancer.healthcheck.path=/health` | Health check endpoint path |
| `loadbalancer.healthcheck.interval=10s` | Health check interval |
| `loadbalancer.healthcheck.timeout=3s` | Health check timeout |
| `loadbalancer.healthcheck.hostname=example.com` | Host header for health check |
| `loadbalancer.healthcheck.scheme=https` | Scheme for health check |
| `loadbalancer.healthcheck.method=GET` | HTTP method for health check |
| `loadbalancer.healthcheck.headers.X-Custom=v` | Custom headers for health check |
| `loadbalancer.healthcheck.followredirects=true` | Follow redirects during health checks |
| `loadbalancer.passivehealthcheck.maxfailedattempts=3` | Failures before considered unhealthy |
| `loadbalancer.passivehealthcheck.failurewindow=3s` | Time window for failure counting |
| `loadbalancer.serverstransport=custom@file` | Reference to a ServersTransport |
| `loadbalancer.responseforwarding.flushinterval=150ms` | Flush interval for streaming |

### HTTP Middleware Labels

Prefix: `traefik.http.middlewares.<name>.`

Middleware types and their label syntax:

| Middleware Type | Label Syntax |
|----------------|-------------|
| `addprefix` | `traefik.http.middlewares.my-mw.addprefix.prefix=/api/v2` |
| `basicauth` | `traefik.http.middlewares.my-mw.basicauth.users=user:$$2y$$10$$...` |
| `digestauth` | `traefik.http.middlewares.my-mw.digestauth.users=user:realm:hash` |
| `forwardauth` | `traefik.http.middlewares.my-mw.forwardauth.address=http://auth:8080/verify` |
| `headers` | `traefik.http.middlewares.my-mw.headers.customrequestheaders.X-Scope=internal` |
| `ipallowlist` | `traefik.http.middlewares.my-mw.ipallowlist.sourcerange=10.0.0.0/8` |
| `ratelimit` | `traefik.http.middlewares.my-mw.ratelimit.average=100` |
| `redirectscheme` | `traefik.http.middlewares.my-mw.redirectscheme.scheme=https` |
| `redirectregex` | `traefik.http.middlewares.my-mw.redirectregex.regex=^http://(.*)` |
| `replacepath` | `traefik.http.middlewares.my-mw.replacepath.path=/fallback` |
| `replacepathregex` | `traefik.http.middlewares.my-mw.replacepathregex.regex=^/api/v1/(.*)` |
| `stripprefix` | `traefik.http.middlewares.my-mw.stripprefix.prefixes=/api,/v1` |
| `stripprefixregex` | `traefik.http.middlewares.my-mw.stripprefixregex.regex=/[a-z]+/public` |
| `retry` | `traefik.http.middlewares.my-mw.retry.attempts=3` |
| `compress` | `traefik.http.middlewares.my-mw.compress=true` |
| `circuitbreaker` | `traefik.http.middlewares.my-mw.circuitbreaker.expression=NetworkErrorRatio() > 0.5` |
| `inflightreq` | `traefik.http.middlewares.my-mw.inflightreq.amount=100` |
| `ratelimit` | `traefik.http.middlewares.my-mw.ratelimit.average=100` |
| `chain` | `traefik.http.middlewares.my-mw.chain.middlewares=mw1,mw2,mw3` |
| `errorpages` | `traefik.http.middlewares.my-mw.errorpages.status=500-599` |
| `contenttype` | `traefik.http.middlewares.my-mw.contenttype=true` |
| `grpcweb` | `traefik.http.middlewares.my-mw.grpcweb=true` |

### TCP Router Labels

Prefix: `traefik.tcp.routers.<name>.`

| Label | Description |
|-------|-------------|
| `rule=HostSNI(\`example.com\`)` | SNI matching rule |
| `entrypoints=postgres` | EntryPoints (must be TCP) |
| `service=my-service` | Target TCP service |
| `tls=true` | Enable TLS passthrough or termination |
| `tls.certresolver=letsencrypt` | Cert resolver for TLS termination |
| `tls.options=default@file` | TLS options |
| `tls.passthrough=false` | TLS passthrough (no termination) |

### TCP Service Labels

Prefix: `traefik.tcp.services.<name>.`

| Label | Description |
|-------|-------------|
| `loadbalancer.server.port=5432` | Backend port |
| `loadbalancer.server.address=10.0.0.1` | Backend IP (auto-detected if omitted) |
| `loadbalancer.server.proxyProtocol.version=1` | Enable PROXY protocol to backend |
| `loadbalancer.terminationdelay=5s` | Connection termination delay |
| `loadbalancer.sticky=false` | Enable sticky sessions |
| `loadbalancer.strategy=wrr` | Load balancing strategy |

### UDP Router Labels

Prefix: `traefik.udp.routers.<name>.`

| Label | Description |
|-------|-------------|
| `entrypoints=dns` | EntryPoints (must be UDP) |
| `service=my-service` | Target UDP service |

### UDP Service Labels

Prefix: `traefik.udp.services.<name>.`

| Label | Description |
|-------|-------------|
| `loadbalancer.server.port=53` | Backend UDP port |

## Reference Examples

### Basic Service with Custom Port

```yaml
services:
  app:
    image: myapp:latest
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.app.rule=Host(`app.example.com`)"
      - "traefik.http.routers.app.entrypoints=websecure"
      - "traefik.http.routers.app.tls=true"
      - "traefik.http.routers.app.tls.certresolver=letsencrypt"
      - "traefik.http.services.app.loadbalancer.server.port=3000"
    networks:
      - proxy
```

### Multiple Routers for One Container

```yaml
labels:
  - "traefik.enable=true"
  # API router
  - "traefik.http.routers.api.rule=Host(`api.example.com`) && PathPrefix(`/v1`)"
  - "traefik.http.routers.api.service=api-service"
  - "traefik.http.services.api-service.loadbalancer.server.port=8080"
  # Admin router (different port)
  - "traefik.http.routers.admin.rule=Host(`admin.example.com`)"
  - "traefik.http.routers.admin.service=admin-service"
  - "traefik.http.services.admin-service.loadbalancer.server.port=9090"
```

### With Middleware Chain

```yaml
labels:
  - "traefik.enable=true"
  # Middleware declarations
  - "traefik.http.middlewares.auth.basicauth.users=admin:$$2y$$10$$..."
  - "traefik.http.middlewares.limit.ratelimit.average=100"
  - "traefik.http.middlewares.limit.ratelimit.burst=200"
  - "traefik.http.middlewares.headers.customrequestheaders.X-Forwarded-Proto=https"
  # Router with middleware chain
  - "traefik.http.routers.app.rule=Host(`app.example.com`)"
  - "traefik.http.routers.app.middlewares=auth,limit,headers"
  - "traefik.http.routers.app.tls=true"
  - "traefik.http.routers.app.tls.certresolver=letsencrypt"
  - "traefik.http.services.app.loadbalancer.server.port=80"
```

### Hashing Passwords for BasicAuth

```bash
# Generate a bcrypt hash for BasicAuth
htpasswd -nbB admin "my-password" | sed -e 's/\$/\$\$/g'
# Output: admin:$$2y$$05$$...
# Note: $$ escaping is required in Docker Compose files
```

### Docker Network Selection

When a container is connected to multiple Docker networks, specify the correct one:

```yaml
networks:
  - proxy       # Network Traefik is on
  - internal    # Network for backend communication

labels:
  - "traefik.docker.network=proxy"
```

### Constraints

Filter which containers Traefik discovers:

```yaml
# Static config
providers:
  docker:
    constraints: "Label(`traefik.enable`, `true`) && !Label(`internal`, `true`)"
```

### Port Detection Rules

Traefik automatically determines the backend port:

1. If a container exposes **one** port, Traefik uses it
2. If it exposes **multiple** ports, Traefik uses the **lowest**
3. Always set `loadbalancer.server.port` to be explicit

### Security Note: Docker Socket

Mounting the Docker socket gives the container root-equivalent access to the host. For production:
- Use [docker-socket-proxy](https://github.com/Tecnativa/docker-socket-proxy) for read-only access
- Or expose the socket over SSH/TCP with TLS client certificates
- Run Traefik as a non-root user when possible (use `--users` flag)

# HTTP Routing Reference

HTTP routers match incoming requests against rules and forward them to services through optional middleware chains.

## Rule Matchers

Rules are defined using backtick-delimited values. Multiple matchers combine with `&&` (AND), `||` (OR), `!` (NOT), and parentheses.

### Host and HostRegexp

```yaml
# Exact host match
rule: "Host(`example.com`)"

# Wildcard (single-level, v3 syntax only)
rule: "Host(`*.example.com`)"     # matches foo.example.com, NOT foo.bar.example.com

# Catch-all
rule: "Host(`*`)"                 # matches every request

# Regex-based match
rule: "HostRegexp(`^.+\\.example\\.com$`)"
```

Rules for `Host` and `HostRegexp`:
- Wildcard `*.example.com` matches exactly one subdomain label
- A bare `*` is a catch-all (not a subdomain wildcard)
- Host matchers are case-insensitive
- Non-ASCII domains must use punycode encoding
- If no `Host` header exists, Traefik checks the request URL's host

### Path, PathPrefix, and PathRegexp

```yaml
# Exact path
rule: "Path(`/api/v1/users`)"

# Path prefix
rule: "PathPrefix(`/api`)"

# Regex path
rule: "PathRegexp(`^/api/v[0-9]+/users$`)"
```

- Paths always start with `/`, except for `PathRegexp`
- `Path` matches exact path only
- `PathPrefix` matches any path starting with the prefix

### Header and HeaderRegexp

```yaml
rule: "Header(`Content-Type`, `application/json`)"
rule: "Header(`X-Api-Key`, `secret`)"

# Case-insensitive regex match
rule: "HeaderRegexp(`Content-Type`, `(?i)^application/(json|yaml)$`)"
```

### Method

```yaml
rule: "Method(`GET`)"
rule: "Method(`GET`, `POST`, `PUT`)"
```

### Query and QueryRegexp

```yaml
rule: "Query(`page`, `1`)"
rule: "QueryRegexp(`version`, `^v\\d+$`)"
```

### ClientIP

```yaml
rule: "ClientIP(`10.0.0.0/24`)"
rule: "ClientIP(`192.168.1.100`)"
```

- Matches the actual client IP, NOT the `X-Forwarded-For` header
- Supports IPv4, IPv6, and CIDR notation

### Combined Rules

```yaml
# Complex expressions with logical operators
rule: "Host(`api.example.com`) && PathPrefix(`/v2`)"
rule: "Host(`app.example.com`) && (Method(`GET`) || Method(`POST`))"
rule: "!(Host(`internal.example.com`)) && PathPrefix(`/public`)"
rule: "Host(`example.com`) && Header(`X-Region`, `us-east`)"
```

## Router Configuration

```yaml
# Dynamic config (File provider YAML)
http:
  routers:
    api:
      rule: "Host(`api.example.com`) && PathPrefix(`/v1`)"
      entryPoints:
        - "websecure"
      middlewares:
        - "auth"
        - "ratelimit"
      service: "api-backend"
      tls:
        certResolver: "letsencrypt"
        options: "mytlsoptions@file"
        domains:
          - main: "api.example.com"
      priority: 100
      ruleSyntax: "v3"        # v3 or v2 (default: v3)
      observability:
        metrics: true
        accessLogs: true
        tracing: true
```

## Priority

Routers are sorted by priority (highest first). By default, priority equals the **length of the rule string**. Longer rules get higher priority.

```yaml
# Explicit priority overrides the default
http:
  routers:
    specific:
      rule: "Host(`foobar.example.com`)"
      priority: 100            # Will be matched before the generic one below
    generic:
      rule: "HostRegexp(`[a-z]+\\.example\\.com`)"
      priority: 10
```

Priority rules:
- Explicit `priority: 0` is IGNORED (uses default length-based sorting)
- Negative priorities are supported
- Max user priority: `MaxInt32 - 1000` for 32-bit, `MaxInt64 - 1000` for 64-bit
- Positive priority = higher number wins
- When routers from DIFFERENT providers have the same priority, `providers.precedence` decides

## Multi-Layer Routing

Traefik v3 supports multi-layer routing — splitting the request flow across two routers at different layers. The first router matches and middleware-runs at one `entryPoint`, then the second router at another entryPoint handles the same request for deeper routing.

```yaml
# Layer 1: External entry point handles TLS and auth
http:
  entryPoints:
    websecure:
      address: ":443"
      http:
        tls: true
        middlewares:
          - "ratelimit@file"
          - "ipallowlist@file"

  routers:
    router1:                    # On websecure: TLS termination, rate limit,
      rule: "Host(`api.example.com`)"
      entryPoints: ["websecure"]
      middlewares: ["cors", "auth"]
      service: "router2@internal"  # Passes to router2 on internal entrypoint

    router2:                    # On internal: no TLS, deeper path routing
      rule: "PathPrefix(`/api/v2`)"
      entryPoints: ["internal"]
      service: "v2-backend"

    router3:
      rule: "PathPrefix(`/api/v1`)"
      entryPoints: ["internal"]
      service: "v1-backend"
```

## Rules Syntax: v3 vs v2

The v3 rule syntax is the default. Key changes from v2:

- **Wildcard support** in `Host()` matcher — `Host(`*.example.com`)` works natively (was `HostRegexp` only)
- **Deprecated** `ruleSyntax` option per-router — use the `core.defaultRuleSyntax` global option to override

## Notes

- Single quotes `'` are NOT accepted in rule values — use backticks `` ` `` or escaped double quotes `\"`
- Regex values use Go's `regexp` package syntax
- The character `@` is not allowed in router names
- Routers can reference services from other providers using the `@provider` suffix (e.g., `api@internal` for the Traefik API service)

# Kubernetes Provider Reference

Traefik acts as a fully-featured Kubernetes Ingress controller. It supports three Kubernetes provider modes: standard Ingress, CRD (Custom Resource Definitions), and Gateway API.

## Architecture Overview

When running inside Kubernetes, Traefik auto-detects the cluster:
- Reads `KUBERNETES_SERVICE_HOST` and `KUBERNETES_SERVICE_PORT` env vars
- Uses service account token from `/var/run/secrets/kubernetes.io/serviceaccount/token`
- Uses CA cert from `/var/run/secrets/kubernetes.io/serviceaccount/ca.crt`
- Each provider watches for Kubernetes resource events (create, update, delete) in real-time

For external-cluster access, set `endpoint` to the API server URL.

## Provider: Kubernetes Ingress

Standard Kubernetes Ingress controller. Uses the native `Ingress` resource with annotations for Traefik-specific behavior.

### Static Configuration

```yaml
providers:
  kubernetesIngress:
    endpoint: ""                          # In-cluster if empty
    namespaces: []                        # Watch specific namespaces (empty = all)
    ingressClass: "traefik"              # IngressClass to handle
    labelSelector: ""                     # Filter Ingress objects by labels
    throttleDuration: 0s
    allowEmptyServices: false
    allowExternalNameServices: false
    nativeLBByDefault: false
    disableClusterScopeResources: false   # Set true if RBAC limited
    strictPrefixMatching: false           # K8s-compliant prefix matching
    ingressEndpoint:
      ip: ""
      hostname: ""
      publishedService: ""                # namespace/service for status propagation
```

### Annotations Reference

Apply to `Ingress` resources with `traefik.ingress.kubernetes.io/` prefix:

| Annotation | Description |
|------------|-------------|
| `router.entrypoints` | EntryPoints (comma-separated, default: web) |
| `router.middlewares` | Middleware references (e.g., `namespace-middlewarename@kubernetescrd`) |
| `router.priority` | Router priority integer |
| `router.tls` | Enable TLS (`true`) |
| `router.tls.options` | TLS options reference |
| `router.tls.certresolver` | ACME cert resolver name |
| `service.serverstransport` | ServersTransport reference |
| `service.nativelb` | Use K8s-native load balancing |
| `service.passhostheader` | Pass Host header to backend (default: true) |
| `service.sticky.cookie.name` | Sticky session cookie name |

### Example Ingress Resource

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: whoami
  namespace: apps
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    traefik.ingress.kubernetes.io/router.tls: "true"
    traefik.ingress.kubernetes.io/router.tls.certresolver: letsencrypt
spec:
  ingressClassName: traefik
  rules:
    - host: whoami.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: whoami
                port:
                  number: 80
  tls:
    - secretName: whoami-tls
```

## Provider: Kubernetes CRD

Uses Traefik's custom resources (`IngressRoute`, `Middleware`, `TLSOption`, etc.) for richer configuration than standard Ingress.

### Static Configuration

```yaml
providers:
  kubernetesCRD:
    endpoint: ""
    namespaces: []
    ingressClass: "traefik"
    labelSelector: ""
    throttleDuration: 0s
    allowEmptyServices: false
    allowCrossNamespace: false             # Allow IngressRoutes to reference other namespaces
    allowExternalNameServices: false
    nativeLBByDefault: false
    disableClusterScopeResources: false
    crossProviderNamespaces: []            # Namespaces allowed for cross-provider refs
```

### Custom Resource Definitions (CRDs)

Install CRDs before using:

```bash
kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v3.7/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml
kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v3.7/docs/content/reference/dynamic-configuration/kubernetes-crd-rbac.yml
```

Available CRDs:

| CRD | API Version | Purpose |
|-----|-------------|---------|
| `IngressRoute` | `traefik.io/v1alpha1` | HTTP routing (routers, services, middlewares) |
| `IngressRouteTCP` | `traefik.io/v1alpha1` | TCP routing |
| `IngressRouteUDP` | `traefik.io/v1alpha1` | UDP routing |
| `Middleware` | `traefik.io/v1alpha1` | Middleware configuration |
| `MiddlewareTCP` | `traefik.io/v1alpha1` | TCP middleware configuration |
| `TLSOption` | `traefik.io/v1alpha1` | TLS options (minVersion, cipherSuites, etc.) |
| `TLSStore` | `traefik.io/v1alpha1` | Certificate stores |
| `TraefikService` | `traefik.io/v1alpha1` | Advanced service types (WRR, mirroring) |
| `ServersTransport` | `traefik.io/v1alpha1` | Backend transport config |
| `ServersTransportTCP` | `traefik.io/v1alpha1` | TCP backend transport config |

### IngressRoute Example

```yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: my-app
  namespace: apps
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`app.example.com`) && PathPrefix(`/api`)
      kind: Rule
      services:
        - name: api-service
          port: 8080
      middlewares:
        - name: auth
        - name: ratelimit
  tls:
    certResolver: letsencrypt
```

### Middleware CRD Example

```yaml
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: auth
  namespace: apps
spec:
  basicAuth:
    secret: auth-credentials    # Kubernetes Secret with users field
```

### Key CRD Features

- **Cross-namespace references**: IngressRoutes can reference Middlewares, Services, etc. in other namespaces when `allowCrossNamespace: true`
- **Provider namespace references**: Use `@kubernetescrd`, `@file`, `@docker` suffix to reference resources from other providers
- **TLS per route**: Each route in an IngressRoute can have its own TLS configuration

## Provider: Kubernetes Gateway API

Implements the Kubernetes Gateway API specification (Standard v1.5.1+). Supports `HTTPRoute`, `GRPCRoute`, `TLSRoute`, and experimental `TCPRoute`.

### Static Configuration

```yaml
providers:
  kubernetesGateway:
    endpoint: ""
    experimentalChannel: false             # Enable TCPRoute, experimental features
    namespaces: []
    labelSelector: ""
    throttleDuration: 0s
    nativeLBByDefault: false
    qps: 50                                # Max queries per sec to K8s API
    burst: 100                             # Max burst to K8s API
    statusAddress:
      ip: ""
      hostname: ""
      service:
        name: ""
        namespace: ""
    crossProviderNamespaces: []
```

### Prerequisites

```bash
# Install Gateway API CRDs (Standard channel)
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.1/standard-install.yaml

# Install Traefik RBAC for Gateway API (if not using Helm)
kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v3.7/docs/content/reference/dynamic-configuration/kubernetes-gateway-rbac.yml
```

### Gateway API Example

```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: traefik
spec:
  controllerName: traefik.io/gateway-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: traefik-gateway
  namespace: default
spec:
  gatewayClassName: traefik
  listeners:
    - name: https
      protocol: HTTPS
      port: 443
      tls:
        mode: Terminate
        certificateRefs:
          - kind: Secret
            name: wildcard-example-tls
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: app-route
  namespace: default
spec:
  parentRefs:
    - name: traefik-gateway
  hostnames:
    - app.example.com
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /api
      backendRefs:
        - name: api-service
          port: 8080
```

### Supported Resources

| Resource | Channel | Notes |
|----------|---------|-------|
| `GatewayClass` | Standard | Controller: `traefik.io/gateway-controller` |
| `Gateway` | Standard | HTTP, HTTPS, TCP, TLS listeners |
| `HTTPRoute` | Standard | Hostname matching, path routing, backend refs |
| `GRPCRoute` | Standard | gRPC service routing |
| `TLSRoute` | Standard | TLS passthrough routing |
| `TCPRoute` | Experimental | TCP port-based routing |
| `BackendTLSPolicy` | Standard | Backend TLS configuration |

## Helm Chart Deployment

The Traefik Helm Chart is the recommended installation method for Kubernetes:

```bash
helm repo add traefik https://traefik.github.io/charts
helm install traefik traefik/traefik \
  --namespace traefik \
  --create-namespace \
  --set providers.kubernetesIngress.enabled=true \
  --set providers.kubernetesCRD.enabled=true \
  --set ingressClass.enabled=true
```

Default entryPoints created by Helm: `web` (80), `websecure` (443), `traefik` (8080), `metrics` (9100).

## Resource Namespacing

In Kubernetes, resources must be referenced with their namespace. The format is:

```
<resource-name>.<namespace>@<provider>
```

Examples:
- `my-middleware.apps@kubernetescrd` — Middleware in the `apps` namespace
- `my-tlsoption.default@kubernetescrd` — TLSOption in `default` namespace
- `my-service@file` — Service from the File provider

# Middleware Catalog

Middleware modifies requests before they reach services or modifies responses before they reach clients. Middleware attaches to routers (applied to matching requests) or services (applied to requests handled by that service). Router middlewares execute before service middlewares.

## Path Modifiers

### AddPrefix

Adds a path prefix before forwarding to the backend.

```yaml
http:
  middlewares:
    api-prefix:
      addPrefix:
        prefix: "/api/v2"
```

**Example:** Request to `/users` becomes `/api/v2/users` to the backend.

### StripPrefix

Removes matching path prefixes before forwarding.

```yaml
http:
  middlewares:
    strip-api:
      stripPrefix:
        prefixes:
          - "/api"
          - "/v1"
        forceSlash: false    # If true, ensures trailing slash after stripping
```

**Example:** `/api/users` → `/users`, `/v1/products` → `/products`

### StripPrefixRegex

Removes path portions matching a regex.

```yaml
http:
  middlewares:
    strip-version:
      stripPrefixRegex:
        regex:
          - "/[a-z]+/[0-9]+"
```

### ReplacePath

Replaces the entire path with a fixed value.

```yaml
http:
  middlewares:
    replace:
      replacePath:
        path: "/fallback"
```

### ReplacePathRegex

Replaces path portions using regex capture groups.

```yaml
http:
  middlewares:
    rewrite-api:
      replacePathRegex:
        regex: "^/api/v1/(.*)"
        replacement: "/v2/$1"
```

## Redirects

### RedirectScheme

Redirects requests based on the scheme.

```yaml
http:
  middlewares:
    https-redirect:
      redirectScheme:
        scheme: https
        port: "443"
        permanent: true
```

### RedirectRegex

Redirects using regex matching and replacement on the entire URL.

```yaml
http:
  middlewares:
    domain-redirect:
      redirectRegex:
        regex: "^http://old-domain.com/(.*)"
        replacement: "https://new-domain.com/$1"
        permanent: true
```

## Security & Authentication

### BasicAuth

HTTP Basic Authentication with bcrypt passwords.

```yaml
http:
  middlewares:
    auth:
      basicAuth:
        users:
          - "admin:$2y$05$..."         # bcrypt hash
          - "user:$apr1$..."           # Apache MD5
        usersFile: "/etc/traefik/auth/.htpasswd"  # Alternative to inline users
        realm: "Traefik"               # Realm sent in WWW-Authenticate header
        headerField: "X-WebAuth-User"  # Inject authenticated user into this header
        removeHeader: true             # Remove Authorization header before backend
```

Generate password hashes:

```bash
# bcrypt (recommended)
htpasswd -nbB admin "password" | sed -e 's/\$/\$\$/g'

# Or use the usersFile with htpasswd
htpasswd -nbB admin "password" >> /etc/traefik/auth/.htpasswd
```

### DigestAuth

HTTP Digest Authentication.

```yaml
http:
  middlewares:
    digest-auth:
      digestAuth:
        users:
          - "user:realm:hash"
        usersFile: "/etc/traefik/auth/.htdigest"
        realm: "Traefik"
        removeHeader: true
        headerField: "X-WebAuth-User"
```

### ForwardAuth

Delegates authentication to an external service.

```yaml
http:
  middlewares:
    ext-auth:
      forwardAuth:
        address: "http://auth-service:8080/verify"  # REQUIRED
        trustForwardHeader: true       # Trust X-Forwarded-* headers
        tls:
          ca: "/etc/traefik/certs/ca.pem"
          cert: "/etc/traefik/certs/cert.pem"
          key: "/etc/traefik/certs/key.pem"
          insecureSkipVerify: false
        tls.caOptional: false          # Client cert not required
        authResponseHeaders:
          X-Auth-User: "X-Auth-User"   # Map response headers to request headers
          X-Auth-Token: "X-Auth-Token"
        authResponseHeadersRegex: "^X-Auth-"  # Regex matching response headers to forward
        authRequestHeaders:
          X-My-Header: "X-Forwarded-Proto"    # Map request headers to forward auth
        authSetHeaders:
          X-Forwarded-User: "{!header.X-Auth-User!}"  # Set headers from auth response
        addAuthCookiesToResponse:
          - "session_token"            # Remove auth cookies from backend response
        maxClientConnectDuration: 30s  # Max time for auth request
        maxBodySize: 0                 # Max body size to forward (0=unlimited)
```

### IPAllowList

Restricts allowed client IPs.

```yaml
http:
  middlewares:
    ip-whitelist:
      ipAllowList:
        sourceRange:
          - "10.0.0.0/8"
          - "192.168.0.0/16"
          - "203.0.113.0/24"
        ipStrategy:
          depth: 1                     # X-Forwarded-For depth to check
          excludedIPs: []              # IPs to exclude from X-Forwarded-For chain
```

### InFlightReq

Limits simultaneous connections.

```yaml
http:
  middlewares:
    concurrency-limit:
      inFlightReq:
        amount: 100                    # Max simultaneous requests
        sourceCriterion:
          requestHost: true            # Track by host
          requestRemoteAddr: true      # Track by remote address
          ipStrategy:
            depth: 1
            excludedIPs: []
```

### RateLimit

Limits request frequency (per-source-IP by default).

```yaml
http:
  middlewares:
    ratelimit:
      rateLimit:
        average: 100                   # Average requests per second (REQUIRED)
        burst: 200                     # Burst size (default: average)
        period: 1s                     # Evaluation period (default: 1s)
        sourceCriterion:
          requestHost: true
          requestRemoteAddr: true
          ipStrategy:
            depth: 1
            excludedIPs: []
        # For Redis-backed distributed rate limiting:
        # Requires Traefik Enterprise or Hub
```

### Distributed RateLimit

Redis-backed rate limiting (requires Traefik Enterprise or Hub).

```yaml
# Distributed rate limiting is a Traefik Hub/Enterprise feature
# Not available in Traefik Proxy OSS
http:
  middlewares:
    dist-ratelimit:
      distributedRateLimit:
        average: 100
        burst: 200
        period: 1s
        sourceCriterion:
          requestRemoteAddr: true
```

### Headers

Modifies request and response headers, and sets security-related headers.

```yaml
http:
  middlewares:
    sec-headers:
      headers:
        # --- Custom Header Modifications ---
        customRequestHeaders:          # Add/override request headers
          X-Scope: "internal"
          X-Forwarded-Proto: "https"
        customResponseHeaders:         # Add/override response headers
          X-Custom: "value"

        # --- CORS Headers ---
        accessControlAllowCredentials: true
        accessControlAllowMethods:
          - "GET"
          - "POST"
          - "PUT"
          - "DELETE"
          - "OPTIONS"
        accessControlAllowOriginList:
          - "https://app.example.com"
          - "https://admin.example.com"
        accessControlAllowOriginListRegex:
          - "^https://[a-z]+\\.example\\.com$"
        accessControlExposeHeaders:
          - "X-Custom-Header"
        accessControlMaxAge: 100
        accessControlAllowHeaders:
          - "Content-Type"
          - "Authorization"
        addVaryHeader: true

        # --- Security Headers ---
        # These set the corresponding HTTP security headers
        hostsProxyHeaders:
          - "X-Forwarded-Host"
        sslRedirect: true              # Redirect HTTP to HTTPS
        sslTemporaryRedirect: true    # Use 302 instead of 301 for SSL redirect
        sslHost: "app.example.com"     # Host in the Location header for SSL redirect
        sslProxyHeaders:
          X-Forwarded-Proto: "https"
        sslForceHost: true             # Force SSL Host header
        stsSeconds: 315360000          # HSTS max-age (seconds)
        stsIncludeSubdomains: true     # HSTS includeSubDomains
        stsPreload: true               # HSTS preload
        forceSTSHeader: true           # Force STS header even for non-TLS
        frameDeny: true                # X-Frame-Options: DENY
        customFrameOptionsValue: ""    # Custom X-Frame-Options value (overrides frameDeny)
        contentTypeNosniff: true       # X-Content-Type-Options: nosniff
        browserXssFilter: true         # X-XSS-Protection: 1; mode=block
        customBrowserXSSValue: ""      # Custom X-XSS-Protection value
        contentSecurityPolicy: "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'"
        contentSecurityPolicyReportOnly: ""
        publicKey: ""                  # HPKP public key (deprecated, avoid)
        referrerPolicy: "strict-origin-when-cross-origin"
        permissionsPolicy: "camera=(), microphone=(), geolocation=()"  # Feature-Policy equivalent
        isDevelopment: false           # Disables Host header checking for dev

        # --- Request/Response Modification ---
        allowedHosts:
          - "example.com"
          - "api.example.com"          # Reject if Host not in this list
        customBrowserXSSValue: ""
        sslProxyHeaders:
          X-Forwarded-Proto: "https"   # Trust this header for SSL detection
```

## Request Lifecycle

### Retry

Retries requests on connection failures.

```yaml
http:
  middlewares:
    retry:
      retry:
        attempts: 3                    # Number of retry attempts (REQUIRED)
        initialInterval: 100ms         # Initial backoff interval
```

- Retries only on network errors, not HTTP status codes
- Requests with body are NOT retried (safety measure for non-idempotent POSTs)

### CircuitBreaker

Prevents requests to unhealthy services.

```yaml
http:
  middlewares:
    cb:
      circuitBreaker:
        expression: "NetworkErrorRatio() > 0.5"  # REQURIED
        checkPeriod: 100ms
        fallbackDuration: 300s         # Time in half-open state before full recovery
        recoveryDuration: 10s          # Time before attempting recovery
```

Available expressions:
- `NetworkErrorRatio()` — ratio of network errors (connection refused, timeout, DNS failure)
- `ResponseCodeRatio(min, max, divisorMin, divisorMax)` — ratio of status codes in range
- `LatencyAtQuantileMS(quantile)` — latency at given quantile (e.g., 50.0 for median)
- `Count40x() / Count50x() / CountGateway()` — HTTP status code counters
- Combine with `&&`, `||`, comparison operators

Examples:

```yaml
expression: "NetworkErrorRatio() > 0.5"          # 50%+ network errors
expression: "ResponseCodeRatio(500, 599, 0, 600) > 0.2"  # 20%+ 5xx
expression: "LatencyAtQuantileMS(50.0) > 5000"   # Median latency > 5s
expression: "NetworkErrorRatio() > 0.1 || ResponseCodeRatio(500, 599, 0, 600) > 0.2"
```

### Buffering

Buffers request/response bodies before forwarding.

```yaml
http:
  middlewares:
    buf:
      buffering:
        maxRequestBodyBytes: 10485760    # 10MB max request body in memory
        memRequestBodyBytes: 2097152     # 2MB in-memory limit before spilling to disk
        maxResponseBodyBytes: 10485760   # 10MB max response body in memory
        memResponseBodyBytes: 2097152    # 2MB in-memory limit before spilling to disk
        retryExpression: "IsNetworkError() && Attempts() < 3"  # Retry conditions
```

### Errors

Defines custom error pages for specific status codes.

```yaml
http:
  middlewares:
    err-pages:
      errors:
        status:
          - "500-599"
          - "400-404"
        service: "error-service@file"   # Service that serves error pages
        query: "/error-pages/{status}.html"  # Query to the error service
```

**How it works:** When a backend returns a status matching the range, the client's request is re-routed to the error service. The error service receives a modified request with the path specified in `query`.

### Compress

Compresses responses using gzip.

```yaml
http:
  middlewares:
    gzip:
      compress:
        excludedContentTypes:           # Don't compress these
          - "text/event-stream"
          - "image/png"
          - "image/jpeg"
        minResponseBodyBytes: 1024      # Minimum size to compress
        defaultEncoding: "gzip"         # or "deflate", "zstd" (v3)
```

### ContentType

Prevents Traefik from auto-detecting Content-Type from response body.

```yaml
http:
  middlewares:
    no-sniff:
      contentType: {}
```

- Just setting this middleware prevents Go's `http.DetectContentType()` from running
- Forces backend to set explicit Content-Type

## Authentication & Protocol

### PassTLSClientCert

Passes the TLS client certificate to the backend via headers.

```yaml
http:
  middlewares:
    pass-cert:
      passTLSClientCert:
        pem: true                             # Pass PEM-encoded cert
        info:
          notAfter: true                      # Include expiry date
          notBefore: true                     # Include issue date
          sans: true                          # Include Subject Alternative Names
          subject:
            commonName: true
            country: true
            domainComponent: true
            locality: true
            organization: true
            organizationalUnit: true
            province: true
            serialNumber: true
          issuer:
            commonName: true
            country: true
            domainComponent: true
            locality: true
            organization: true
            organizationalUnit: true
            province: true
            serialNumber: true
```

### GrpcWeb

Converts gRPC-web requests to gRPC (HTTP/2) for backends.

```yaml
http:
  middlewares:
    grpcweb:
      grpcWeb:
        allowOrigins:
          - "*"
```

### EncodedCharacters

Controls handling of encoded characters in request paths.

```yaml
http:
  middlewares:
    encode-check:
      encodedCharacters:
        allowEncodedSlash: false
        allowEncodedBackSlash: false
        allowEncodedNullCharacter: false
        allowEncodedSemicolon: false
        allowEncodedPercent: false
        allowEncodedQuestionMark: false
        allowEncodedHash: false
```

## Combining Middleware

### Chain

Groups multiple middlewares into a reusable chain.

```yaml
http:
  middlewares:
    standard-chain:
      chain:
        middlewares:
          - "ratelimit"
          - "auth"
          - "sec-headers"

# Reference the chain as a single middleware
http:
  routers:
    app:
      rule: "Host(`app.example.com`)"
      middlewares:
        - "standard-chain"
        - "custom-middleware"          # Chain + additional middleware
```

## Middleware Execution Order

When multiple middlewares are attached to a router, they execute in the order listed. Common ordering patterns:

1. **Rate limiting** — apply limits early before expensive processing
2. **IP allowlist** — block IPs before any processing
3. **Authentication** — verify identity before processing
4. **Headers** — modify headers after identity is established
5. **AddPrefix / StripPrefix** — adjust path before routing
6. **Redirect** — redirect before backend processing
7. **Compress** — compress the response on the way back
8. **Errors** — final error page handling
9. **Retry / CircuitBreaker** — at the service level

## Middleware Scope

Middlewares declared via Docker labels are scoped to that Docker provider. They are NOT accessible from other providers unless qualified with the provider namespace.

```yaml
# Reference a middleware from the "file" provider in a Docker router
labels:
  - "traefik.http.routers.app.middlewares=my-mw@file"

# Reference a middleware from the current provider (Docker)
labels:
  - "traefik.http.routers.app.middlewares=my-mw"
```

When using the File provider, middlewares defined in that file are accessible by name without namespace. To reference Docker-defined middlewares from a File provider router, use the `@docker` suffix.

# v2 to v3 Migration Reference

This covers the key changes when migrating from Traefik v2 to v3. Full details: https://doc.traefik.io/traefik/migrate/v2-to-v3/

## Quick Summary of Breaking Changes

| Change | v2 | v3 |
|--------|-----|-----|
| Default rule syntax | v2 | v3 |
| TLS min version default | TLS 1.0 | TLS 1.2 |
| `Path` matcher wildcards | Supported | Removed (use `PathRegexp` or `PathPrefix`) |
| `insecure` field renamed | N/A | `tls.insecureSkipVerify` on serversTransport |
| X-Forwarded-For behavior | Always appended | Configurable via `forwardedHeaders` |
| Tracing providers | Jaeger, Zipkin, Datadog, Elastic | OpenTelemetry (OTLP) only |
| Content-Type handling | Auto-detected | Must use `ContentType` middleware |
| IPWhiteList renamed | `IPWhiteList` | `IPAllowList` |
| Provider removals | Marathon, Rancher v1, InfluxDB v1, Pilot | Removed entirely |
| `tls.caOptional` | Available | Removed from all providers |
| Deprecated field removals | Various | Strictly enforced — v3 refuses to start with deprecated fields |

## Pre-Migration Audit Checklist

Before upgrading, run through these checks:

- [ ] **Rule syntax audit** — Search config files for `HostRegexp` with wildcard patterns (`.+\.`); replace with `Host(`*.`)`
- [ ] **Path wildcard audit** — Search for curly-brace path patterns like `/api/{version:...}`; these need `PathRegexp` replacements
- [ ] **TLS compatibility check** — Verify clients support TLS 1.2+ (default changed from TLS 1.0)
- [ ] **ServersTransport rename** — Replace all `insecure: true` with `insecureSkipVerify: true`
- [ ] **Tracing migration** — If using Jaeger/Zipkin/Datadog/Elastic APM, migrate to OpenTelemetry
- [ ] **ContentType audit** — If you relied on Traefik auto-detecting Content-Type, add the `ContentType` middleware
- [ ] **IPWhiteList → IPAllowList** — Rename any `ipWhiteList` config blocks to `ipAllowList`
- [ ] **Remove deprecated providers** — If using Marathon, Rancher, InfluxDB v1 metrics, or Pilot, find alternatives
- [ ] **Default certificate** — If you relied on SNI fallback, configure a default certificate explicitly
- [ ] **Test in staging** — Run a v3 instance alongside v2 with `core.defaultRuleSyntax: v2` for gradual migration

## Rule Syntax Changes

The v3 rule syntax is the default. If you have v2 rules, they'll get a deprecation warning.

### Host Wildcard

```yaml
# v2 — required HostRegexp for wildcards
rule: "HostRegexp(`.+\\.example\\.com`)"

# v3 — native wildcard support (preferred)
rule: "Host(`*.example.com`)"     # matches foo.example.com, NOT foo.bar.example.com
```

### Path Wildcards Removed

```yaml
# v2 — Path supported wildcards
rule: "Path(`/api/{version: v[0-9]+}/users`)"

# v3 — Path no longer supports wildcards, use PathRegexp
rule: "PathRegexp(`/api/v[0-9]+/users`)"
```

### Migrating Rules

To use v2 syntax during migration, set per-router:

```yaml
http:
  routers:
    legacy-router:
      rule: "HostRegexp(`.+\\.example\\.com`)"
      ruleSyntax: "v2"              # Per-router override
```

Or globally in static config:

```yaml
core:
  defaultRuleSyntax: "v2"
```

## TLS Changes

```yaml
# v3 default TLS security is higher
tls:
  options:
    default:
      # minVersion default changed from VersionTLS10 to VersionTLS12
      minVersion: VersionTLS12
```

### TLS Certificate Fallback

In v3, when no TLS certificate matches the requested SNI, Traefik returns a certificate error instead of falling back to a default certificate. Configure a default certificate explicitly:

```yaml
tls:
  stores:
    default:
      defaultCertificate:
        certFile: "/certs/default.pem"
        keyFile: "/certs/default-key.pem"
```

## Tracing Changes (OTLP Only)

v3 removed direct support for Jaeger, Zipkin, Datadog APM, and Elastic APM tracing. All tracing now goes through OpenTelemetry (OTLP):

```yaml
# v3 — OTLP only
tracing:
  serviceName: "traefik"
  sampleRate: 1.0
  otlp:
    grpc:
      endpoint: "localhost:4317"
      insecure: true
    http:
      endpoint: "localhost:4318"
```

Migrate from Jaeger/Zipkin agents by running an OpenTelemetry Collector as a sidecar that forwards to your existing backend.

## Content-Type Handling

v3 removed automatic Content-Type detection. Responses without an explicit Content-Type header may be handled differently:

```yaml
# v3 — add this middleware to preserve v2 behavior
http:
  middlewares:
    auto-content-type:
      contentType: {}
```

## ServersTransport Changes

The `insecure` field has been renamed for clarity:

```yaml
# v2 — ambiguous
serversTransport:
  insecure: true

# v3 — explicit
serversTransport:
  insecureSkipVerify: true
```

## Docker Provider Changes

```yaml
# v2 — no port spec means ambiguous
traefik.http.services.my-svc.loadbalancer.server.port=80

# v3 — same behavior, but port detection priority updated.
# Always set the port explicitly.
```

## X-Forwarded-For Header

v3 provides more granular control over forwarded headers:

```yaml
entryPoints:
  web:
    address: ":80"
    forwardedHeaders:
      trustedIPs:
        - "10.0.0.0/8"
        - "172.16.0.0/12"
      insecure: false
    # When set to true, Traefik will NOT append client's RemoteAddr to X-Forwarded-For
```

## Provider Removals

The following providers were removed in v3 with no replacement:

| Removed Provider | Alternative |
|-----------------|-------------|
| Marathon | Use File provider or migrate to Kubernetes/Nomad |
| Rancher v1 | Use File provider or Rancher v2's Kubernetes ingress |
| InfluxDB v1 metrics | Use InfluxDB v2 or Prometheus |
| Pilot | Use Traefik Hub or forward auth middlewares |

## Migration Steps

1. **Run the pre-migration audit** — check all 10 items above before changing anything
2. **Update static config** — review all entryPoints, providers, and TLS settings
3. **Update rule syntax** — replace `HostRegexp` wildcards with `Host(*)`, remove wildcards from `Path` matchers
4. **Check TLS defaults** — verify min TLS version compatibility with clients
5. **Update ServersTransport** — rename `insecure` to `insecureSkipVerify`
6. **Migrate tracing** — switch from direct Jaeger/Zipkin to OpenTelemetry
7. **Test with staging** — use `core.defaultRuleSyntax: v2` during migration and test each router
8. **Add default certificate** — if you relied on the TLS fallback behavior
9. **Run a parallel stack** — deploy v3 alongside v2, redirect a subset of traffic, verify all routes work
10. **Cut over** — switch production traffic to v3, keep v2 as rollback target for 48 hours

# Observability Reference

Traefik provides comprehensive observability: metrics, access logs, tracing, and health checks.

## Metrics

Configure metrics collection in static configuration:

```yaml
metrics:
  addInternals: false         # Include metrics for Traefik's own services

  # --- Prometheus (most common) ---
  prometheus:
    buckets:
      - 0.1
      - 0.3
      - 1.2
      - 5.0                    # Request latency buckets (seconds)
    addEntryPointsLabels: true
    addRoutersLabels: true
    addServicesLabels: true
    entryPoint: "metrics"       # Dedicated entryPoint for metrics scraping
    manualRouting: false        # Set true to create your own router
    headerLabels:               # Additional labels from request headers
      X-Custom: "custom_label"

  # --- Datadog ---
  datadog:
    address: "localhost:8125"
    pushInterval: 10s
    addEntryPointsLabels: true
    addRoutersLabels: true
    addServicesLabels: true
    prefix: "traefik"

  # --- StatsD ---
  statsD:
    address: "localhost:8125"
    pushInterval: 10s
    addEntryPointsLabels: true
    addRoutersLabels: true
    addServicesLabels: true
    prefix: "traefik"

  # --- InfluxDB v2 ---
  influxDB2:
    address: "http://localhost:8086"
    token: "my-token"
    pushInterval: 10s
    org: "my-org"
    bucket: "traefik"
    addEntryPointsLabels: true
    addRoutersLabels: true
    addServicesLabels: true
    additionalLabels: {}

  # --- OpenTelemetry ---
  otlp:
    grpc:
      endpoint: "localhost:4317"
      insecure: true
    http:
      endpoint: "localhost:4318"
    pushInterval: 10s
    addEntryPointsLabels: true
    addRoutersLabels: true
    addServicesLabels: true
    explicitBoundaries:
      - 0.0
```

### Prometheus Scraping Setup

```yaml
# Static config — dedicated entryPoint for metrics
entryPoints:
  metrics:
    address: ":9100"

metrics:
  prometheus:
    entryPoint: "metrics"
    addRoutersLabels: true
    addServicesLabels: true
```

```bash
# Verify Prometheus metrics
curl http://localhost:9100/metrics | grep traefik
```

### Key Prometheus Metrics

| Metric | Type | Description |
|--------|------|-------------|
| `traefik_http_requests_total` | Counter | Total request count |
| `traefik_http_request_duration_seconds` | Histogram | Request duration |
| `traefik_http_requests_in_flight` | Gauge | Current in-flight requests |
| `traefik_backend_server_up` | Gauge | Backend server health (0/1) |
| `traefik_config_reloads_total` | Counter | Config reload count |
| `traefik_config_last_reload_success` | Gauge | Last reload success (0/1) |
| `traefik_tls_certs_not_after` | Gauge | TLS cert expiry timestamp |
| `traefik_entrypoint_open_connections` | Gauge | Open connections per entryPoint |
| `traefik_entrypoint_request_duration_seconds` | Histogram | Per-entryPoint request duration |

## Access Logs

Traefik can log every request (similar to Apache/Nginx access logs). Configure in static config:

```yaml
accessLog:
  filePath: "/var/log/traefik/access.log"   # File path (stdout if empty)
  format: "common"                          # common or json
  bufferingSize: 0                          # Buffer N lines (0=unbuffered)
  addInternals: false                       # Log Traefik's internal requests

  filters:
    statusCodes:
      - "200-299"
      - "400-499"
      - "500-599"                           # Only log these status code ranges
    retryAttempts: true                     # Only log requests that were retried
    minDuration: "10s"                      # Only log requests over this duration

  fields:
    defaultMode: "keep"                     # keep, drop, redact
    names:
      ClientHost: "keep"
      ClientPort: "drop"                    # Per-field override — drop client port
      RequestHost: "keep"
      RequestPath: "keep"
      RequestMethod: "keep"
      RequestProtocol: "keep"
      ResponseStatus: "keep"
      Duration: "keep"
      RetryAttempts: "keep"

    headers:
      defaultMode: "drop"                   # By default, don't log headers
      names:
        Authorization: "redact"             # Redact auth header values
        User-Agent: "keep"                  # But keep user-agent
        Referer: "keep"
```

### Access Log Fields

| Field Name | Description |
|-----------|-------------|
| `ClientHost` | Client IP address |
| `ClientPort` | Client port |
| `ClientUsername` | Authenticated username |
| `RequestHost` | Requested host |
| `RequestPath` | Request path |
| `RequestMethod` | HTTP method |
| `RequestProtocol` | HTTP protocol version |
| `RequestContentSize` | Request body size |
| `RequestLine` | Full request line |
| `ResponseStatus` | HTTP status code |
| `ResponseContentSize` | Response body size |
| `Duration` | Request duration |
| `OriginDuration` | Duration at origin |
| `RouterName` | Router that matched |
| `ServiceName` | Service that handled request |
| `ServiceURL` | Backend URL that handled request |
| `DownstreamStatus` | Downstream connection status |
| `StartUTC` | Request start time (UTC) |
| `StartLocal` | Request start time (local) |
| `RetryAttempts` | Number of retry attempts |
| `FrontendName` | (deprecated) Legacy router name |

### Common Log Format

```
<ClientHost> - <ClientUsername> [<StartUTC>] "<RequestLine>" <ResponseStatus> <ResponseContentSize> "<RequestRefererHeader>" "<RequestUserAgentHeader>" <Duration> <RequestCount>
```

Real example:
```
192.168.1.100 - - [05/Jul/2026:10:15:30 +0000] "GET /api/users HTTP/2" 200 1234 "-" "curl/8.0" 0.045 1
```

### JSON Log Format

```json
{
  "ClientHost": "192.168.1.100",
  "ClientPort": 54321,
  "StartUTC": "2026-07-05T10:15:30Z",
  "RequestMethod": "GET",
  "RequestPath": "/api/users",
  "RequestProtocol": "HTTP/2.0",
  "ResponseStatus": 200,
  "ResponseContentSize": 1234,
  "Duration": 45000000,
  "RouterName": "api-router",
  "ServiceName": "api-backend",
  "ServiceURL": "http://10.0.0.5:3000",
  "RetryAttempts": 0
}
```

## Tracing

Traefik supports OpenTelemetry tracing:

```yaml
tracing:
  serviceName: "traefik"
  sampleRate: 0.1                           # Sample 10% of requests (0.0 to 1.0)
  addInternals: false
  globalAttributes:
    environment: "production"

  # OpenTelemetry Protocol (OTLP)
  otlp:
    grpc:
      endpoint: "localhost:4317"
      insecure: true
    http:
      endpoint: "localhost:4318"

  # Headers to capture in spans
  capturedRequestHeaders:
    - "User-Agent"
    - "X-Request-Id"
  capturedResponseHeaders:
    - "Content-Type"
  safeQueryParams:                           # Query params NOT redacted
    - "id"
    - "page"
```

Tracing verbosity:
- **minimal** (default): One server span, one client span per request
- **detailed**: Additional spans for each middleware

```yaml
entryPoints:
  websecure:
    address: ":443"
    observability:
      tracing: true
      traceVerbosity: detailed              # Or: minimal
```

## Health Check (Ping)

```yaml
ping:
  entryPoint: "web"                         # EntryPoint for /ping
  manualRouting: false
  terminatingStatusCode: 503                # Return this during graceful shutdown
```

The ping endpoint returns:
- `200 OK` — Traefik is healthy and accepting requests
- `503 Service Unavailable` — Traefik is shutting down (graceful termination)

```bash
# Health check
curl -s -o /dev/null -w "%{http_code}" http://localhost:80/ping

# With Docker healthcheck
healthcheck:
  test: ["CMD", "wget", "-q", "-O-", "http://localhost:80/ping"]
  interval: 30s
  timeout: 3s
  retries: 3
```

## Per-Router Observability Control

Router-level overrides for observability:

```yaml
http:
  routers:
    internal-only:
      rule: "Host(`internal.example.com`)"
      service: "internal-backend"
      observability:
        metrics: false                       # Don't emit metrics for this router
        accessLogs: true                     # Do log this router
        tracing: false                       # Don't trace this router
```

# Traefik Operational Audit

Full-stack audit methodology for evaluating a running Traefik deployment. Covers what to check, which commands to run, and how to classify findings.

## Audit Flow

Run these phases in order. Each builds on the previous. **Shortcut:** Start with Phase 4 — `/api/rawdata` returns the complete runtime state (routers, middlewares, services, TCP middlewares) in one JSON blob. Cross-reference what's actually running against what the static/dynamic config claims, then work backward through phases to trace discrepancies. A full audit takes ~15 minutes starting from rawdata vs ~40 minutes starting from static config — and catches runtime-only issues like duplicate middleware applications that static config review misses.

### Phase 1 — Surface Inventory

Identify what's running and where config lives.

```bash
# What's running
ssh $HOST "docker ps --filter name=traefik --format '{{.Names}} {{.Image}} {{.Status}}'"

# Compose file location
ssh $HOST "docker compose ls --format json | python3 -c \"import json,sys; [print(p['Name'], p['ConfigFiles']) for p in json.load(sys.stdin) if 'traefik' in p['Name'].lower()]\""

# Restart policy and mounts
ssh $HOST "docker inspect traefik --format '{{json .HostConfig.RestartPolicy}}'"
ssh $HOST "docker inspect traefik --format '{{json .Mounts}}' | python3 -m json.tool"
```

### Phase 2 — Static Configuration

Pull the full static config from the running container.

```bash
ssh $HOST "docker exec traefik cat /etc/traefik/traefik.yml"
```

Key things to check:
- `exposedByDefault: false` (should be false)
- `api.insecure: true` — if true, check whether port 8080 is mapped in compose. Even if not mapped, prefer `false` unless debugging
- `api.dashboard: true` — check whether auth middleware is applied to the dashboard router
- `log.level` — should be `info` or `warn` in production, not `debug`
- `certificatesResolvers` — DNS-01 preferred for homelab/internal services
- Entrypoint-level middlewares — apply baseline protection (security headers, rate limiting) to all routers
- `forwardedHeaders.trustedIPs` — should include LAN and any upstream proxy ranges

### Phase 3 — Dynamic Configuration

List all files loaded by the file provider and pull their contents.

```bash
# List dynamic config files
ssh $HOST "docker exec traefik ls /etc/traefik/dynamic/"

# Pull all dynamic configs
ssh $HOST "docker exec traefik sh -c 'for f in /etc/traefik/dynamic/*.yml /etc/traefik/dynamic/*.yaml; do echo \"--- \$f ---\"; cat \$f; done'"
```

Key things to check:
- **Extension-less files are silently ignored** — `providers.file.directory` only watches `.yml`, `.yaml`, and `.toml`. A file named `default` (no extension) is dead config. Check for these: `ls /etc/traefik/dynamic/ | grep -v '\\.'`. If found, either delete or rename with an extension.
- **Naming collisions across files** — if two files define the same middleware name (e.g., `default` as a chain in `config.yml` and `default` as bare headers in another file), the last file alphabetically wins and silently overwrites the earlier definition. This can replace a middleware chain with a single middleware. Cross-reference middleware names across all dynamic files.
- Duplicate router names across files — undefined behavior
- Middleware definitions — chains, forwardAuth, rate limiting, security headers
- Backend URLs — hardcoded IPs are single points of failure
- Inconsistent indentation — doesn't affect functionality but signals maintenance debt

### Phase 4 — Runtime State via API

Query the API for the actual running router/middleware/service state. **Prefer `/api/rawdata`** — it returns a single JSON blob with all routers, middlewares, services, and TCP middlewares. This is the single most useful endpoint for auditing and resolves ambiguity from static/dynamic config review.

```bash
# Complete runtime state (routers, middlewares, services, TCP middlewares)
ssh $HOST "docker exec traefik wget -qO- http://localhost:8080/api/rawdata | python3 -m json.tool"

# Inspect a specific router's effective middlewares
ssh $HOST "docker exec traefik wget -qO- http://localhost:8080/api/http/routers/NAME@file | python3 -m json.tool"

# List all middlewares and their usedBy routers
ssh $HOST "docker exec traefik wget -qO- http://localhost:8080/api/rawdata | python3 -c \"
import json, sys
data = json.load(sys.stdin)
for name, mw in data.get('middlewares', {}).items():
    used = mw.get('usedBy', [])
    print(f'{name} → used by: {used[:5]}{\"...\" if len(used) > 5 else \"\"}'  if used else f'{name} → unused')
\""
```

**Duplicate middleware detection:** Look for router middleware arrays like `["default@file","default@file"]`. This means the same middleware is applied both at the entrypoint level (static config) AND via Docker labels or file config on the specific router — the chain executes twice. This wastes cycles and, for rate limiting, doubles the effective limits. The fix: remove the explicit declaration from the service/router config; the entrypoint already provides it.

### Phase 5 — Logs

Check for errors, warnings, and patterns.

```bash
# Recent logs
ssh $HOST "docker logs traefik --tail 50"

# Filter errors and warnings
ssh $HOST "docker logs traefik --since 24h 2>&1 | grep -E 'level\":\"(error|warn)'"
```

Key patterns to recognize:
- `maxResponseBodySize is not configured` — forwardAuth middleware missing size limit; DoS vector
- `context deadline exceeded` on health checks — timeout too short or endpoint doesn't respond
- `server misbehaving` on DNS lookups — container DNS misconfiguration
- `context canceled` on forwardAuth calls — usually client-side disconnect, not a server issue (check container uptime before investigating)
- `Error calling http://oauth:4181` — forwardAuth backend unreachable; check container health

### Phase 6 — Live Verification

Test critical endpoints directly.

```bash
# Dashboard accessibility (should return 401 if auth is configured)
ssh $HOST "curl -sk -o /dev/null -w '%{http_code}' https://traefik.$DOMAIN/dashboard/"

# Health check endpoints for services with health checks configured
ssh $HOST "curl -sv --connect-timeout 5 --max-time 15 http://$BACKEND_IP:$PORT/health 2>&1"

# DNS from inside the container
ssh $HOST "docker exec traefik wget -qO- --timeout=5 https://update.traefik.io 2>&1"
```

## Classification Framework

| Grade | Criteria |
|-------|----------|
| **HIGH** | Security exposure (no auth on sensitive endpoints), active DoS vectors, duplicate router definitions |
| **MEDIUM** | Failing health checks, DNS issues, intermittent errors, stale backups |
| **LOW** | Cosmetic (indentation, stale password files, commented-out features) |

**Rule of thumb for LOW findings:** Ask "does this cause operational harm or security exposure right now?" If no, it's LOW or skip it entirely. A 4-year-old `.htpasswd` file is not a vulnerability — it's a note. Access logs being disabled is a choice, not a bug.

### What to skip

These are NOT problems:
- `tls.yml.old` or backup files outside the `dynamic/` directory (not loaded by file provider)
- `.htpasswd` file age (unless credentials are compromised)
- Inconsistent indentation in YAML (zero functional impact)
- Disabled access logs or tracing (intentional configuration choice)
- `acme.json` modification date within 60 days (LE renews at 30 days before expiry)
- Multiple containers with same labels during rolling deploys (zero-downtime pattern)

## Middleware Audit Scoring

When rating a middleware configuration, score three independent axes. Each axis gets point values for what's present and deductions for what's missing or misconfigured.

### Security (100 pts)

| Check | Points |
|-------|--------|
| TLS configured (ACME or static certs) | +10 |
| HSTS with subdomains + preload | +10 |
| DNS-01 ACME (no port 80 exposed) | +5 |
| `exposedByDefault=false` on Docker provider | +5 |
| Rate limiting at entrypoint covers all services | +10 |
| In-flight request limiting | +5 |
| OAuth/ForwardAuth on sensitive services | +10 |
| Dashboard behind auth (BasicAuth or OAuth) | +5 |
| `forwardedHeaders.trustedIPs` limited to LAN/proxy ranges | +5 |
| IP allowlist for internal-only endpoints | +5 |

| Deduction | Points |
|-----------|--------|
| `api.insecure: true` (host-mapped port 8080) | -15 |
| `api.insecure: true` (container-only, no host mapping) | -8 |
| No CSP configured | -5 |
| No Permissions-Policy configured | -3 |
| No `frameDeny` or `X-Frame-Options` | -3 |
| Access logs disabled | -3 |
| No CrowdSec/fail2ban threat intelligence | -3 |

### Performance (100 pts)

| Check | Points |
|-------|--------|
| HTTP/3 enabled | +10 |
| Compression (gzip + brotli + zstd) | +15 |
| Rate limiting prevents abuse without blocking LAN | +10 |
| In-flight request limiting | +10 |
| Prometheus metrics with custom buckets | +10 |
| JSON structured logging | +5 |
| Health checks on services | +5 |
| CurveP256 preferred over CurveP384 (2-4x faster, equivalent security) | +10 |

| Deduction | Points |
|-----------|--------|
| No `serversTransport` tuning (pool size, timeouts) | -5 |
| No keep-alive tuning | -3 |
| CurveP384 preferred without CurveP256 (slower handshakes) | -5 |
| No passive health checks on most services | -2 |

### Correctness (100 pts)

| Check | Points |
|-------|--------|
| Entrypoint-level middleware applied consistently | +15 |
| Chain order correct (headers → compress → rate-limit → inflightreq) | +15 |
| LAN exclusions on rate-limit and inflightreq | +10 |
| OAuth on appropriate services, not over-applied | +10 |
| Dashboard behind auth | +10 |
| TLS resolver properly configured and referenced | +10 |
| HTTP→HTTPS redirect | +5 |

| Deduction | Points |
|-----------|--------|
| Duplicate middleware in router arrays (`default@file` twice) | -8 |
| Dead config files (extension-less, `.old` in dynamic dir) | -3 |
| Middleware defined but never used (commented out of chain) | -3 |
| Duplicate router definitions across providers | -5 |

## Post-Fix Verification

After applying fixes:
1. Check logs for clean state: `docker logs traefik --since 2m | grep -E 'ERROR|WARN'`
2. Verify auth on protected endpoints: `curl -sk -o /dev/null -w '%{http_code}' $URL` → expect 401
3. Verify no duplicate routers via API
4. For compose label changes, run `docker compose up -d` to re-apply labels (may recreate container)
5. For file provider changes, no restart needed — `watch: true` picks them up

## DNS Debugging Pattern

If the Traefik container shows `server misbehaving` on DNS lookups (127.0.0.11:53):
1. Check Docker daemon DNS config: `cat /etc/docker/daemon.json | python3 -m json.tool`
2. If Tailscale or a VPN is interfering, set explicit DNS: `{"dns": ["1.1.1.1", "8.8.8.8"]}`
3. Restart Docker after daemon.json changes
4. Recreate the Traefik container to pick up new DNS settings (old containers may cache bad DNS)
5. Verify: `docker exec traefik wget -qO- --timeout=5 https://update.traefik.io`

# Other Providers Reference

Beyond Docker and Kubernetes, Traefik supports several other infrastructure providers for automatic service discovery.

## AWS ECS Provider

Discovers services from Amazon ECS clusters using task definition labels.

### Configuration

```yaml
providers:
  ecs:
    clusters: ["default"]                  # Clusters to watch
    autoDiscoverClusters: false            # Auto-discover all clusters
    exposedByDefault: true                 # Auto-expose services (set false for security)
    defaultRule: "Host(`{{ normalize .Name }}`)"
    constraints: ""                        # Label filter expressions
    healthyTasksOnly: false                # Only discover healthy tasks
    ecsAnywhere: false                     # Enable ECS Anywhere support
    refreshSeconds: 15
    region: ""                             # AWS region
    accessKeyID: ""                        # AWS access key (or use IAM role)
    secretAccessKey: ""                    # AWS secret key
```

### Label Syntax

Uses Docker labels on ECS task definitions:

```json
{
  "containerDefinitions": [{
    "name": "my-app",
    "dockerLabels": {
      "traefik.enable": "true",
      "traefik.http.routers.app.rule": "Host(`app.example.com`)",
      "traefik.http.services.app.loadbalancer.server.port": "8080"
    }
  }]
}
```

- Labels use the same syntax as Docker provider labels
- ECS identifies services by task definition labels
- IAM roles preferred over static credentials for production

## HashiCorp Nomad Provider

Discovers services from Nomad using service tags.

### Configuration

```yaml
providers:
  nomad:
    endpoint:
      address: "http://127.0.0.1:4646"
      region: ""                            # Nomad region
      token: ""                             # ACL token
      endpointWaitTime: 0s
      tls:
        ca: ""
        cert: ""
        key: ""
        insecureSkipVerify: false
    namespaces: []                          # Discover services in these namespaces
    prefix: "traefik"                       # Tag prefix
    exposedByDefault: true
    defaultRule: "Host(`{{ normalize .Name }}`)"
    constraints: ""
    allowEmptyServices: false
    refreshInterval: 15s
    stale: false                            # Allow stale reads for performance
    watch: false                            # Watch for events (vs polling)
```

### Tag Syntax

Uses Consul-style tags on Nomad services:

```hcl
service {
  name = "my-app"
  tags = [
    "traefik.http.routers.app.rule=Host(`app.example.com`)",
    "traefik.http.services.app.loadbalancer.server.port=80",
  ]
}
```

## HashiCorp Consul Catalog Provider

Discovers services from Consul's service catalog.

### Configuration

```yaml
providers:
  consulCatalog:
    endpoint:
      address: "127.0.0.1:8500"
      scheme: "http"
      datacenter: ""
      token: ""
      endpointWaitTime: 0s
      tls:
        ca: ""
        cert: ""
        key: ""
        insecureSkipVerify: false
      httpAuth:
        username: ""
        password: ""
    prefix: "traefik"
    exposedByDefault: true
    defaultRule: "Host(`{{ normalize .Name }}`)"
    constraints: ""
    connectAware: false                     # Support Consul Connect
    connectByDefault: false
    serviceName: ""
    refreshInterval: 0s
    requireConsistent: false
    stale: false
    cache: true
    watch: true
    namespaces: []                          # Consul Enterprise namespaces
```

## KV Store Providers (Redis, Consul, etcd, ZooKeeper)

Traefik can read dynamic configuration from key-value stores. The configuration is stored as JSON under a root key.

### Redis

```yaml
providers:
  redis:
    rootKey: "traefik"
    endpoints: ["127.0.0.1:6379"]
    username: ""
    password: ""
    db: 0
    tls:
      ca: ""
      cert: ""
      key: ""
      insecureSkipVerify: false
    sentinel:
      masterName: ""
      username: ""
      password: ""
      latencyStrategy: false
      randomStrategy: false
      replicaStrategy: false
      useDisconnectedReplicas: false
```

### Consul

```yaml
providers:
  consul:
    rootKey: "traefik"
    endpoints: ["127.0.0.1:8500"]
    token: ""
    namespaces: []
    tls:
      ca: ""
      cert: ""
      key: ""
      insecureSkipVerify: false
```

### etcd

```yaml
providers:
  etcd:
    rootKey: "traefik"
    endpoints: ["127.0.0.1:2379"]
    username: ""
    password: ""
    tls:
      ca: ""
      cert: ""
      key: ""
      insecureSkipVerify: false
```

### ZooKeeper

```yaml
providers:
  zooKeeper:
    rootKey: "traefik"
    endpoints: ["127.0.0.1:2181"]
    username: ""
    password: ""
```

### KV Store Data Format

The value at the root key must be a JSON representation of Traefik's dynamic configuration:

```json
{
  "http": {
    "routers": {
      "my-router": {
        "rule": "Host(`example.com`)",
        "service": "my-service"
      }
    },
    "services": {
      "my-service": {
        "loadBalancer": {
          "servers": [{"url": "http://10.0.0.1:80"}]
        }
      }
    }
  }
}
```

## File Provider

For static/dynamic config managed as files (no service discovery). The most commonly used non-discovery provider.

```yaml
providers:
  file:
    directory: "/etc/traefik/dynamic/"    # Watch this directory
    watch: true                            # Auto-reload on changes
    filename: ""                           # Single file (alternative to directory)
    debugLogGeneratedTemplate: false
```

Files must have `.yml`, `.yaml`, or `.toml` extension. They are merged alphabetically.

## HTTP Provider

Fetches dynamic configuration from an HTTP(S) endpoint.

```yaml
providers:
  http:
    endpoint: "http://config-server:8080/traefik-config"
    pollInterval: 5s
    pollTimeout: 30s
    headers:                               # Custom request headers
      Authorization: "Bearer my-token"
    tls:
      ca: ""
      cert: ""
      key: ""
      insecureSkipVerify: false
```

## REST Provider

Accepts dynamic configuration updates via a REST API endpoint.

```yaml
providers:
  rest:
    insecure: false                        # Expose on Traefik's entryPoint
```

The REST provider exposes `PUT /api/providers/rest` to accept dynamic configuration updates.

## Provider Precedence

When multiple providers define the same router/service, the `providers.precedence` option determines which wins:

```yaml
providers:
  precedence:
    - "docker"
    - "kubernetesCRD"
    - "file"
    - "rest"
```

Providers listed first have higher priority. If not configured, the order is: plugin → http → docker/ecs/consulCatalog/nomad/rancher → marathon → kubernetes → rest → file → consul/etcd/zooKeeper/redis.

## Watching Multiple Providers

Traefik can run multiple providers simultaneously. This is common for hybrid setups (Docker + File provider for shared middlewares):

```yaml
providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
  file:
    directory: "/etc/traefik/dynamic/"
    watch: true
```

Resources from different providers are isolated by their provider namespace. Reference cross-provider resources using the `@provider` suffix (e.g., `my-middleware@file`).

# Plugins & Extending Traefik

Traefik supports two plugin systems for extending its capabilities with custom middlewares and providers.

## Plugin Systems Comparison

| Feature | Yaegi | WebAssembly (WASM) |
|---------|-------|-------------------|
| Language | Go | Any language compiling to WASM (Go, Rust, C++, TinyGo) |
| Compilation | Not required (interpreted) | Required (compiled to .wasm binary) |
| Performance | Interpreted | Near-native |
| Hot-reload | Yes | No (binary change requires reload) |
| Safety | Go sandbox | WASM sandbox |
| Extensibility | Middleware + Provider | Middleware only |
| Use case | Rapid development, prototyping | Production-grade, performance-critical |
| Plugin Catalog | Available | Available |

## Plugin Catalog

Browse available plugins: https://plugins.traefik.io/

The Plugin Catalog is accessible from the Traefik Dashboard under the **Plugins** menu entry. Each plugin provides:
- Installation instructions (static config snippet)
- Dynamic configuration syntax
- Usage examples

## Configuring Plugins (Static Config)

### Remote Plugins (from catalog)

```yaml
# Static config
experimental:
  plugins:
    plugin-name:
      moduleName: "github.com/org/plugin"
      version: "v0.1.0"
      # Optional settings (WASM only):
      settings:
        envs:
          - "CACHE_TTL=300"
        mounts:
          - "/data/cache:/cache"
```

### Local Plugins (development)

```yaml
experimental:
  localPlugins:
    plugin-name:
      moduleName: "github.com/org/plugin"
      settings:
        envs:
          - "DEBUG=true"
```

## Using Plugins in Dynamic Config

Plugins are referenced as middlewares with the `@plugin` provider namespace:

```yaml
# Dynamic config (File provider)
http:
  middlewares:
    my-auth:
      plugin:
        auth-plugin-name:
          headerName: "X-API-Key"
          secret: "my-secret"

http:
  routers:
    app:
      rule: "Host(`app.example.com`)"
      middlewares:
        - "my-auth@file"          # File-based middleware
        - "plugin-name@plugin"    # Plugin middleware

  services:
    app:
      loadBalancer:
        servers:
          - url: "http://backend:80"
```

With Docker labels:

```yaml
labels:
  - "traefik.http.middlewares.my-plugin.plugin.plugin-name.key=value"
  - "traefik.http.routers.app.middlewares=my-plugin@plugin"
```

## Plugin Development

### Yaegi Plugin (Go, no compilation)

Create a Go package with a module path and implement the middleware/provider interface. Import pattern:

```go
package myplugin

import (
    "context"
    "net/http"
)

func New(ctx context.Context, conf map[string]interface{}) (func(next http.Handler) http.Handler, error) {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
            // Custom logic
            next.ServeHTTP(rw, req)
        })
    }, nil
}
```

### WASM Plugin (any language)

Compile to a `.wasm` binary following the http-wasm specification (https://http-wasm.io/). HTTP middleware handler with request/response manipulation capability.

### Development Resources

- **Developer docs**: https://plugins.traefik.io/install
- **Plugin creation guide**: https://plugins.traefik.io/create
- **Plugin catalog submission**: Publish to https://plugins.traefik.io/ for community use

## Provider Plugins

Yaegi plugins can extend Traefik with custom providers, enabling dynamic configuration from any source:

```yaml
experimental:
  plugins:
    my-provider-plugin:
      moduleName: "github.com/org/provider-plugin"
      version: "v0.1.0"

providers:
  plugin:
    my-provider-plugin:
      endpoint: "http://custom-source:8080"
      pollInterval: "30s"
```

## Security Considerations

- Plugins run with the same privileges as Traefik itself
- WASM provides stronger isolation (sandboxed by design)
- Only install plugins from trusted sources (verified hashes)
- Test plugins in staging before production deployment
- Plugin hash verification is optional but recommended for remote plugins:

```yaml
experimental:
  plugins:
    plugin-name:
      moduleName: "github.com/org/plugin"
      version: "v0.1.0"
      hash: "sha256:abc123..."
```

## FastProxy (Experimental)

FastProxy is an experimental HTTP/2 multiplexing optimization that reduces connection overhead to backends:

```yaml
# Static config
experimental:
  fastProxy: true
```

FastProxy connects to backend servers using a shared HTTP/2 connection pool, reducing connection churn for services with many concurrent requests. Available in Traefik v3.x experimental channel.

# Production Deployment Reference

Production-ready patterns for deploying Traefik with Docker Compose, including security hardening, high availability considerations, and advanced configurations.

## Production Docker Compose Template

```yaml
version: "3.8"

services:
  traefik:
    image: traefik:v3.7
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE
    networks:
      - proxy
    ports:
      - "80:80"
      - "443:443"
      # Do NOT expose the dashboard port externally — access via router
    environment:
      - CF_DNS_API_TOKEN=${CF_DNS_API_TOKEN}    # DNS challenge env vars
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro   # Read-only socket
      - ./traefik.yml:/etc/traefik/traefik.yml:ro
      - ./letsencrypt:/letsencrypt               # ACME certificate storage
    labels:
      # Dashboard
      - "traefik.enable=true"
      - "traefik.http.routers.dashboard.rule=Host(`traefik.example.com`)"
      - "traefik.http.routers.dashboard.service=api@internal"
      - "traefik.http.routers.dashboard.tls=true"
      - "traefik.http.routers.dashboard.tls.certresolver=letsencrypt"
      - "traefik.http.routers.dashboard.middlewares=dashboard-auth"
      - "traefik.http.middlewares.dashboard-auth.basicauth.users=${DASHBOARD_AUTH}"
```

Full static config (`traefik.yml`):

```yaml
global:
  checkNewVersion: false
  sendAnonymousUsage: false

entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
          permanent: true

  websecure:
    address: ":443"
    http:
      tls:
        certResolver: letsencrypt

providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false                    # Secure: only expose labeled containers
    network: "proxy"

certificatesResolvers:
  letsencrypt:
    acme:
      email: "admin@example.com"
      storage: "/letsencrypt/acme.json"
      httpChallenge:
        entryPoint: "web"
      # For wildcard certs use dnsChallenge instead:
      # dnsChallenge:
      #   provider: "cloudflare"

api:
  dashboard: false                             # Dashboard disabled in static config
  # We create the dashboard router dynamically
```

## Security Hardening

### Docker Socket Security

**Never mount the Docker socket directly in production without precautions:**

Option 1: Read-only socket mount (minimal)

```yaml
volumes:
  - /var/run/docker.sock:/var/run/docker.sock:ro
```

Option 2: Docker Socket Proxy (recommended)

```yaml
services:
  docker-proxy:
    image: tecnativa/docker-socket-proxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - CONTAINERS=1                           # Only allow container read access
      - NETWORKS=1
      - SERVICES=1
      - TASKS=1

  traefik:
    image: traefik:v3.7
    environment:
      - DOCKER_HOST=tcp://docker-proxy:2375
    # Do NOT mount the docker socket directly
```

### Container Security

```yaml
services:
  traefik:
    security_opt:
      - no-new-privileges:true      # Prevent privilege escalation
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE            # Allow binding to privileged ports (<1024)
    # For running on port 80/443 as non-root, use:
    # sysctls:
    #   - net.ipv4.ip_unprivileged_port_start=0
    user: "1000:1000"               # Run as non-root user
```

### Headers Security Baseline

```yaml
http:
  middlewares:
    sec-headers:
      headers:
        sslRedirect: true
        stsSeconds: 31536000
        stsIncludeSubdomains: true
        stsPreload: true
        frameDeny: true
        contentTypeNosniff: true
        browserXssFilter: true
        referrerPolicy: "strict-origin-when-cross-origin"
        permissionsPolicy: "camera=(), microphone=(), geolocation=()"
        contentSecurityPolicy: "default-src 'self'"
```

### Rate Limiting

```yaml
http:
  middlewares:
    ratelimit:
      rateLimit:
        average: 100
        burst: 200
        sourceCriterion:
          ipStrategy:
            depth: 1

  routers:
    app:
      rule: "Host(`app.example.com`)"
      middlewares:
        - "ratelimit"
```

## Environment Variable Management

Use a `.env` file for sensitive values (never hardcode in docker-compose.yml):

```env
# .env — DO NOT COMMIT
CF_DNS_API_TOKEN=your_cloudflare_token_here
DASHBOARD_AUTH=admin:$2y$05$...
ACME_EMAIL=admin@example.com
```

Reference in docker-compose.yml:

```yaml
services:
  traefik:
    environment:
      - CF_DNS_API_TOKEN=${CF_DNS_API_TOKEN}
```

## Multiple Environments

```yaml
# docker-compose.override.yml — local development overrides
services:
  traefik:
    ports:
      - "8080:8080"                     # Expose dashboard for local dev
    labels:
      - "traefik.http.routers.dashboard.rule=Host(`traefik.localhost`)"
      # No TLS for local
```

```bash
# Dev
docker compose up -d

# Production (excludes override)
docker compose -f docker-compose.yml up -d
```

## High Availability Considerations

Traefik v3 OSS runs as a single instance. For HA:

- **DNS round-robin** — multiple Traefik instances behind DNS
- **Shared ACME storage** — Use a shared filesystem (NFS) or redis KV for ACME storage
- **Layer 4 load balancer** — Put a TCP load balancer (HAProxy, AWS NLB) in front
- **Redis KV provider** — Use Redis as a central configuration store for all instances
- **SO_REUSEPORT** — `reusePort: true` on entryPoints for kernel-level load balancing

```yaml
entryPoints:
  web:
    address: ":80"
    reusePort: true          # Multiple Traefik instances share the port
  websecure:
    address: ":443"
    reusePort: true
```

Note: `reusePort` works on Linux, FreeBSD, OpenBSD, and macOS. There's a known Linux kernel bug that may cause TCP connection issues — test thoroughly.

## File Provider Patterns

For complex configurations that don't fit in Docker labels:

```yaml
# Static config
providers:
  file:
    directory: "/etc/traefik/dynamic/"
    watch: true
```

```yaml
# /etc/traefik/dynamic/tls.yml
tls:
  options:
    default:
      minVersion: VersionTLS12
      sniStrict: false
    mtls:
      minVersion: VersionTLS12
      clientAuth:
        caFiles:
          - "/etc/traefik/certs/ca.pem"
        clientAuthType: RequireAndVerifyClientCert
```

```yaml
# /etc/traefik/dynamic/middlewares.yml
http:
  middlewares:
    global-rate-limit:
      rateLimit:
        average: 100
        burst: 200
```

## Logging Configuration

```yaml
# Static config
log:
  level: "INFO"                 # In production: INFO or WARN
  format: "json"                # JSON for log aggregators
  filePath: "/var/log/traefik/traefik.log"

accessLog:
  format: "json"                # Structured logging
  filePath: "/var/log/traefik/access.log"
  filters:
    statusCodes:
      - "400-599"               # Only log errors and client errors
  fields:
    headers:
      defaultMode: "drop"
      names:
        Authorization: "redact"
```

## TLS Certificate Storage

```yaml
volumes:
  - ./letsencrypt:/letsencrypt
```

**Critical:** The `acme.json` file must have permissions `600`:

```bash
chmod 600 ./letsencrypt/acme.json
```

If deploying via Docker, Traefik creates this file with correct permissions automatically (as long as the directory exists and is writable).

## Monitoring Stack

Combine with Prometheus for a complete monitoring setup:

```yaml
services:
  traefik:
    labels:
      - "traefik.http.routers.app.rule=Host(`app.example.com`)"

  prometheus:
    image: prom/prometheus
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    networks:
      - proxy
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.prometheus.rule=Host(`monitor.example.com`)"
```

# ServersTransport Reference

Defines how Traefik connects to backend servers. Configured in static config and referenced per-service in dynamic config.

## HTTP ServersTransport

Controls the connection between Traefik and HTTP backend services.

```yaml
# Static config
serversTransport:
  insecureSkipVerify: false         # Skip TLS verification to backend
  rootCAs:                          # List of root CA certificate paths
    - /etc/traefik/certs/ca.pem
  maxIdleConnsPerHost: 200          # Max idle connections per backend host
  forwardingTimeouts:
    dialTimeout: 30s                # TCP dial timeout to backend
    responseHeaderTimeout: 0s       # Timeout for backend response headers (0=unlimited)
    idleConnTimeout: 90s            # Idle keep-alive connection timeout
  spiffe:
    ids: []                         # SPIFFE IDs for workload identity
    trustDomain: ""                 # SPIFFE trust domain
```

### Configuration Options

| Option | Description | Default |
|--------|-------------|---------|
| `insecureSkipVerify` | Skip TLS certificate verification for backend connections | `false` |
| `rootCAs` | Paths to PEM-encoded CA certificates for backend TLS verification | `[]` |
| `maxIdleConnsPerHost` | Maximum idle connections kept per backend host | `200` |
| `forwardingTimeouts.dialTimeout` | Maximum time to establish TCP connection to backend | `30s` |
| `forwardingTimeouts.responseHeaderTimeout` | Maximum time to wait for response headers from backend | `0s` (no timeout) |
| `forwardingTimeouts.idleConnTimeout` | Maximum time a keep-alive connection can remain idle | `90s` |
| `spiffe.ids` | Allowed SPIFFE identities for workload identity | `[]` |
| `spiffe.trustDomain` | SPIFFE trust domain for workload identity | `""` |

## TCP ServersTransport

Controls connections for TCP routing (non-HTTP backends).

```yaml
# Static config
tcpServersTransport:
  dialKeepAlive: 30s                # Keep-alive probe interval for TCP connections
  dialTimeout: 30s                   # TCP dial timeout
  terminationDelay: 0s               # Delay before terminating TCP connection on shutdown
  tls:
    insecureSkipVerify: false        # Skip TLS verification
    rootCAs:                         # Root CA certificates for backend TLS
      - /etc/traefik/certs/ca.pem
    spiffe:
      ids: []
      trustDomain: ""
```

### Configuration Options

| Option | Description | Default |
|--------|-------------|---------|
| `dialKeepAlive` | TCP keep-alive probe interval | `30s` |
| `dialTimeout` | TCP dial timeout | `30s` |
| `terminationDelay` | Delay before TCP connection termination on shutdown | `0s` |
| `tls.insecureSkipVerify` | Skip TLS verification for backend | `false` |
| `tls.rootCAs` | Root CA certificate paths | `[]` |
| `tls.spiffe.ids` | Allowed SPIFFE identities | `[]` |
| `tls.spiffe.trustDomain` | SPIFFE trust domain | `""` |

## Per-Service ServersTransport Override

Reference a named serversTransport from a specific service in dynamic config:

```yaml
# Dynamic config (File provider)
http:
  serversTransports:
    mTLS-to-backend:
      insecureSkipVerify: false
      rootCAs:
        - /etc/traefik/certs/internal-ca.pem
      forwardingTimeouts:
        dialTimeout: 10s

  services:
    secure-api:
      loadBalancer:
        servers:
          - url: "https://backend.internal:443"
        serversTransport: "mTLS-to-backend@file"
```

Via Docker labels:

```yaml
labels:
  - "traefik.http.services.api.loadbalancer.serverstransport=mTLS-to-backend@file"
```

## mTLS Between Traefik and Backends

For mutual TLS (backends that require client certificates):

```yaml
# Dynamic config
http:
  serversTransports:
    mtls:
      insecureSkipVerify: false
      rootCAs:
        - /etc/traefik/certs/ca.pem        # CA that signed the backend cert
      # NOTE: serversTransport does NOT support client cert/key fields.
      # For client certificate authentication to backends, configure
      # TLS at the service level or use ForwardAuth.

  services:
    internal-api:
      loadBalancer:
        serversTransport: "mtls@file"
        servers:
          - url: "https://api.internal:8443"
```

## Kubernetes CRD Reference

In Kubernetes, define a `ServersTransport` CRD:

```yaml
apiVersion: traefik.io/v1alpha1
kind: ServersTransport
metadata:
  name: mtls-transport
spec:
  serverName: "api.internal"
  insecureSkipVerify: false
  rootCAs:
    - secret: internal-ca-secret         # Kubernetes Secret reference
  forwardingTimeouts:
    dialTimeout: 30s
    responseHeaderTimeout: 30s
    idleConnTimeout: 90s
```

Reference in an IngressRoute:

```yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
spec:
  routes:
    - kind: Rule
      match: Host(`app.example.com`)
      services:
        - name: api-service
          port: 8443
          serversTransport: mtls-transport
```

For TCP services, use `ServersTransportTCP`:

```yaml
apiVersion: traefik.io/v1alpha1
kind: ServersTransportTCP
metadata:
  name: tcp-mtls
spec:
  tls:
    insecureSkipVerify: false
    serverName: "db.internal"
    rootCAs:
      - secret: db-ca-secret
```

## Connection Pooling Best Practices

```yaml
# For high-throughput APIs — minimize connection churn
serversTransport:
  maxIdleConnsPerHost: 500
  forwardingTimeouts:
    dialTimeout: 5s                    # Fast fail on unavailable backends
    responseHeaderTimeout: 30s
    idleConnTimeout: 120s              # Longer idle time reduces reconnects

# For latency-sensitive services — tight timeouts
serversTransport:
  maxIdleConnsPerHost: 50
  forwardingTimeouts:
    dialTimeout: 3s
    responseHeaderTimeout: 10s
    idleConnTimeout: 30s
```

# Static Configuration Reference

Traefik's static configuration is set at startup and cannot be changed without restarting. It can be provided via:

1. **YAML file** (`traefik.yml`) — recommended for production
2. **TOML file** (`traefik.toml`) — traditional format
3. **CLI arguments** — `--flag=value` syntax
4. **Environment variables** — `TRAEFIK_<SECTION>_<KEY>` syntax

Traefik loads static config from these locations in order (last wins):
1. `traefik.yml` / `traefik.yaml` in the working directory
2. `$HOME/.traefik/traefik.yml`
3. `/etc/traefik/traefik.yml`
4. CLI flags
5. Environment variables

## Complete YAML Reference

Below is every top-level section with all configurable fields. Placeholder values shown indicate the type expected.

```yaml
## Global Settings
global:
  checkNewVersion: true          # Check for new Traefik versions
  sendAnonymousUsage: true       # Send anonymous usage stats

## Core
core:
  defaultRuleSyntax: v3          # Default rule parser: v2 or v3 (default: v3)

## SPIFFE
spiffe:
  workloadAPIAddr: ""            # SPIRE agent socket path

## Server Transport (HTTP)

See `references/servers-transport.md` for full detail on serversTransport, tcpServersTransport, connection pooling, backend TLS, and per-service transport overrides.

```yaml
serversTransport:
  insecureSkipVerify: false      # Skip TLS verification to backend
  rootCAs:                       # List of root CA certificates
    - /path/to/ca.crt
  maxIdleConnsPerHost: 200       # Max idle connections per host
  forwardingTimeouts:
    dialTimeout: 30s             # TCP dial timeout
    responseHeaderTimeout: 0s    # Timeout for response headers (0=no timeout)
    idleConnTimeout: 90s         # Idle connection timeout
  spiffe:
    ids: []
    trustDomain: ""

## TCP Server Transport

```yaml
tcpServersTransport:
  dialKeepAlive: 30s
  dialTimeout: 30s
  terminationDelay: 0s           # Delay before terminating TCP connection
  tls:
    insecureSkipVerify: false
    rootCAs: []
    spiffe:
      ids: []
      trustDomain: ""
```

## EntryPoints — Network listeners
entryPoints:
  web:
    address: ":80"                # Required: [host]:port[/tcp|/udp]
    asDefault: false              # Apply this entryPoint to routers by default
    allowACMEByPass: false        # Allow ACME challenges through custom routers
    reusePort: false              # SO_REUSEPORT for multiple processes
    transport:
      lifeCycle:
        requestAcceptGraceTimeout: 0s
        graceTimeOut: 10s          # Grace period for active requests on shutdown
      respondingTimeouts:
        readTimeout: 60s           # Max time to read entire request
        writeTimeout: 0s           # Max time to write response (0=no timeout)
        idleTimeout: 180s          # Max idle keep-alive time
      keepAliveMaxTime: 0s         # Max keep-alive connection lifetime
      keepAliveMaxRequests: 0      # Max requests per keep-alive connection (0=unlimited)
    proxyProtocol:
      insecure: false              # Trust ALL proxy protocol headers (unsafe)
      trustedIPs: []               # IPs/CIDRs allowed to send proxy protocol
    forwardedHeaders:
      insecure: false              # Trust ALL X-Forwarded-* headers
      trustedIPs: []               # IPs/CIDRs trusted to send forwarded headers
      connection: []               # Connection headers to allow through middleware chain
    http:
      redirections:
        entryPoint:
          to: websecure            # Target entryPoint name or port
          scheme: https            # Target scheme
          permanent: true          # 301 vs 302 redirect
          priority: 9223372036854775806  # Router priority for redirect
      middlewares: []              # Default middlewares prepended to all routers
      tls:
        options: ""                # Default TLS options name
        certResolver: ""           # Default ACME cert resolver
        domains:
          - main: example.com
            sans:
              - www.example.com
      encodeQuerySemicolons: false
      maxHeaderBytes: 1048576      # Max request header size (bytes)
    http2:
      maxConcurrentStreams: 250    # HTTP/2 concurrent streams per connection
    http3:
      advertisedPort: 0            # UDP port to advertise for HTTP/3
    udp:
      timeout: 3s                  # UDP idle timeout

  websecure:
    address: ":443"
    http:
      tls: true                     # Enable TLS on all routers on this entryPoint

## Providers
providers:
  providersThrottleDuration: 2s    # Min time between config reloads

  # --- Docker Provider ---
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: true          # Auto-expose containers (set false for security)
    network: ""                     # Default network for container connections
    defaultRule: "Host(`{{ normalize .Name }}`)"
    useBindPortIP: false
    watch: true
    constraints: ""                 # Filter containers (Label("key","value"))
    allowEmptyServices: false
    httpClientTimeout: 0s
    tls:
      ca: ""
      cert: ""
      key: ""
      insecureSkipVerify: false

  # --- Docker Swarm Provider ---
  swarm:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: true
    network: ""
    defaultRule: "Host(`{{ normalize .Name }}`)"
    useBindPortIP: false
    watch: true
    constraints: ""
    allowEmptyServices: false
    refreshSeconds: 15s
    httpClientTimeout: 0s

  # --- File Provider ---
  file:
    directory: "/etc/traefik/dynamic/"  # Watch directory for .yml/.yaml/.toml files
    filename: ""                         # Single file (alternative to directory)
    watch: true
    debugLogGeneratedTemplate: false

  # --- Kubernetes Ingress Provider ---
  kubernetesIngress:
    endpoint: ""                         # In-cluster if empty
    token: ""
    certAuthFilePath: ""
    namespaces: []                       # Restrict to specific namespaces (empty=all)
    labelSelector: ""
    ingressClass: ""
    ingressEndpoint:
      ip: ""
      hostname: ""
      publishedService: ""
    throttleDuration: 0s
    allowEmptyServices: false
    allowExternalNameServices: false
    disableIngressClassLookup: false
    disableClusterScopeResources: false
    nativeLBByDefault: false

  # --- Kubernetes CRD Provider ---
  kubernetesCRD:
    endpoint: ""
    token: ""
    certAuthFilePath: ""
    namespaces: []
    allowCrossNamespace: false
    allowExternalNameServices: false
    labelSelector: ""
    ingressClass: ""
    throttleDuration: 0s
    allowEmptyServices: false
    nativeLBByDefault: false
    disableClusterScopeResources: false

  # --- Kubernetes Gateway API Provider ---
  kubernetesGateway:
    endpoint: ""
    token: ""
    certAuthFilePath: ""
    namespaces: []
    labelSelector: ""
    throttleDuration: 0s
    experimentalChannel: false
    statusAddress:
      ip: ""
      hostname: ""
      service:
        name: ""
        namespace: ""

  # --- REST Provider ---
  rest:
    insecure: false               # Enable REST provider on default entryPoint

  # --- HTTP Provider ---
  http:
    endpoint: "http://..."
    pollInterval: 5s
    pollTimeout: 30s
    headers: {}                   # Request headers to send
    tls:
      ca: ""
      cert: ""
      key: ""
      insecureSkipVerify: false

  # --- Redis (KV) Provider ---
  redis:
    rootKey: "traefik"
    endpoints: ["127.0.0.1:6379"]
    username: ""
    password: ""
    db: 0
    tls:
      ca: ""
      cert: ""
      key: ""
      insecureSkipVerify: false
    sentinel:
      masterName: ""
      username: ""
      password: ""
      latencyStrategy: false
      randomStrategy: false
      replicaStrategy: false
      useDisconnectedReplicas: false

  # --- Plugin Provider Config ---
  plugin:
    pluginName:
      key: value                  # Arbitrary plugin-specific config

## API & Dashboard
api:
  insecure: false                 # Expose API on Traefik's entryPoint without auth
  dashboard: true                 # Enable dashboard (requires api.insecure or a router)
  debug: false                    # Enable debugging endpoints
  disableDashboardAd: false       # Remove Traefik ad from dashboard

## Ping
ping:
  entryPoint: ""                  # EntryPoint for ping endpoint
  manualRouting: false            # Don't auto-create ping router
  terminatingStatusCode: 503      # Status code when Traefik is shutting down

## Logging
log:
  level: "ERROR"                  # DEBUG, PANIC, FATAL, ERROR, WARN, INFO
  format: "common"                # common or json
  noColor: false
  filePath: ""                    # Log file (stdout if empty)
  maxSize: 100                    # Max megabytes before rotate
  maxAge: 30                      # Max days to retain
  maxBackups: 5                   # Max old log files
  compress: true

## Access Log
accessLog:
  filePath: ""                    # Access log file (stdout if empty)
  format: "common"                # common or json
  bufferingSize: 0                # Buffer size (0=unbuffered)
  addInternals: false             # Log internal services (ping, dashboard, etc.)
  filters:
    statusCodes: []               # Keep logs matching status codes/range
    retryAttempts: false          # Keep logs of retried requests
    minDuration: 0s               # Keep logs for requests over duration
  fields:
    defaultMode: "keep"           # keep, drop
    names:
      ClientHost: "keep"         # Per-field override
      ClientPort: "keep"         # keep, drop, redact
      ...                        # All field names: ClientHost, ClientPort, ClientUsername,
                                 # RequestHost, RequestPath, RequestMethod, RequestProtocol,
                                 # RequestContentSize, RequestLine, ResponseStatus,
                                 # ResponseContentSize, Duration, OriginDuration,
                                 # FrontendName, BackendName, BackendURL, RouterName,
                                 # ServiceName, ServiceURL, StartUTC, StartLocal, DownstreamStatus,
                                 # DownstreamContentSize, DownstreamContentSize, RequestCount,
                                 # RetryAttempts
    headers:
      defaultMode: "drop"         # keep, drop, redact
      names:
        Authorization: "redact"   # Per-header override

## Metrics
metrics:
  addInternals: false
  prometheus:
    buckets: [0.1, 0.3, 1.2, 5.0]  # Latency buckets in seconds
    addEntryPointsLabels: true
    addRoutersLabels: true
    addServicesLabels: true
    entryPoint: "metrics"         # EntryPoint for Prometheus metrics endpoint
    manualRouting: false
    headerLabels: {}              # Additional labels from request headers
  datadog:
    address: "localhost:8125"
    pushInterval: 10s
    addEntryPointsLabels: true
    addRoutersLabels: true
    addServicesLabels: true
    prefix: "traefik"
  statsD:
    address: "localhost:8125"
    pushInterval: 10s
    addEntryPointsLabels: true
    addRoutersLabels: true
    addServicesLabels: true
    prefix: "traefik"
  influxDB2:
    address: "http://localhost:8086"
    token: ""
    pushInterval: 10s
    org: ""
    bucket: ""
    addEntryPointsLabels: true
    addRoutersLabels: true
    addServicesLabels: true
    additionalLabels: {}
  otlp:
    addEntryPointsLabels: true
    addRoutersLabels: true
    addServicesLabels: true
    pushInterval: 10s
    explicitBoundaries: [0.0]      # Custom histogram boundaries
    grpc:
      endpoint: ""
      insecure: false
      headers: {}
      tls:
        ca: ""
        cert: ""
        key: ""
        insecureSkipVerify: false
    http:
      endpoint: ""
      headers: {}
      tls:
        ca: ""
        cert: ""
        key: ""
        insecureSkipVerify: false

## Tracing
tracing:
  serviceName: "traefik"
  sampleRate: 1.0                 # 0.0 to 1.0
  addInternals: false
  globalAttributes: {}            # Key-value pairs added to all spans
  capturedRequestHeaders: []      # Headers to capture in request spans
  capturedResponseHeaders: []     # Headers to capture in response spans
  safeQueryParams: []             # Query params to NOT redact
  otlp:
    grpc:
      endpoint: ""
      insecure: false
      headers: {}
      tls:
        ca: ""
        cert: ""
        key: ""
        insecureSkipVerify: false
    http:
      endpoint: ""
      headers: {}
      tls:
        ca: ""
        cert: ""
        key: ""
        insecureSkipVerify: false

## Certificate Resolvers (ACME / Let's Encrypt)
certificatesResolvers:
  letsencrypt:
    acme:
      email: "admin@example.com"    # REQUIRED — ACME registration email
      caServer: "https://acme-v02.api.letsencrypt.org/directory"  # or acme-staging
      storage: "/letsencrypt/acme.json"  # Certificate storage path
      keyType: "RSA4096"            # EC256, EC384, RSA2048, RSA4096, RSA8192
      preferredChain: ""            # Preferred certificate chain
      certificatesDuration: 2160    # Hours before renewal begins (default: 90 days)
      disableCommonName: false      # Disable CN in CSR
      profile: ""                   # Certificate profile
      caCertificates: []            # Custom CA for ACME server verification
      caSystemCertPool: false
      caServerName: ""
      emailAddresses: []
      clientTimeout: 2m
      responseHeaderTimeout: 30s
      certificateTimeout: 30s
      eab:
        kid: ""                     # External Account Binding key ID
        hmacEncoded: ""             # External Account Binding HMAC key
      httpChallenge:
        entryPoint: "web"           # EntryPoint for HTTP-01 (port 80)
        delay: 0
      tlsChallenge: {}              # Enable TLS-ALPN-01 (port 443)
      dnsChallenge:
        provider: ""                # e.g., "cloudflare", "route53", "gcloud"
        delayBeforeCheck: 0s        # Wait before DNS propagation check
        resolvers: []               # Custom DNS resolvers
        disablePropagationCheck: false
        requireAllRNS: false        # Check all recursive nameservers
        disableANSChecks: false     # Skip authoritative NS checks
        propagation:
          delayBeforeChecks: 0s
          disableChecks: false
          requireAllRNS: false
          disableANSChecks: false
    tailscale: {}                   # Enable Tailscale certificate support

## Host Resolver
hostResolver:
  cnameFlattening: false
  resolvConfig: ""
  resolvDepth: 5

## Experimental Features
experimental:
  plugins:
    pluginName:
      moduleName: "github.com/example/plugin"
      version: "v0.1.0"
      settings: {}                  # Plugin-specific config
  localPlugins:
    pluginName:
      moduleName: "path/to/plugin"  # Local plugin module path
      settings: {}
  kubernetesGateway: false          # Enable K8s Gateway API provider
  fastProxy:
    # FastProxy settings (HTTP/2 multiplexing optimization)
```

# TCP & UDP Routing Reference

Traefik can route non-HTTP traffic including databases (PostgreSQL, MySQL), messaging (RabbitMQ, Kafka), and DNS. TCP and UDP routing are configured similarly to HTTP routing but with dedicated router/service types.

## TCP Routing

TCP routers match connections based on SNI (Server Name Indication) or other criteria, and forward to TCP services.

### TCP Router Configuration

```yaml
# Dynamic config (File provider)
tcp:
  routers:
    postgres:
      rule: "HostSNI(`db.example.com`)"     # SNI-based matching
      entryPoints:
        - "postgres"                         # Must be a TCP entryPoint
      service: "postgres-backend"
      tls:
        certResolver: "letsencrypt"          # TLS termination
        options: "default@file"
        passthrough: false                   # true = forward TLS without termination
      priority: 0

    mysql-tls-passthrough:
      rule: "HostSNI(`mysql.example.com`)"
      entryPoints:
        - "mysql"
      service: "mysql-backend"
      tls:
        passthrough: true                    # TLS passthrough (no termination)
```

### TCP Rule Matchers

| Matcher | Description | Example |
|---------|-------------|---------|
| `HostSNI(domain)` | Match SNI value exactly | `` HostSNI(`db.example.com`) `` |
| `HostSNIRegexp(regex)` | Match SNI with regex | `` HostSNIRegexp(`^db\..*\.example\.com$`) `` |
| `HostSNI(`*`)` | Match all connections (catch-all) | `` HostSNI(`*`) `` |

### TCP Service Configuration

```yaml
tcp:
  services:
    postgres-backend:
      loadBalancer:
        servers:
          - address: "10.0.0.1:5432"
          - address: "10.0.0.2:5432"
        sticky: false
        strategy: "wrr"
        terminationDelay: 5s                 # Delay before terminating connection
        proxyProtocol:
          version: 1                         # Enable PROXY protocol to backend
```

TCP service options:

| Option | Description | Default |
|--------|-------------|---------|
| `servers[].address` | Backend address (`host:port`) | Required |
| `servers[].weight` | Load balancing weight | 1 |
| `sticky` | Enable sticky connections | false |
| `strategy` | LB strategy: `wrr` | wrr |
| `terminationDelay` | Connection termination delay | 0s |
| `proxyProtocol.version` | PROXY protocol version | 0 (disabled) |

### TCP Middleware

TCP middlewares are limited — only two built-in types:

```yaml
tcp:
  routers:
    secured-tcp:
      rule: "HostSNI(`app.example.com`)"
      entryPoints: ["secure-tcp"]
      middlewares:
        - "tcp-allowlist"
        - "tcp-inflight"
      service: "tcp-backend"
      tls: {}

  middlewares:
    tcp-allowlist:
      ipAllowList:
        sourceRange:
          - "10.0.0.0/8"
          - "192.168.0.0/16"

    tcp-inflight:
      inFlightConn:
        amount: 50
```

Available TCP middleware:
- `ipAllowList` — Restrict source IPs (same syntax as HTTP version)
- `inFlightConn` — Limit concurrent connections

## UDP Routing

UDP routers handle connectionless UDP traffic. Note that UDP has different semantics — there are no connections, only packets.

### UDP Router Configuration

```yaml
udp:
  routers:
    dns-server:
      entryPoints:
        - "dns-udp"                       # Must be a UDP entryPoint
      service: "dns-backend"

    syslog:
      entryPoints:
        - "syslog-udp"
      service: "syslog-backend"
```

### UDP Service Configuration

```yaml
udp:
  services:
    dns-backend:
      loadBalancer:
        servers:
          - address: "10.0.0.3:53"
          - address: "10.0.0.4:53"
        timeout: 2s                        # UDP session timeout
```

### UDP EntryPoint Configuration

```yaml
entryPoints:
  dns-udp:
    address: ":53/udp"                     # /udp suffix required
    udp:
      timeout: 3s                          # Default UDP timeout
```

## Docker Labels for TCP/UDP

TCP router via Docker labels:

```yaml
labels:
  # TCP router
  - "traefik.tcp.routers.postgres.rule=HostSNI(`db.example.com`)"
  - "traefik.tcp.routers.postgres.entrypoints=postgres"
  - "traefik.tcp.routers.postgres.tls=true"
  - "traefik.tcp.routers.postgres.service=postgres-svc"
  - "traefik.tcp.services.postgres-svc.loadbalancer.server.port=5432"
```

UDP router via Docker labels:

```yaml
labels:
  - "traefik.udp.routers.dns.entrypoints=dns-udp"
  - "traefik.udp.routers.dns.service=dns-svc"
  - "traefik.udp.services.dns-svc.loadbalancer.server.port=53"
```

## Complete Example — PostgreSQL over TCP with TLS

```yaml
# docker-compose.yml
services:
  traefik:
    image: traefik:v3.7
    command:
      - "--entrypoints.postgres.address=:5432"
      - "--providers.docker=true"
    ports:
      - "5432:5432"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - proxy

  postgres:
    image: postgres:16
    labels:
      - "traefik.tcp.routers.pg.rule=HostSNI(`db.example.com`)"
      - "traefik.tcp.routers.pg.entrypoints=postgres"
      - "traefik.tcp.routers.pg.tls=true"
      - "traefik.tcp.routers.pg.tls.certresolver=letsencrypt"
      - "traefik.tcp.services.pg.loadbalancer.server.port=5432"
    networks:
      - proxy
    environment:
      POSTGRES_PASSWORD: secret
```

## Key Differences: TCP/UDP vs HTTP

| Feature | HTTP | TCP | UDP |
|---------|------|-----|-----|
| EntryPoint protocol | /tcp (default) | /tcp (default) | /udp (explicit) |
| Router match | Host, Path, Header, etc. | HostSNI, HostSNIRegexp | entryPoint only |
| Middleware types | 25+ | 2 (ipAllowList, inFlightConn) | 0 |
| TLS termination | Yes | Yes | No |
| TLS passthrough | N/A | Yes | N/A |
| Sticky sessions | Cookie-based | Connection-based | N/A |
| Health checks | Active + Passive | Passive only | None |
| Service auto-creation | Yes (Docker) | No (must declare service) | No (must declare service) |

# TLS & ACME Reference

Traefik can automatically obtain and renew TLS certificates from Let's Encrypt (and other ACME CAs), handle custom certificates, and configure TLS options.

## ACME Certificate Resolvers (Let's Encrypt)

Define certificate resolvers in the static configuration:

```yaml
# Static config (traefik.yml)
certificatesResolvers:
  letsencrypt:
    acme:
      email: "admin@example.com"           # REQUIRED — registration email
      storage: "/letsencrypt/acme.json"    # Certificate storage (JSON)
      caServer: "https://acme-v02.api.letsencrypt.org/directory"
      # Use staging for testing to avoid rate limits:
      # caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
      keyType: "RSA4096"                   # EC256, EC384, RSA2048, RSA4096, RSA8192
      preferredChain: ""                   # Preferred certificate chain issuer
      certificatesDuration: 2160           # Hours before renewal (default 2160 = 90 days)

      # HTTP-01 Challenge (port 80)
      httpChallenge:
        entryPoint: "web"                  # EntryPoint listening on port 80

      # TLS-ALPN-01 Challenge (port 443)
      tlsChallenge: {}

      # DNS-01 Challenge (required for wildcard certs)
      dnsChallenge:
        provider: "cloudflare"             # DNS provider name
        resolvers:                         # Optional custom DNS resolvers
          - "1.1.1.1:53"
          - "8.8.8.8:53"
        delayBeforeCheck: 0s               # Wait before checking propagation
        disablePropagationCheck: false

      # External Account Binding (for ACME providers that require it)
      eab:
        kid: ""                            # Key identifier
        hmacEncoded: ""                    # Base64 URL-encoded HMAC key
```

### Challenge Types

| Challenge | Port Required | Wildcard Certs | Notes |
|-----------|--------------|----------------|-------|
| HTTP-01 | Port 80 | No | Simplest — just needs port 80 reachable |
| TLS-ALPN-01 | Port 443 | No | No port 80 needed, but port 443 must be reachable |
| DNS-01 | No ports needed | Yes | Requires DNS provider API credentials |

### Configuring DNS Providers

Traefik uses [Lego](https://go-acme.github.io/lego/dns/) for DNS challenges. Each provider requires specific environment variables.

```yaml
# Cloudflare example — requires env vars at Traefik's runtime
# CF_DNS_API_TOKEN=your_token
# or
# CF_API_EMAIL=email@example.com
# CF_API_KEY=your_global_api_key
certificatesResolvers:
  letsencrypt:
    acme:
      dnsChallenge:
        provider: "cloudflare"
```

Common providers and their env vars:

| Provider | `provider` value | Required env vars |
|----------|-----------------|-------------------|
| Cloudflare | `cloudflare` | `CF_DNS_API_TOKEN` or `CF_API_EMAIL` + `CF_API_KEY` |
| AWS Route53 | `route53` | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` (or IAM role) |
| Google Cloud DNS | `gcloud` | `GCE_PROJECT`, `GCE_SERVICE_ACCOUNT_FILE` |
| DigitalOcean | `digitalocean` | `DO_AUTH_TOKEN` |
| Azure | `azure` | `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, `AZURE_SUBSCRIPTION_ID`, `AZURE_TENANT_ID` |
| Namecheap | `namecheap` | `NAMECHEAP_API_USER`, `NAMECHEAP_API_KEY` |
| Linode | `linode` | `LINODE_TOKEN` |
| Vultr | `vultr` | `VULTR_API_KEY` |
| OVH | `ovh` | `OVH_ENDPOINT`, `OVH_APPLICATION_KEY`, `OVH_APPLICATION_SECRET`, `OVH_CONSUMER_KEY` |

Full provider list: https://go-acme.github.io/lego/dns/

### Wildcard Certificate with DNS-01

```yaml
# DNS challenge is REQUIRED for wildcard certs
certificatesResolvers:
  letsencrypt:
    acme:
      email: "admin@example.com"
      storage: "/letsencrypt/acme.json"
      dnsChallenge:
        provider: "cloudflare"
        delayBeforeCheck: 0
      # HTTP and TLS challenges are NOT needed for wildcard-only
```

## Using Certificates in Routers

Reference a certificate resolver on the router:

```yaml
# Dynamic config (File provider)
http:
  routers:
    app:
      rule: "Host(`app.example.com`)"
      tls:
        certResolver: "letsencrypt"
        options: "default@file"       # Optional TLS options
        domains:
          - main: "example.com"
            sans:
              - "www.example.com"
              - "api.example.com"
      service: "app-backend"
```

Or via Docker labels:

```yaml
labels:
  - "traefik.http.routers.app.rule=Host(`app.example.com`)"
  - "traefik.http.routers.app.tls=true"
  - "traefik.http.routers.app.tls.certresolver=letsencrypt"
```

## TLS Options

Define TLS connection parameters:

```yaml
# Dynamic config (File provider)
tls:
  options:
    default:                           # Applies to all routers unless overridden
      minVersion: VersionTLS12         # Minimum TLS version
      maxVersion: VersionTLS13         # Maximum TLS version
      cipherSuites:                    # Specific cipher suites
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
      sniStrict: false                 # Reject if SNI doesn't match a server cert
      preferServerCipherSuites: false
      curvePreferences:                # ECC curves for key exchange
        - CurveP256
        - CurveP384
        - CurveP521

    modern:                            # Modern security profile
      minVersion: VersionTLS13
      sniStrict: true

    mtls:                              # Mutual TLS (client certificates)
      minVersion: VersionTLS12
      clientAuth:
        caFiles:
          - "/etc/traefik/certs/ca.pem"
        clientAuthType: RequireAndVerifyClientCert  # or VerifyClientCertIfGiven
```

TLS reference in a router:

```yaml
http:
  routers:
    secured-app:
      rule: "Host(`app.example.com`)"
      tls:
        options: "modern@file"         # Namespace:name@provider
```

## Custom Certificates

Manually load certificates instead of using ACME:

```yaml
# Dynamic config
tls:
  certificates:
    - certFile: "/etc/traefik/certs/example.com.pem"
      keyFile: "/etc/traefik/certs/example.com-key.pem"
      stores:
        - "default"                    # Certificate store to use
    - certFile: "/etc/traefik/certs/wildcard.example.com.pem"
      keyFile: "/etc/traefik/certs/wildcard.example.com-key.pem"
```

## Default Certificate

Traefik generates a self-signed certificate by default. To set a custom default:

```yaml
# Dynamic config
tls:
  stores:
    default:
      defaultCertificate:
        certFile: "/etc/traefik/certs/default.pem"
        keyFile: "/etc/traefik/certs/default-key.pem"
```

## TLS on EntryPoints

Enable TLS for all routers on an entryPoint:

```yaml
entryPoints:
  websecure:
    address: ":443"
    http:
      tls: true                        # All routers on websecure get TLS
      tls:
        certResolver: "letsencrypt"    # Default cert resolver
        options: "default@file"        # Default TLS options
```

Or per-router override:

```yaml
http:
  routers:
    no-tls-route:
      rule: "Host(`public.example.com`)"
      entryPoints: ["websecure"]
      # No tls block = no TLS for this specific route
```

## Automatic Certificate Renewal

Traefik automatically tracks certificate expiry and renews:
- Default cert validity: 90 days
- Renewal starts: 30 days before expiry
- Configured by `certificatesDuration` (in hours, default 2160 = 90 days)
- Unused certificates may still be renewed

## Certificate Storage

ACME certificates are stored as JSON in the `storage` file (default `acme.json`):

```yaml
certificatesResolvers:
  letsencrypt:
    acme:
      storage: "/letsencrypt/acme.json"
```

**Important:** `acme.json` must have permissions `600` (owner read/write only) or Traefik will refuse to start.

## v3 TLS Changes

- The `tls` option on routers now defaults based on the entryPoint's `http.tls` setting
- `tls.minVersion` now defaults to `VersionTLS12` instead of `VersionTLS10`
- The deprecated `tls` section in static config for default cert has been replaced by the dynamic `tls.stores` approach

#!/usr/bin/env bash
# traefik-healthcheck.sh — Check a running Traefik instance health
# Usage: ./traefik-healthcheck.sh [--json] [--url https://traefik.example.com]
#
# Requires: curl, jq
# Non-interactive, agent-friendly.

set -euo pipefail

# Defaults
URL="${TRAEFIK_URL:-http://localhost:8080}"
JSON=false
EXIT_CODE=0

while [[ $# -gt 0 ]]; do
  case "$1" in
    --json) JSON=true; shift ;;
    --url) URL="$2"; shift 2 ;;
    --help|-h)
      echo "Usage: traefik-healthcheck.sh [--json] [--url <traefik-url>]"
      echo ""
      echo "Checks ping, API, version, router counts, and certificate expiry."
      echo "Defaults to http://localhost:8080. Set TRAEFIK_URL env var to override."
      exit 0
      ;;
    *) echo "Unknown option: $1"; exit 1 ;;
  esac
done

json_out() {
  if $JSON; then
    echo "$1"
  fi
}

text_out() {
  if ! $JSON; then
    echo "$1"
  fi
}

# 1. Ping check
ping_status=$(curl -s -o /dev/null -w "%{http_code}" "${URL}/ping" 2>/dev/null || echo "000")
if [ "$ping_status" = "200" ]; then
  text_out "✓ Ping: OK (200)"
  json_out "{\"ping\": {\"status\": \"ok\", \"code\": 200}"
else
  text_out "✗ Ping: FAILED ($ping_status)"
  json_out "{\"ping\": {\"status\": \"fail\", \"code\": $ping_status}"
  EXIT_CODE=1
fi

# 2. API health check
api_status=$(curl -s -o /dev/null -w "%{http_code}" "${URL}/api/version" 2>/dev/null || echo "000")
if [ "$api_status" = "200" ]; then
  version=$(curl -s "${URL}/api/version" 2>/dev/null | jq -r '.Version // "unknown"' 2>/dev/null || echo "unknown")
  text_out "✓ API: OK ($api_status) — Traefik $version"
  json_out ", \"api\": {\"status\": \"ok\", \"version\": \"$version\"}"
else
  text_out "✗ API: FAILED ($api_status) — is the API enabled?"
  json_out ", \"api\": {\"status\": \"fail\", \"code\": $api_status}"
  EXIT_CODE=1
fi

# 3. Router count
routers=$(curl -s "${URL}/api/http/routers" 2>/dev/null | jq length 2>/dev/null || echo "N/A")
text_out "  HTTP routers: $routers"
json_out ", \"routers\": {\"count\": $routers}"

tcp_routers=$(curl -s "${URL}/api/tcp/routers" 2>/dev/null | jq length 2>/dev/null || echo "N/A")
text_out "  TCP routers: $tcp_routers"
json_out ", \"tcp_routers\": {\"count\": $tcp_routers}"

# 4. Overview
overview=$(curl -s "${URL}/api/overview" 2>/dev/null || echo "{}")
if [ "$overview" != "{}" ]; then
  total=$(echo "$overview" | jq -r '.http.routers.total // 0' 2>/dev/null)
  text_out "  Total HTTP routers (overview): $total"
fi

# 5. Certificate expiry check
certs=$(curl -s "${URL}/api/rawconfig" 2>/dev/null | jq -r '.tls.certificates // []' 2>/dev/null || echo "[]")
if [ "$certs" != "[]" ]; then
  cert_count=$(echo "$certs" | jq length)
  text_out "  TLS certificates: $cert_count"
  json_out ", \"tls_certificates\": {\"count\": $cert_count}"
else
  json_out ", \"tls_certificates\": null"
fi

# Close JSON
if $JSON; then
  echo "}"
fi

exit $EXIT_CODE

# Production Traefik Deployment — Docker Compose
# Target: Traefik v3.7+
# Usage: docker compose up -d
#
# Features:
# - Docker provider (read-only socket via docker-socket-proxy)
# - Let's Encrypt ACME (HTTP-01 challenge)
# - Dashboard with BasicAuth (internal-only)
# - Security hardening (read-only rootfs, no-new-privileges, capability drop)
# - Prometheus metrics endpoint
# - JSON structured logging
# - HTTP/3 (QUIC) enabled

version: "3.8"

x-logging: &default-logging
  driver: json-file
  options:
    max-size: "10m"
    max-file: "3"

services:
  # --- Docker Socket Proxy (Security) ---
  # Replaces direct docker.sock mount with a read-only API proxy
  docker-proxy:
    image: tecnativa/docker-socket-proxy:latest
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - CONTAINERS=1
      - NETWORKS=1
      - SERVICES=1
      - TASKS=1
      - INFO=1
    networks:
      - traefik
    logging: *default-logging

  # --- Traefik Reverse Proxy ---
  traefik:
    image: traefik:v3.7
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE
    networks:
      - traefik
    ports:
      - "80:80"
      - "443:443"
      # UDP port for HTTP/3 (QUIC)
      - "443:443/udp"
    environment:
      # DNS challenge credentials (example: Cloudflare)
      # - CF_DNS_API_TOKEN=${CF_DNS_API_TOKEN}
      - TZ=UTC
    volumes:
      - ./traefik.yml:/etc/traefik/traefik.yml:ro
      - ./dynamic:/etc/traefik/dynamic:ro
      - ./letsencrypt:/letsencrypt
      # Do NOT mount docker.sock directly — use docker-proxy
    command:
      # Providers
      - "--providers.docker=true"
      - "--providers.docker.endpoint=tcp://docker-proxy:2375"
      - "--providers.docker.exposedbydefault=false"
      - "--providers.docker.network=traefik"
      # File provider for shared middlewares
      - "--providers.file.directory=/etc/traefik/dynamic"
      - "--providers.file.watch=true"
      # EntryPoints
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--entrypoints.websecure.http3=true"
      # TLS
      - "--certificatesresolvers.letsencrypt.acme.email=admin@example.com"
      - "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge=true"
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web"
      # API (dashboard served through a router, not directly)
      - "--api.dashboard=true"
      - "--api.insecure=false"
      # Logging
      - "--log.level=INFO"
      - "--log.format=json"
      - "--accesslog=true"
      - "--accesslog.format=json"
      - "--accesslog.addinternals=false"
    labels:
      # Dashboard router
      - "traefik.enable=true"
      - "traefik.http.routers.dashboard.rule=Host(`traefik.example.com`)"
      - "traefik.http.routers.dashboard.service=api@internal"
      - "traefik.http.routers.dashboard.tls=true"
      - "traefik.http.routers.dashboard.tls.certresolver=letsencrypt"
      - "traefik.http.routers.dashboard.middlewares=dashboard-auth"
      - "traefik.http.middlewares.dashboard-auth.basicauth.users=${DASHBOARD_AUTH:-admin:$$2y$$10$$...}"
    logging: *default-logging

networks:
  traefik:
    name: traefik
    driver: bridge