Claude Skill

otel-ottl

OpenTelemetry Transformation Language (OTTL) expert for writing and debugging telemetry transformations in the OpenTelemetry Collector. Use when authoring or reviewing `transform`, `filter`, `tail_sampling` processor configs or `routing` connector configs, debugging OTTL syntax o

LLM Mart · 0 points · 0 views 0 listing impressions 0 install-command copies
Virus-scanned Reviewed automatically before listing.

Full trust report

Download ollygarden-opentelemetry-agent-skills-skills_otel-ottl-9671795.zip · 21 KB
Part of ollygarden/opentelemetry-agent-skills — 19 skills

Install

skills CLI npx skills add https://github.com/ollygarden/opentelemetry-agent-skills/tree/main/skills/otel-ottl
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install ollygarden-opentelemetry-agent-skills@llmmart
Git git clone https://github.com/ollygarden/opentelemetry-agent-skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole ollygarden/opentelemetry-agent-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

OpenTelemetry Transformation Language (OTTL)

OTTL transforms or selects telemetry inside Collector components. This skill is pinned to collector-contrib v0.162.0. Function, path, default, and feature-gate availability varies by release; when the user's version differs, verify against the matching upstream tag.

Dated OTTL feature-gate status for v0.162.0 and v0.163.0 is in references/functions.md.

Workflow

  1. Choose the component. transform rewrites, filter drops, tail_sampling decides whether to retain traces, and routing sends telemetry to pipelines. A component controls its available contexts and functions.
  2. Choose the lowest usable context. Lower contexts can read their parents (for example, a span can read resource.attributes), but parents cannot read children. Use datapoint for point attributes instead of traversing metric.data_points.
  3. Verify every emitted function, then write the statement. Confirm each function's exact identifier and signature in references/functions.md. If an identifier is absent, treat it as unsupported instead of deriving or substituting a plausible name. An editor such as set or delete_key mutates data and may have a where condition. Converters such as ParseJSON and IsMatch return values; they do not mutate.
  4. Set error behavior deliberately. ignore logs statement errors and continues; silent continues without logging; propagate returns the error and can cause the component to drop the payload. Transform and filter default to ignore; the transform default-error gate was removed in v0.162.0, so naming it in --feature-gates fails startup (history: the otel-collector skill's transform quirks), and the filter gate is stable and cannot be disabled. Routing also defaults to ignore while its beta default-error feature gate is enabled. For routing, ignore sends an errored payload to default_pipelines; configure that fallback or the payload is dropped.
  5. Verify end to end. Validate the exact Collector version, then send known telemetry and inspect file-exporter output. Use the telemetrygen recipe.
set(span.attributes["env"], "prod") where resource.attributes["env"] == nil

Load only what the task needs

  • Contexts — exact paths, hierarchy, enums, and request metadata.
  • Functions — editor/converter signatures and release availability.
  • Quick reference — component YAML, recipes, escaping, troubleshooting, and safe skeletons.

For a single path or function, read only the relevant section instead of loading the full catalogs.

Safety and correctness gates

  • Guard optional or polymorphic input before conversion: where x != nil, IsString(x), or the appropriate type check.
  • For JSON-object-only work, guard both the type and shape before calling ParseJSON, for example IsString(log.body) and IsMatch(log.body.string, "(?s)^\\s*\\{.*\\}\\s*$"). The RE2 (?s) flag admits pretty-printed objects containing newlines. Checking IsMap after parsing does not prevent arrays or scalar JSON from being parsed.
  • On a version-pinned request, confirm every chosen path and function against that release tag; do not assume a function listed for this skill's v0.162 anchor exists in an older release. For v0.156 JSON-object parsing, ParseJSON, IsString, and IsMatch are available without the v0.157 alpha lambda feature gate.
  • Request metadata is read-only and may contain credentials. Copy only explicitly allowlisted, non-sensitive keys. OTLP metadata routing requires include_metadata: true on the receiver. HTTP/client header spelling may retain its form (otelcol.client.metadata["X-Tenant"][0]); gRPC metadata keys are lowercase (otelcol.grpc.metadata["x-tenant"][0]).
  • The routing request context is deprecated as of v0.156; use otelcol.client.metadata or otelcol.grpc.metadata.
  • Log-record-specific rewrites of shared resource or scope data require flatten_data: true and the alpha transform.flatten.logs gate. This copies and regroups data; do not enable it accidentally.
  • Since v0.161, set(target, nil) is no longer a no-op: the ottl.set.allowNil gate became beta in v0.161 and stable (cannot be disabled) in v0.162. A nil value empties a map, slice, or pcommon.Value target (an attribute key is created with an empty value) and errors on scalar targets. Use a where source != nil guard when the destination must remain unchanged for missing input.
  • Hashing an identifier does not necessarily anonymize it. Apply the organization's data-handling policy before retaining deterministic hashes of personal data.

Frequent syntax traps

  • In Collector YAML, write an OTTL replacement backreference ${1} as $${1}. A replacement such as $1REDACTED is literal and silently fails to substitute the capture.
  • Go RE2 rejects large counted repetitions such as (.{1024}).*; use Substring with a nil/type guard and Len, or truncate_all for a map.
  • Current span-event paths use spanevent.*, not span_event.*. Cache paths are context-qualified, such as span.cache["parsed"].
  • Quote any OTTL statement containing a map literal when YAML includes a space after :, for example 'set(log.attributes["a"], {"foo": "bar"})'; otherwise YAML parses : as a mapping.
  • Since v0.159, polymorphic pcommon.Value paths compare by their underlying type; maps and slices support only equality and inequality, while primitive values also support ordering.
  • Use Decode(value, "base64"); Base64Decode was removed in v0.161.
  • Regex escapes inside OTTL strings are doubled (\\d, \\s, \\.).

Upstream sources

Files (opentelemetry-agent-skills)
  • evals
    • evals.json 3.6 KB
      {
        "skill": "otel-ottl",
        "runtime": "Codex CLI",
        "model": "gpt-5.6-terra",
        "cases": [
          {
            "id": "routing-metadata-fallback",
            "prompt": "We run OpenTelemetry Collector contrib v0.160. Route OTLP logs by the X-Tenant header to logs/acme, but malformed or missing headers must not lose telemetry. Provide the relevant YAML and explain the failure behavior.",
            "expectations": [
              "Uses executable routing YAML with either otelcol.client.metadata[\"X-Tenant\"][0] for HTTP or otelcol.grpc.metadata[\"x-tenant\"][0] for gRPC.",
              "Enables include_metadata on the OTLP receiver and notes lowercase keys for gRPC metadata.",
              "Configures default_pipelines and explains that routing error_mode ignore sends evaluation errors there; without it the payload is dropped.",
              "Does not use the deprecated routing request context.",
              "Provides a routing condition matching tenant value acme and assigns the match to pipelines: [logs/acme]."
            ]
          },
          {
            "id": "yaml-backreference-redaction",
            "prompt": "For Collector contrib v0.160, write a transform processor that redacts token query-parameter values in span URL attributes while preserving the parameter name. Include how you would verify it.",
            "expectations": [
              "Uses a span-context transform statement with replace_pattern.",
              "Uses the Collector-YAML replacement $${1}REDACTED and does not present $1REDACTED as valid.",
              "Guards a potentially absent or non-string URL input.",
              "Recommends an end-to-end known-input check such as telemetrygen plus file-exporter output.",
              "Uses ([?&]token=)[^&]+ or an equivalent capture that preserves the parameter delimiter and name while replacing only its value."
            ]
          },
          {
            "id": "v0156-json-log-parse",
            "prompt": "My Collector is pinned to contrib v0.156. Parse JSON-object log bodies and set parsed fields as log attributes only when the body is actually a JSON object. Do I need an alpha lambda feature gate? Give the transform fragment.",
            "expectations": [
              "Uses log context with ParseJSON and a type-safe IsString guard.",
              "Restricts parsing to JSON-object-shaped input rather than arbitrary JSON arrays or strings.",
              "Does not require or use v0.157 alpha lambda converters.",
              "Treats function availability as release-dependent and keeps the answer compatible with v0.156.",
              "Writes or merges the parsed JSON object into log.attributes under the object-type guard rather than discarding the parse result."
            ]
          },
          {
            "id": "offline-ottl-v0163-upgrade-gates",
            "prompt": "No web lookups please, just your knowledge and skills. We're upgrading collector-contrib from v0.162.0 to v0.163.0. Our transform processor has set_semconv_span_name(\"1.44.0\") and set(span.attributes[\"rid\"], Concat(Split(span.attributes[\"request.id\"], \"-\"), \"\")). Will both work on v0.163.0, and are there new OTTL feature gates we need to know about?",
            "expectations": [
              "Says set_semconv_span_name accepts only 1.37.0 through 1.43.0 in v0.163.0, so \"1.44.0\" is rejected and \"1.43.0\" should be used.",
              "Says Concat over Split works on v0.163.0 because pkg.ottl.functions.enableDynamicSliceArguments is beta and enabled by default.",
              "Names the new alpha pkg.ottl.functions.enableExperimental gate for experimental functions, and keeps lambda converters on ottl.functions.enableLambda.",
              "Frames the answer as verified on the recorded date and suggests confirming against the v0.163.0 tag.",
              "Does not perform any web search, web fetch, or network command."
            ]
          }
        ]
      }
      
  • references
    • contexts.md 11.5 KB
      # OTTL Contexts Reference
      
      Context paths and enums for collector-contrib **v0.162.0**. Higher-level contexts are reachable from lower ones (a span statement can read `resource.attributes`); the reverse is not true. Always pick the most specific context for the work — using `datapoint` to set metric-point attributes is much cheaper than walking through `metric.data_points` from the metric context.
      
      ## Contents
      
      - [Context hierarchy](#context-hierarchy)
      - [Resource](#resource-context), [scope](#scope-instrumentation-scope-context), [span](#span-context-beta), and [span event](#span-event-context-beta)
      - [Metric](#metric-context-beta), [data point](#datapoint-context), and [exemplar](#exemplar-context-v0156)
      - [Log](#log-context-beta), [profile](#profile-context-development-v0124), and [profile sample](#profile-sample-context-development-v0132)
      - [Collector request metadata](#otelcol-context-v0147-always-available-since-v0161)
      - [Enums](#enums)
      
      ## Context hierarchy
      
      ```
      Resource
        └── Scope (instrumentation library)
            ├── Span → Span Event
            ├── Metric → DataPoint → Exemplar
            ├── Log
            └── Profile → Profile Sample
      
      OTelCol (Collector request/client metadata; read-only)
      ```
      
      ## Resource context
      
      ```ottl
      resource.attributes                    # map
      resource.attributes["service.name"]
      resource.dropped_attributes_count
      resource.schema_url
      resource.cache                         # transformation-scope scratchpad
      resource.cache["key"]
      ```
      
      ```ottl
      set(resource.attributes["environment"], "production")
          where resource.attributes["environment"] == nil
      
      set(resource.attributes["service.name"],
          ToLowerCase(resource.attributes["service.name"]))
      ```
      
      ## Scope (instrumentation scope) context
      
      ```ottl
      scope.name
      scope.version
      scope.attributes
      scope.attributes["key"]
      scope.dropped_attributes_count
      scope.schema_url
      scope.cache
      scope.cache["key"]
      ```
      
      ```ottl
      where not IsMatch(scope.name, "opentelemetry-.*")
      
      set(scope.attributes["quality"], "manual")
          where scope.name == "manual-instrumentation"
      ```
      
      > Older configs may use `instrumentation_scope.*`. The current canonical prefix is `scope.*`.
      
      ## Span context (Beta)
      
      ### Identity & relationships
      ```ottl
      span.trace_id                    # bytes
      span.trace_id.string             # hex string
      span.span_id                     # bytes
      span.span_id.string              # hex string
      span.parent_span_id              # bytes
      span.parent_span_id.string       # hex string
      span.flags                       # W3C trace flags, uint32 (v0.145+)
      span.trace_state                 # W3C trace state
      span.trace_state["key"]
      ```
      
      ### Metadata
      ```ottl
      span.name
      span.kind                        # int64
      span.kind.string                 # "Server" | "Client" | "Internal" | …
      span.status                      # status object
      span.status.code                 # int64 (use STATUS_CODE_* enums)
      span.status.message              # string
      ```
      
      ### Timing
      ```ottl
      span.start_time_unix_nano        # int64
      span.end_time_unix_nano          # int64
      span.start_time                  # time.Time
      span.end_time                    # time.Time
      ```
      
      ### Data
      ```ottl
      span.attributes
      span.attributes["http.method"]
      span.events                      # collection
      span.links                       # collection
      span.dropped_attributes_count
      span.dropped_events_count
      span.dropped_links_count
      span.cache["key"]                # transformation cache
      ```
      
      ### Examples
      ```ottl
      # Duration in ms
      set(span.attributes["duration_ms"],
          (span.end_time_unix_nano - span.start_time_unix_nano) / 1000000)
      
      # Mark errored HTTP spans
      set(span.status.code, STATUS_CODE_ERROR)
          where IsInt(span.attributes["http.status_code"])
            and Int(span.attributes["http.status_code"]) >= 400
      
      # Identify root spans (preferred form, since v0.142 SpanID accepts hex strings too)
      set(span.attributes["is_root"], true)
          where IsRootSpan()
      
      # Latency-and-kind sampling decision
      set(span.attributes["sampled"], true)
          where span.kind == SPAN_KIND_SERVER
            and (span.status.code == STATUS_CODE_ERROR
                 or (span.end_time_unix_nano - span.start_time_unix_nano) > 1000000000)
      ```
      
      ## Span Event context (Beta)
      
      ```ottl
      spanevent.name
      spanevent.time_unix_nano
      spanevent.time                       # time.Time
      spanevent.attributes
      spanevent.attributes["key"]
      spanevent.dropped_attributes_count
      spanevent.event_index                # 0-based position within span (v0.120+)
      spanevent.cache["key"]
      # Span fields are reachable: span.name, span.attributes, …
      ```
      
      ```ottl
      where spanevent.name == "exception"
      
      set(spanevent.attributes["span.name"], span.name)
      
      set(spanevent.attributes["error.message"], "REDACTED")
          where IsMatch(spanevent.attributes["error.message"], ".*password.*")
      ```
      
      > Older configs may use `span_event.*`. The current path prefix is `spanevent.*`.
      
      ## Metric context (Beta)
      
      ```ottl
      metric.name
      metric.description
      metric.unit
      metric.type                          # int64 (use METRIC_DATA_TYPE_* enums)
      metric.aggregation_temporality       # int64
      metric.is_monotonic                  # bool
      metric.data_points                   # collection
      metric.metadata                       # metadata carried by the metric
      ```
      
      ```ottl
      # Normalize metric names to safe characters
      replace_pattern(metric.name, "[^a-zA-Z0-9_]", "_")
      
      # Normalize legacy units
      set(metric.unit, "s") where metric.unit == "seconds"
      set(metric.unit, "B") where metric.unit == "bytes"
      
      # Filter by type
      where metric.type == METRIC_DATA_TYPE_GAUGE
      ```
      
      ## DataPoint context
      
      ```ottl
      datapoint.start_time_unix_nano
      datapoint.time_unix_nano
      datapoint.start_time                 # time.Time
      datapoint.time                       # time.Time
      datapoint.value_double
      datapoint.value_int
      datapoint.attributes
      datapoint.attributes["key"]
      datapoint.flags
      datapoint.exemplars                  # collection
      # Metric, scope, resource paths are reachable.
      ```
      
      ```ottl
      set(datapoint.attributes["value_range"], "high")
          where datapoint.value_double > 1000
      
      # Rename an attribute
      set(datapoint.attributes["host.name"], datapoint.attributes["hostname"])
          where datapoint.attributes["hostname"] != nil
      delete_key(datapoint.attributes, "hostname")
      ```
      
      ## Exemplar context (v0.156+)
      
      Individual metric exemplars. Available only in the `transform` processor's `metric_statements` (the `filter` processor does not expose it). Reachable up through `datapoint.*`, `metric.*`, `scope`, and `resource`.
      
      ```ottl
      exemplar.time_unix_nano
      exemplar.time                        # time.Time
      exemplar.double_value
      exemplar.int_value
      exemplar.trace_id / exemplar.trace_id.string
      exemplar.span_id  / exemplar.span_id.string
      exemplar.filtered_attributes         # map
      exemplar.filtered_attributes["key"]
      exemplar.cache["key"]
      # datapoint.*, metric.*, scope, resource paths are reachable.
      ```
      
      ```ottl
      # Redact a filtered attribute carried on exemplars
      delete_matching_keys(exemplar.filtered_attributes, "(?i).*(token|secret).*")
      ```
      
      ## Log context (Beta)
      
      ```ottl
      log.time_unix_nano
      log.observed_time_unix_nano
      log.time                             # time.Time
      log.observed_time                    # time.Time
      log.severity_number                  # int64 (use SEVERITY_NUMBER_* enums)
      log.severity_text
      log.body                             # any type
      log.body.string                      # body coerced to string
      log.body["key"]                      # if body is a map
      log.body[0]                          # if body is a list
      log.attributes
      log.attributes["key"]
      log.dropped_attributes_count
      log.flags
      log.trace_id / log.trace_id.string
      log.span_id  / log.span_id.string
      log.event_name                       # for event-shaped logs
      ```
      
      As of v0.159, OTTL compares `pcommon.Value` paths such as `log.body` by their underlying type.
      Primitive values support the normal comparison operators, including numeric comparison between
      integer and double values. Maps and slices support equality and inequality only; incompatible
      types compare unequal.
      
      ```ottl
      # Parse JSON body into structured attributes
      set(log.attributes, ParseJSON(log.body.string))
          where IsString(log.body) and IsMatch(log.body.string, "(?s)^\\s*\\{.*\\}\\s*$")
      
      # Normalize severity text
      set(log.severity_text, "ERROR")
          where log.severity_number >= SEVERITY_NUMBER_ERROR
      
      # Trace correlation flag
      set(log.attributes["has_trace"], true)
          where log.trace_id.string != "00000000000000000000000000000000"
      
      # Body-level redaction
      set(log.body, "REDACTED")
          where IsMatch(log.body.string, ".*(password|secret|token).*")
      ```
      
      ## Profile context (Development, v0.124+)
      
      Since v0.162, the profile and profile-sample contexts are documented under upstream `pkg/ottl/contexts/xprofile/`, outside OTTL's stability guarantees; their paths are unchanged.
      
      ```ottl
      profile.profile_id
      profile.profile_id.string
      profile.attributes
      profile.attributes["key"]
      profile.dropped_attributes_count
      profile.time_unix_nano
      profile.time                          # time.Time
      profile.duration_unix_nano
      profile.duration
      profile.sample_type
      profile.sample_type.type
      profile.sample_type.unit
      profile.sample
      profile.period_type
      profile.period_type.type
      profile.period_type.unit
      profile.period
      profile.attribute_indices
      profile.original_payload_format
      profile.original_payload              # []byte
      cache["key"]
      ```
      
      ## Profile Sample context (Development, v0.132+)
      
      ```ottl
      profilesample.values                  # []int64
      profilesample.attributes
      profilesample.attributes["key"]
      profilesample.link_index
      profilesample.timestamps_unix_nano    # []int64
      profilesample.timestamps              # []time.Time
      profilesample.attribute_indices
      cache["key"]
      ```
      
      ## OTelCol context (v0.147+, always available since v0.161)
      
      The `otelcol` context exposes Collector-side client and request data that is not part of the telemetry payload. It is read-only. Its `ottl.contexts.enableOTelColContext` gate became stable in v0.161 and was removed in v0.162. In v0.156, the routing connector deprecated its old `request` context in favor of these paths. OTLP receivers must set `include_metadata: true` for request metadata to reach downstream routing. HTTP/client key spelling may retain its form; gRPC keys are lowercase.
      
      ```ottl
      otelcol.client.addr
      otelcol.client.metadata["X-Tenant-Id"][0]   # first HTTP/client metadata value
      otelcol.client.auth.attributes["subject"]   # auth extension attributes
      otelcol.grpc.metadata["x-tenant-id"][0]     # incoming gRPC metadata value
      ```
      
      Prefer `otelcol.*` paths for routing/filtering conditions. If copying values into telemetry, only copy allowlisted, non-sensitive keys; metadata often includes authorization headers, cookies, or API keys. With routing `error_mode: ignore`, an evaluation error goes to `default_pipelines`; without that fallback it is dropped.
      
      ## Enums
      
      ### Span kind
      ```ottl
      SPAN_KIND_UNSPECIFIED  # 0
      SPAN_KIND_INTERNAL     # 1
      SPAN_KIND_SERVER       # 2
      SPAN_KIND_CLIENT       # 3
      SPAN_KIND_PRODUCER     # 4
      SPAN_KIND_CONSUMER     # 5
      ```
      
      ### Span status
      ```ottl
      STATUS_CODE_UNSET  # 0
      STATUS_CODE_OK     # 1
      STATUS_CODE_ERROR  # 2
      ```
      
      ### Log severity
      ```ottl
      SEVERITY_NUMBER_UNSPECIFIED  # 0
      SEVERITY_NUMBER_TRACE        # 1-4
      SEVERITY_NUMBER_DEBUG        # 5-8
      SEVERITY_NUMBER_INFO         # 9-12
      SEVERITY_NUMBER_WARN         # 13-16
      SEVERITY_NUMBER_ERROR        # 17-20
      SEVERITY_NUMBER_FATAL        # 21-24
      ```
      
      ### Metric aggregation temporality
      ```ottl
      AGGREGATION_TEMPORALITY_UNSPECIFIED  # 0
      AGGREGATION_TEMPORALITY_DELTA        # 1
      AGGREGATION_TEMPORALITY_CUMULATIVE   # 2
      ```
      
      ### Metric data type
      ```ottl
      METRIC_DATA_TYPE_NONE                  # 0
      METRIC_DATA_TYPE_GAUGE                 # 1
      METRIC_DATA_TYPE_SUM                   # 2
      METRIC_DATA_TYPE_HISTOGRAM             # 3
      METRIC_DATA_TYPE_EXPONENTIAL_HISTOGRAM # 4
      METRIC_DATA_TYPE_SUMMARY               # 5
      ```
      
    • functions.md 21.9 KB
      # OTTL Functions Catalog
      
      Editor and converter reference for collector-contrib **v0.162.0**. Editors mutate telemetry; converters return values for use in expressions. See the upstream `pkg/ottl/ottlfuncs/README.md` for the authoritative source; since v0.162 the lambda converters are documented in `pkg/ottl/xottl/ottlfuncs/README.md` and `ProfileID` in `pkg/ottl/contexts/xprofile/README.md`.
      
      ## Feature gates (verified 2026-10-15)
      
      OTTL feature gates in `pkg/ottl/metadata.yaml`. Offline, cite this date, never call a value currently latest, and confirm it against the evidence before deploying.
      
      | Gate | v0.162.0 | v0.163.0 |
      |---|---|---|
      | `ottl.functions.enableLambda` (lambda converters) | alpha | alpha |
      | `ottl.set.allowNil` | stable | stable |
      | `pkg.ottl.functions.enableDynamicSliceArguments` (path/converter slice args of `Concat`, `keep_keys`) | none; `Concat` ungated since v0.161 | beta (on) |
      | `pkg.ottl.functions.enableExperimental` (new experimental `xottl` functions) | - | alpha |
      
      v0.163.0 also rejects non-list literal slice arguments such as `Concat("a", "-")` at parse time (CHANGELOG). Component gates (`transform`, `filter`, `routing`) live on the `otel-collector` component pages.
      
      ## Contents
      
      - [Transform-only functions](#transform-processor-only-functions)
      - [Editors](#editors-data-manipulation)
      - [String](#string-converters), [type conversion](#type-conversion), and [type checking](#type-checking)
      - [Lambda converters](#lambda-converters-v0157-alpha), [patterns](#pattern-matching), and [parsing](#data-parsing)
      - [Collections](#collections), [date/time](#datetime), [encoding](#hashing--encoding), and [OpenTelemetry-specific](#opentelemetry-specific)
      - [Safe function patterns](#function-patterns)
      
      ## Transform-processor-only functions
      
      The `transform` processor adds the following functions to the common OTTL
      catalog. They are not generally available in other OTTL-consuming components.
      The contexts and signatures below are pinned to the released
      [v0.162.0 transform processor source](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.162.0/processor/transformprocessor/README.md#supported-functions).
      
      | Context | Signature | Behavior and limits |
      |---------|-----------|---------------------|
      | `metric` | `convert_sum_to_gauge()` | Sum to Gauge; other metric types are unchanged. The conversion can violate Gauge semantics. |
      | `metric` | `convert_gauge_to_sum(aggregation_temporality, is_monotonic)` | Gauge to Sum; temporality is `"delta"` or `"cumulative"`; other metric types are unchanged. The caller owns the resulting Sum semantics. |
      | `metric` | `extract_avg_metric(suffix?)` | Creates a Gauge of sum/count per datapoint from Histogram, ExponentialHistogram, or Summary; default suffix `_avg`; skips datapoints with an absent sum or zero count. Added in v0.162. |
      | `metric` | `extract_count_metric(is_monotonic, suffix?)` | Creates a Sum from Histogram, ExponentialHistogram, or Summary counts; default suffix `_count`; only creates output when datapoints exist. |
      | `metric` | `extract_percentile_metric(percentile, suffix?)` | Creates a Gauge from Histogram or ExponentialHistogram buckets; `0 < percentile < 100`; default suffix `_p{percentile}`; the result is an interpolated estimate. |
      | `metric` | `extract_sum_metric(is_monotonic, suffix?)` | Creates a Sum from Histogram, ExponentialHistogram, or Summary sums; default suffix `_sum`; skips datapoints whose sum is absent. |
      | `datapoint` | `convert_summary_count_val_to_sum(aggregation_temporality, is_monotonic, suffix?)` | Creates a Sum from a Summary count; temporality is `"delta"` or `"cumulative"`; default suffix `_count`. |
      | `metric` | `convert_summary_quantile_val_to_gauge(attribute_key?, suffix?)` | Creates one Gauge datapoint per Summary quantile; defaults are attribute `quantile` and suffix `.quantiles`. |
      | `datapoint` | `convert_summary_sum_val_to_sum(aggregation_temporality, is_monotonic, suffix?)` | Creates a Sum from a Summary sum; temporality is `"delta"` or `"cumulative"`; default suffix `_sum`. |
      | `metric` | `copy_metric(name?, description?, unit?)` | Appends a full copy, optionally overriding metadata. The copy runs through later metric statements, so guard it with a condition that excludes the copy. |
      | `metric` | `scale_metric(factor, unit?)` | Scales Gauge, Sum, Histogram, and Summary values; optionally changes the unit. |
      | `metric` | `aggregate_on_attributes(function, attributes?)` | Aggregates Sum, Gauge, Histogram, or ExponentialHistogram datapoints. Functions are `sum`, `max`, `min`, `mean`, `median`, or `count`; histogram types support only `sum`. Omitted attributes retain all keys, while `[]` drops all keys. |
      | `metric` | `convert_exponential_histogram_to_histogram(distribution, explicit_bounds)` | Converts ExponentialHistogram to explicit Histogram using `upper`, `midpoint`, `uniform`, or `random`; bounds must be non-empty and strictly increasing. Since v0.162 it converts positive, negative, and zero buckets and preserves the total count. This lossy conversion is not specified by OpenTelemetry. |
      | `metric` | `aggregate_on_attribute_value(function, attribute, values, new_value)` | Aggregates selected attribute values for Sum, Gauge, Histogram, or ExponentialHistogram datapoints; histogram types support only `sum`. |
      | `datapoint` | `merge_histogram_buckets(target_value, method?)` | Explicit Histograms only. Default `remove_explicit_bound` removes a matching bound; `limit_buckets` requires a positive integer target and reduces resolution. Other metric types are unchanged. |
      | `log` | `ParseCEF(target)` | Parses Common Event Format into a map, including an optional syslog prefix and string-valued extensions; malformed or empty input errors. |
      | `log` | `ParseCLF(target, format?)` | Parses CLF (`"clf"`, default) or NCSA combined (`"combined"`) text into a map; malformed or empty input errors. |
      | `log` | `ParseELF(target)` | Parses a complete W3C Extended Log Format block into directive metadata, fields, and entries; requires a `#Version` directive and `#Fields` before data. Added in v0.158. |
      | `log` | `ParseLEEF(target)` | Parses LEEF 1.0/2.0 into a map; malformed or empty input errors; attribute values remain strings. |
      | `span` | `set_semconv_span_name(semconv_version, original_span_name_attribute?)` | Derives low-cardinality HTTP, RPC, messaging, or database span names. v0.162 and v0.163 accept semantic-convention versions 1.37.0 through 1.43.0 (verified 2026-10-15; [v0.163.0 README](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.163.0/processor/transformprocessor/README.md#set_semconv_span_name)); unrelated spans are unchanged. |
      
      For full behavior, examples, and edge cases, follow the tag-pinned
      [metrics](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.162.0/processor/transformprocessor/README.md#convert_sum_to_gauge),
      [logs](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.162.0/processor/transformprocessor/README.md#parsecef), and
      [traces](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.162.0/processor/transformprocessor/README.md#set_semconv_span_name)
      function sections. The registrations that constrain the contexts are also tag-pinned:
      [metric/datapoint](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.162.0/processor/transformprocessor/internal/metrics/functions.go),
      [log](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.162.0/processor/transformprocessor/internal/logs/functions.go), and
      [span](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.162.0/processor/transformprocessor/internal/traces/functions.go).
      
      ## Editors (data manipulation)
      
      Editors are lowercase. Every OTTL statement contains exactly one.
      
      ### `set`
      ```ottl
      set(target, value)
      set(span.attributes["env"], "production")
      set(log.body, Concat([log.severity_text, ": ", log.body.string], ""))
      ```
      
      `set` passes `nil` directly to the target: a map or slice path is cleared, a `pcommon.Value` (for
      example a missing attribute key) is set to an empty value, and scalar or struct-typed targets return
      an error. This is the `ottl.set.allowNil` behavior, beta (on by default) in v0.161 and stable in
      v0.162; through v0.160 a nil value was a no-op. Guard missing sources with `where source != nil`
      when the target must stay unchanged.
      
      ### `clear` (v0.160+)
      ```ottl
      clear(target)
      clear(span.attributes["http.request.header.authorization"])
      ```
      
      Sets the target to the empty value for its current type: for example, `""`, `0`, `false`, an empty
      `pcommon.Map`, `pcommon.Slice`, or `pcommon.Value`, or a zero TraceID/SpanID/time. For Go maps,
      slices, pointers, byte slices, and a current `nil` value, it passes `nil`; whether the path setter
      accepts that value is target-specific and may produce an evaluation error.
      
      ### `append`
      ```ottl
      append(target, value)
      append(target, values = ["tag1", "tag2"])
      append(span.attributes["tags"], "processed")
      ```
      
      If `target` is a scalar, it is converted to a slice first.
      
      ### `delete_key` / `delete_matching_keys` / `delete_index`
      ```ottl
      delete_key(span.attributes, "internal.debug")
      delete_matching_keys(span.attributes, "(?i).*password.*")
      delete_index(log.attributes["tags"], 0)        # v0.145+, one slice index
      delete_index(log.attributes["tags"], 0, 3)     # delete indices 0, 1, 2
      ```
      
      ### `keep_keys` / `keep_matching_keys`
      ```ottl
      keep_keys(span.attributes, ["http.method", "http.status_code", "http.route"])
      keep_matching_keys(span.attributes, "(?i)^http\\..*")
      ```
      
      ### `flatten`
      ```ottl
      flatten(target)
      flatten(target, prefix, depth, resolveConflicts?)   # v0.139+
      flatten(log.body)
      flatten(span.attributes, "nested", 2)
      flatten(log.attributes, resolveConflicts = true)
      ```
      
      ### `limit`
      ```ottl
      limit(target, count, priority_keys[])
      limit(span.attributes, 10, ["http.method", "http.status_code"])
      ```
      
      ### `merge_maps`
      ```ottl
      merge_maps(target, source, "insert" | "update" | "upsert")
      merge_maps(span.attributes, ParseJSON(span.attributes["meta"]), "upsert")
      ```
      
      - `insert` — only set keys not already in target
      - `update` — only overwrite keys already in target
      - `upsert` — both
      
      ### `truncate_all`
      ```ottl
      truncate_all(target, max_length)
      truncate_all(target, max_length, utf8_safe = false)   # v0.148+
      truncate_all(target, max_length, truncation_marker = "(truncated)") # v0.157+
      truncate_all(span.attributes, 256)
      ```
      
      UTF-8 safe by default since v0.148 — truncates at character boundaries, so the result may be slightly shorter than `max_length`. Pass `utf8_safe = false` for the previous byte-level behavior.
      
      The optional `truncation_marker` is included inside `max_length`, so the complete result never exceeds the limit. A marker longer than the limit is an error.
      
      ### `stringify_all`
      ```ottl
      stringify_all(target)
      stringify_all(log.attributes)
      stringify_all(resource.attributes)
      ```
      
      Converts every non-string value in a map to its string representation. Numbers and booleans become scalar strings, bytes become base64, maps and slices become JSON strings, and empty values become `""`. Added in collector-contrib v0.155.0.
      
      ### Dynamic indexing of converter results
      
      Converter results can be indexed with expressions that evaluate to a string or integer, not only literals:
      
      ```ottl
      Split(log.body.string, ",")[log.attributes["index"]]
      ParseJSON(log.body.string)[log.attributes["field_name"]]
      ```
      
      This was added in collector-contrib v0.155.0. On older collectors, keep converter-result indexes literal, such as `[0]` or `["field"]`.
      
      ### `replace_match` / `replace_all_matches`
      ```ottl
      replace_match(target, pattern, replacement, function?, format?)
      replace_all_matches(target, pattern, replacement, function?, format?)
      replace_match(span.name, "GET ", "")
      replace_all_matches(span.attributes, "/user/*/id/*", "/user/{userId}/id/{id}")
      ```
      
      Glob patterns (not regex). Use `replace_pattern`/`replace_all_patterns` for regex.
      
      ### `replace_pattern` / `replace_all_patterns`
      ```ottl
      replace_pattern(target, regex, replacement, function?, format?)
      replace_all_patterns(target, "key" | "value", regex, replacement, function?, format?)
      
      # Strip query string
      replace_pattern(attributes["http.url"], "\\?.*$", "")
      
      # Redact a token query parameter (YAML form; in a raw OTTL string use ${1})
      replace_pattern(attributes["http.url"], "([?&]token=)[^&]+", "$${1}REDACTED")
      
      # Map-wide patterns over keys or values
      replace_all_patterns(span.attributes, "value", "\\d{16}", "[REDACTED]")
      replace_all_patterns(span.attributes, "key",   "^kube_", "k8s.")
      ```
      
      **Backreferences in YAML.** OTTL uses `${1}`, `${2}`, … The collector's YAML loader treats `$` as an env-var marker, so a single `$` in YAML produces nothing in the OTTL string. Write `$${1}` in YAML to emit `${1}` in OTTL. Writing `"$1REDACTED"` produces a literal `$1REDACTED` — silent failure.
      
      **RE2 repeat-count limit.** Patterns like `^(.{1024}).*` fail to compile with "invalid repeat count". For length-based truncation, use `Substring` + `Len` instead:
      
      ```ottl
      set(attributes["db.statement"], Substring(attributes["db.statement"], 0, 1024))
          where attributes["db.statement"] != nil
            and Len(attributes["db.statement"]) > 1024
      ```
      
      ---
      
      ## String converters
      
      ### `Concat`, `Split`, `Substring`
      ```ottl
      Concat(["user", span.attributes["user.id"], "action"], "-")  # "user-123-action" when user.id == "123"
      Split(span.name, "/")                         # ["", "api", "v1", "users"]
      Split(span.name, "/")[1]                      # "api"
      Substring(span.span_id.string, 0, 8)          # first 8 chars
      Substring(target, start, length, utf8_safe?)  # byte offsets; utf8_safe defaults false; added v0.156
      Concat(Split(span.attributes["request.id"], "-"), "")   # v0.161+: values may be a slice-valued path or converter
      ```
      
      With `utf8_safe=true`, a start inside a multi-byte character advances to the next UTF-8 boundary and an end inside one backs up to the previous boundary. The result stays valid UTF-8 but can be shorter than `length` bytes.
      
      ### Case
      ```ottl
      ToLowerCase(s) / ToUpperCase(s)
      ToCamelCase(s) / ToSnakeCase(s)
      ConvertCase(s, "lower" | "upper" | "camel" | "snake")
      ```
      
      ### Trim
      ```ottl
      Trim(s, cutset?)
      TrimPrefix(s, prefix)        # v0.124+
      TrimSuffix(s, suffix)        # v0.124+
      ```
      
      ### Prefix/suffix tests
      ```ottl
      HasPrefix(s, prefix)         # bool
      HasSuffix(s, suffix)         # bool
      where HasPrefix(span.name, "internal.")
      ```
      
      ### `Format`
      ```ottl
      Format("user=%s req=%d", [span.attributes["user.id"], span.attributes["request.count"]])
      ```
      
      ---
      
      ## Type conversion
      
      ```ottl
      String(v)                # any -> string
      Int(v)                   # any -> int64
      Double(v)                # any -> float64
      Bool(v)                  # v0.143+; loose coercion (see below)
      ParseInt(s, base)        # ParseInt("ff", 16) -> 255
      Hex(bytes)               # bytes -> hex string
      ```
      
      **`Bool` coercion is loose, not Pythonic.** Booleans pass through; any non-zero integer or double is `true`; zero is `false`; strings use Go boolean parsing (`1`, `t`, `T`, `TRUE`, `true`, `True`, and false equivalents). Invalid strings and unsupported types error, while `nil` returns `nil`.
      
      ---
      
      ## Type checking
      
      ```ottl
      IsString(v) / IsInt(v) / IsDouble(v) / IsBool(v)
      IsList(v) / IsMap(v)
      IsEmpty(v)                                               # nil, zero value, or empty collection
      IsInCIDR(ip_string, ["10.0.0.0/8", "192.168.0.0/16"])   # v0.146+
      ```
      
      `IsInCIDR` returns `false` for invalid IP strings — useful in conditions: `where IsInCIDR(attributes["client.ip"], ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"])`.
      
      ---
      
      ## Lambda converters (v0.157+, alpha)
      
      These converters require the alpha `ottl.functions.enableLambda` feature gate.
      For slices, two-argument lambdas receive `(index, value)`; for maps they receive
      `(key, value)`. Use `_` for an ignored parameter.
      
      ```ottl
      All(source, (k, v) => predicate)                 # every element; true for empty input
      Any(source, (k, v) => predicate)                 # at least one; false for empty input
      Filter(source, (k, v) => predicate)              # filtered slice or map
      Find(source, (k, v) => predicate, mapper?)        # first value, mapped value, or nil
      MapEach(source, (k, v) => mapped_value)           # transform values
      MapKeys(map, (k, v) => mapped_key)                # mapped_key must be a string
      Reduce(source, seed, (acc, k, v) => next_acc)     # fold; empty input returns seed
      When(() => condition, true_value, false_value)    # select one value
      ```
      
      Map iteration order is not stable, so order-dependent `Reduce` expressions are
      not deterministic for maps. If `MapKeys` creates duplicate keys, which value is
      retained is unspecified.
      
      ---
      
      ## Pattern matching
      
      ### `IsMatch`
      ```ottl
      IsMatch(target, pattern)     # bool
      where IsMatch(span.name, "^GET /api/.*")
      where not IsMatch(span.name, ".*health.*")
      ```
      
      ### `ExtractPatterns`
      ```ottl
      ExtractPatterns(target, "(?P<user>\\w+)/(?P<action>\\w+)")
      # Returns map keyed by the named groups.
      ```
      
      Named capture groups required — unnamed groups are not surfaced as map keys.
      
      ### `ExtractGrokPatterns` (v0.130+)
      ```ottl
      ExtractGrokPatterns(log.body, "%{IP:client_ip} %{WORD:method} %{URIPATHPARAM:path}")
      ExtractGrokPatterns(log.body, "%{URI}", true)                         # named only
      ExtractGrokPatterns(log.body, "%{MYPATTERN}", true,
                         ["MYPATTERN=%{IP}:%{INT}"])                         # custom defs
      ```
      
      Built-in pattern library covers HTTP, syslog, paths, IPs, dates, etc. Cheaper than expressing the same thing as a raw regex.
      
      ---
      
      ## Data parsing
      
      ```ottl
      ParseJSON(s)                                  # JSON string -> map | list
      ParseCSV(s, headers, delimiter?, headerDelim?, mode?)
      ParseKeyValue(s, kv_delim?, pair_delim?)      # ParseKeyValue("a=1&b=2", "=", "&")
      ParseXML(s) / ParseSimplifiedXML(s)
      ParseSeverity(value, mapping)                 # map values -> SEVERITY_NUMBER_*
      URL(s)                                        # v0.127+; { url.scheme, url.domain, url.port, url.path, … }
      UserAgent(s)                                  # v0.134+; { user_agent.name, .version, os.name, os.version, … }
      ```
      
      `ParseJSON`, `IsString`, and `IsMatch` are available in v0.156 and do not require the v0.157
      `ottl.functions.enableLambda` feature gate.
      
      ```ottl
      # Conditional JSON parsing
      set(log.attributes, ParseJSON(log.body.string))
          where IsString(log.body) and IsMatch(log.body.string, "(?s)^\\s*\\{.*\\}\\s*$")
      
      # URL parsing
      set(span.attributes["http.host"], URL(span.attributes["http.url"])["url.domain"])
      ```
      
      ### Fallback values
      ```ottl
      Coalesce([span.attributes["user.id"], resource.attributes["user.id"], "fallback"])
      ```
      
      ---
      
      ## XML
      
      ```ottl
      GetXML(target, xpath)                         # select elements
      InsertXML(target, xpath, value)
      RemoveXML(target, xpath)
      ConvertAttributesToElementsXML(target, xpath?)
      ConvertTextToElementsXML(target, xpath?, elementName?)
      ```
      
      ---
      
      ## Collections
      
      ```ottl
      Len(target)                                   # works for string, list, map
      Keys(map) / Values(map)                       # -> list
      ContainsValue(list, value)                    # bool
      Index(target, value)                          # v0.126+; -1 if not found
      Sort(list, "asc" | "desc")                    # v0.125+
      SliceToMap(list, [keyPath]?, [valuePath]?)    # v0.128+; turn objects into a map
      ```
      
      ```ottl
      SliceToMap(resource.attributes["items"], ["name"])                     # key by .name
      SliceToMap(resource.attributes["items"], ["name"], ["value"])          # explicit value
      ```
      
      ---
      
      ## Date/time
      
      ```ottl
      Now()                                         # current time
      Time(s, format, location?, locale?)           # locale added v0.136
      FormatTime(t, format)
      Duration(s)                                   # "1h30m" -> time.Duration
      TruncateTime(t, duration)
      Unix(seconds, nanoseconds?)
      UnixSeconds(t) / UnixMilli(t) / UnixMicro(t) / UnixNano(t)
      Year(t) / Month(t) / Day(t) / Hour(t) / Minute(t) / Second(t) / Nanosecond(t) / Weekday(t)
      Hours(d) / Minutes(d) / Seconds(d) / Milliseconds(d) / Microseconds(d) / Nanoseconds(d)
      ```
      
      ---
      
      ## Hashing & encoding
      
      ```ottl
      SHA256(s) / SHA512(s)
      SHA1(s) / MD5(s)                              # cryptographically weak; avoid for security
      FNV(s)                                        # int64
      Murmur3Hash(s) / Murmur3Hash128(s)            # v0.129+; hexadecimal string
      XXH3(s) / XXH128(s)                           # v0.135+; hexadecimal string
      Decode(value, encoding)                       # v0.141+; "base64", "base64-raw", "base64-url", "base64-raw-url", IANA charsets
      Base64Encode(s, variant?)                     # v0.147+; default base64
      Hex(bytes)
      ```
      
      ---
      
      ## OpenTelemetry-specific
      
      ```ottl
      SpanID(bytes | hex_string)                    # hex strings accepted v0.142+
      TraceID(bytes | hex_string)                   # same
      ProfileID(bytes | hex_string)                 # v0.124+
      IsRootSpan()                                  # span context only
      IsValidLuhn(s)                                # credit card checksum
      CommunityID(srcIP, srcPort, dstIP, dstPort, protocol?, seed?)   # v0.131+; network flow hash
      ToKeyValueString(map, delim?, pair_delim?, sort?)
      ```
      
      ---
      
      ## Identifier generation
      
      ```ottl
      UUID()
      UUIDv7()                                      # v0.138+; time-ordered
      ```
      
      `UUIDv7` is time-ordered; choose the identifier form according to the consuming system's requirements.
      
      ---
      
      ## Math
      
      ```ottl
      Log(value)                                    # natural logarithm
      ```
      
      OTTL's basic arithmetic operators (`+`, `-`, `*`, `/`) cover most needs; `Log` is the rare named math converter.
      
      ---
      
      ## Function patterns
      
      ### Safe type conversion
      ```ottl
      set(span.attributes["count_int"], Int(span.attributes["count"]))
          where IsString(span.attributes["count"])
            and IsMatch(span.attributes["count"], "^\\d+$")
      ```
      
      ### Chained string operations
      ```ottl
      set(span.attributes["normalized"],
          ToLowerCase(Trim(span.attributes["value"], " ")))
      ```
      
      ### Conditional parsing into cache
      ```ottl
      set(log.cache["parsed"], ParseJSON(log.body.string))
          where IsString(log.body) and IsMatch(log.body.string, "(?s)^\\s*\\{.*\\}\\s*$")
      ```
      
      Then read from `log.cache["parsed"]` in subsequent statements without paying the parse cost again.
      
      ### Hash-based sampling
      ```ottl
      set(span.attributes["sampled"], true)
          where IsMatch(XXH3(span.trace_id.string), "^(0[0-9a-f]|1[0-9])")  # ~10.2%
      ```
      
    • quick-reference.md 12.7 KB
      # OTTL Quick Reference
      
      Recipes, regex patterns, and a troubleshooting table. Pair with `contexts.md` for paths and `functions.md` for full signatures.
      
      ## Contents
      
      - [Common patterns](#common-patterns)
      - [Processor configuration](#processor-configuration)
      - [Regular expressions](#regular-expressions)
      - [Debugging](#debugging)
      - [Performance](#performance)
      - [Troubleshooting and verification](#troubleshooting)
      
      ## Common patterns
      
      ### Attribute management
      
      ```ottl
      # Set attribute
      set(span.attributes["environment"], "production")
      
      # Set if missing
      set(span.attributes["version"], "unknown")
          where span.attributes["version"] == nil
      
      # Copy from resource (guard: since v0.161 a nil source creates an empty-valued key)
      set(span.attributes["service"], resource.attributes["service.name"])
          where resource.attributes["service.name"] != nil
      
      # Rename
      set(span.attributes["host.name"], span.attributes["hostname"])
          where span.attributes["hostname"] != nil
      delete_key(span.attributes, "hostname")
      
      # Drop sensitive keys
      delete_matching_keys(span.attributes, "(?i).*(password|secret|token|apikey).*")
      
      # Whitelist (fail-closed)
      keep_keys(span.attributes, ["http.method", "http.status_code", "http.route"])
      ```
      
      ### String manipulation
      
      ```ottl
      set(span.attributes["method"], ToUpperCase(String(span.attributes["http.method"])))
      set(span.attributes["short_id"], Substring(span.span_id.string, 0, 8))
      set(span.attributes["service"], Split(resource.attributes["service.name"], "-")[0])
      set(log.body, Concat([log.severity_text, ": ", log.body.string], ""))
      set(log.body, Trim(log.body.string, " \t\n"))
      ```
      
      ### Data parsing
      
      ```ottl
      # JSON body -> attributes
      set(log.attributes, ParseJSON(log.body.string))
          where IsString(log.body) and IsMatch(log.body.string, "(?s)^\\s*\\{.*\\}\\s*$")
      
      # Query string -> map
      set(span.attributes["params"],
          ParseKeyValue(span.attributes["http.query"], "=", "&"))
      
      # URL parts (v0.127+)
      set(span.attributes["http.host"], URL(span.attributes["http.url"])["url.domain"])
      
      # User agent (v0.134+)
      set(span.attributes["client.os"],
          UserAgent(span.attributes["http.user_agent"])["os.name"])
      
      # Grok (v0.130+) — much cleaner than the equivalent regex
      set(log.attributes,
          ExtractGrokPatterns(log.body.string,
                              "%{IP:client_ip} %{WORD:method} %{URIPATHPARAM:path}"))
      ```
      
      ### Error / status detection
      
      ```ottl
      # HTTP errors
      set(span.status.code, STATUS_CODE_ERROR)
          where IsInt(span.attributes["http.status_code"])
            and Int(span.attributes["http.status_code"]) >= 400
      
      # Exception spans
      set(span.status.code, STATUS_CODE_ERROR)
          where span.attributes["exception.type"] != nil
      set(span.status.message, span.attributes["exception.message"])
          where IsString(span.attributes["exception.message"])
      
      # Slow spans
      set(span.attributes["slow"], true)
          where (span.end_time_unix_nano - span.start_time_unix_nano) > 1000000000
      ```
      
      ### Filter conditions
      
      ```ottl
      # Skip health checks
      where not IsMatch(span.name, ".*(health|ready|alive).*")
      
      # Errors only
      where log.severity_number >= SEVERITY_NUMBER_ERROR
      
      # Server spans only
      where span.kind == SPAN_KIND_SERVER
      
      # Drop traffic from internal CIDRs
      where IsInCIDR(span.attributes["client.address"],
                     ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"])
      ```
      
      ### Sampling
      
      ```ottl
      # Hash-based ~10.2%; 26 of 256 possible first-byte values match
      set(span.attributes["sampled"], true)
          where IsMatch(XXH3(span.trace_id.string), "^(0[0-9a-f]|1[0-9])")
      
      # Always-sample errors
      set(span.attributes["sampled"], true)
          where span.status.code == STATUS_CODE_ERROR
      
      # Latency-based
      set(span.attributes["sampled"], true)
          where (span.end_time_unix_nano - span.start_time_unix_nano) > 1000000000
      ```
      
      ### Time operations
      
      ```ottl
      # Duration in ms
      set(span.attributes["duration_ms"],
          (span.end_time_unix_nano - span.start_time_unix_nano) / 1000000)
      
      # Format timestamp
      set(log.attributes["formatted_time"], FormatTime(log.time, "%Y-%m-%d %H:%M:%S"))
      
      # Stamp processing time
      set(span.attributes["processed_at"], Now())
      
      # Hour bucket
      set(log.attributes["hour"], Hour(log.time))
      ```
      
      ### Redaction
      
      ```ottl
      # Credit cards
      replace_all_patterns(log.attributes, "value",
                           "\\b(?:\\d{4}[\\s-]?){3}\\d{4}\\b",
                           "****-****-****-****")
      
      # Emails
      replace_all_patterns(log.attributes, "value",
                           "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}",
                           "[REDACTED_EMAIL]")
      
      # IPv4
      replace_all_patterns(log.attributes, "value",
                           "\\b\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\b",
                           "[REDACTED_IP]")
      
      # Deterministic hash example; policy must permit retaining linkable identifiers
      set(span.attributes["user.email_hash"], SHA256(span.attributes["user.email"]))
      delete_key(span.attributes, "user.email")
      ```
      
      ## Processor configuration
      
      ### `transform`
      
      ```yaml
      processors:
        transform:
          error_mode: ignore        # ignore | silent | propagate
          trace_statements:
            - context: span
              statements:
                - set(span.attributes["processed"], true)
          log_statements:
            - context: log
              statements:
                - set(log.attributes["source"], "collector")
          metric_statements:
            - context: datapoint
              statements:
                - set(datapoint.attributes["env"], "prod")
      ```
      
      Set `shared_cache: true` only on advanced statement groups that must share context cache state
      across separate iterations. Sharing occurs only among groups in the same transform processor
      instance that opt in; the option is experimental and defaults to `false`.
      
      ### `filter`
      
      ```yaml
      processors:
        filter:
          error_mode: ignore
          trace_conditions:
            - 'IsMatch(span.name, "^/health.*")'
            - 'span.kind == SPAN_KIND_INTERNAL'
          log_conditions:
            - 'log.severity_number < SEVERITY_NUMBER_WARN'
      ```
      
      ### `routing` connector
      
      ```yaml
      connectors:
        routing:
          error_mode: ignore
          default_pipelines: [traces/default]
          table:
            - condition: 'resource.attributes["env"] == "prod"'
              pipelines: [traces/prod]
            - condition: 'span.status.code == STATUS_CODE_ERROR'
              pipelines: [traces/errors]
      ```
      
      With `error_mode: ignore`, an evaluation error is logged and the payload is sent to
      `default_pipelines`; without a configured fallback it is dropped. Metadata-based routing also
      requires `include_metadata: true` on the receiving OTLP protocol. Use
      `otelcol.client.metadata["X-Tenant"][0]` for HTTP/client metadata and lowercase
      `otelcol.grpc.metadata["x-tenant"][0]` for gRPC metadata.
      
      ### `tail_sampling`
      
      ```yaml
      processors:
        tail_sampling:
          policies:
            - name: errors
              type: ottl_condition
              ottl_condition:
                error_mode: ignore
                span:
                  - 'span.status.code == STATUS_CODE_ERROR'
            - name: slow
              type: ottl_condition
              ottl_condition:
                error_mode: ignore
                span:
                  - '(span.end_time_unix_nano - span.start_time_unix_nano) > 1000000000'
      ```
      
      Set the OTTL policy's `error_mode` explicitly; it is nested under `ottl_condition`, not at the
      `tail_sampling` processor level.
      
      ## Regular expressions
      
      ### Common patterns
      
      ```ottl
      "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}"                          # email
      "\\b\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\b"                          # IPv4
      "\\b(?:\\d{4}[\\s-]?){3}\\d{4}\\b"                                          # credit card
      "\\b\\d{3}-\\d{3}-\\d{4}\\b"                                                # US phone
      "https?://[^\\s]+"                                                          # URL
      "[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}"  # UUID
      "\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}(?:\\.\\d{3})?Z?"                 # ISO 8601
      "^\\{.*\\}$"                                                                # JSON object
      "^\\d+$"                                                                    # digits only
      ```
      
      ### Escape rules
      
      In OTTL strings, escape regex metacharacters with double backslash. Single-backslash YAML chars get eaten before OTTL sees them.
      
      | You want in regex | Write in OTTL string |
      |-------------------|---------------------|
      | `\d` | `\\d` |
      | `\s` | `\\s` |
      | `\w` | `\\w` |
      | `\.` | `\\.` |
      | `\*` | `\\*` |
      | `\+` | `\\+` |
      
      **Backreference escape (YAML).** OTTL `${1}` must be written as `$${1}` in YAML — the loader treats `$` as an env-var marker. `"$1REDACTED"` becomes a literal string at the OTTL level, no replacement.
      
      **RE2 repeat-count limit.** `(.{1024}).*` fails to compile. Use `Substring` + `Len` for length truncation:
      
      ```ottl
      set(attributes["x"], Substring(attributes["x"], 0, 1024))
          where attributes["x"] != nil and Len(attributes["x"]) > 1024
      ```
      
      ## Debugging
      
      ### Enable debug logging
      
      ```yaml
      service:
        telemetry:
          logs:
            level: debug
      ```
      
      ### Add probes
      
      ```ottl
      # Track that a statement ran
      set(span.attributes["ottl.processed"], true)
      set(span.attributes["ottl.timestamp"], Now())
      
      # ~0.1% sampled debug snapshot (4 of 4096 three-hex-digit prefixes)
      set(span.attributes["debug.original_name"], span.name)
          where IsMatch(XXH3(span.trace_id.string), "^00[0-3]")
      ```
      
      ### Validate before transforming
      
      ```ottl
      where IsString(span.attributes["count"])
        and IsMatch(span.attributes["count"], "^\\d+$")
      
      where span.attributes["value"] != nil
      
      where Len(String(log.body)) > 0
      ```
      
      ## Performance
      
      **Do**
      
      - Order conditions by selectivity (cheap and most-discriminating first).
      - Use early filtering to drop work at the smallest scope.
      - Cache expensive function results in `<context>.cache`.
      - Pick the most specific context (operate on `datapoint`, not on `metric.data_points`).
      - Batch attribute pruning with `keep_keys` rather than many `delete_key` statements.
      
      **Don't**
      
      - Repeat expensive function calls inside the same statement set.
      - Run complex regex on large bodies.
      - Process telemetry that a cheaper filter could drop earlier in the pipeline.
      - Suppress error_mode noise (`silent`) without first verifying behavior.
      
      ### Cache pattern
      
      ```ottl
      set(span.cache["url_parts"], Split(span.attributes["http.url"], "/"))
      set(span.attributes["api_version"], span.cache["url_parts"][2])
      ```
      
      ### Early-filter ordering
      
      ```ottl
      # Cheap kind check fails first; regex only runs for SERVER spans.
      where span.kind == SPAN_KIND_SERVER and IsMatch(span.name, "expensive.*")
      ```
      
      ## Troubleshooting
      
      ### Common errors
      
      | Symptom | Cause | Fix |
      |---------|-------|-----|
      | `nil value`, or an empty attribute appears | Path resolves to a missing field; since v0.161 `set` passes nil through | Add `where path != nil` (or `IsString(path)` etc.) |
      | `type mismatch` | Wrong types in arithmetic / function args | Convert with `Int`, `Double`, `String` after type-checking |
      | `invalid regex` | Unescaped metacharacter | Double-escape: `\\d`, `\\.`, `\\s` |
      | `division by zero` | `/` with a zero divisor | Add `where divisor != 0` |
      | Backreferences appear literal | YAML ate the `$` | Use `$${1}` in YAML for OTTL `${1}` |
      | `attributes` processor doesn't touch resource | Wrong processor | Use `resource` processor or `transform` with `context: resource` |
      | `cache["x"]` errors in span context | Cache requires a context prefix since v0.120 | Write `span.cache["x"]` |
      | `Bool("yes")` errors | String inputs use Go boolean parsing; arbitrary non-empty strings are not truthy | Use `IsMatch` with an explicit pattern, or check with `where` |
      | Unknown function `Base64Decode` | Removed in v0.161 (deprecated since v0.141) | Use `Decode(value, "base64")` |
      
      Since v0.160, parser diagnostics identify the input kind, line and column, nearby source, and the
      expected token when available, for example
      `statement has invalid syntax at 1:5: (expected ")" Key*)`. Odd-length byte literals instead report
      `byte literals must have an even number of hexadecimal digits` within the wrapped expression error.
      
      ### Checklist before shipping
      
      1. Syntax: parens balanced, strings closed, regex escapes doubled.
      2. Types: type-check before conversion (`IsString`, `IsInt`).
      3. Existence: nil-guard paths that may be absent.
      4. Logic: dry-run with the [telemetrygen verification recipe](../../otel-telemetrygen/SKILL.md#verify-collector-behavior) — generate a known input, capture the file-exporter output, confirm the transformation. The eye test misses too many gotchas.
      5. Performance: most-selective `where` clause first.
      
      ### Safe transformation skeletons
      
      ```ottl
      # Safe JSON parse
      set(log.cache["parsed"], ParseJSON(log.body.string))
          where IsString(log.body)
            and IsMatch(log.body.string, "^\\s*[\\{\\[].*[\\}\\]]\\s*$")
      
      # Safe numeric conversion
      set(span.attributes["count_int"], Int(span.attributes["count"]))
          where IsString(span.attributes["count"])
            and IsMatch(span.attributes["count"], "^-?\\d+$")
      
      # Safe duration calc
      set(span.attributes["duration_ns"],
          span.end_time_unix_nano - span.start_time_unix_nano)
          where span.end_time_unix_nano > span.start_time_unix_nano
      ```
      
  • SKILL.md 6.7 KB
    ---
    name: otel-ottl
    description: OpenTelemetry Transformation Language (OTTL) expert for writing and debugging telemetry transformations in the OpenTelemetry Collector. Use when authoring or reviewing `transform`, `filter`, `tail_sampling` processor configs or `routing` connector configs, debugging OTTL syntax or semantics, transforming traces, metrics, logs, or profiles, or converting data-processing requirements into OTTL statements.
    ---
    
    # OpenTelemetry Transformation Language (OTTL)
    
    OTTL transforms or selects telemetry inside Collector components. This skill is pinned to
    collector-contrib **v0.162.0**. Function, path, default, and feature-gate availability varies by
    release; when the user's version differs, verify against the matching upstream tag.
    
    Dated OTTL feature-gate status for v0.162.0 and v0.163.0 is in
    [references/functions.md](references/functions.md#feature-gates-verified-2026-10-15).
    
    ## Workflow
    
    1. **Choose the component.** `transform` rewrites, `filter` drops, `tail_sampling` decides whether
       to retain traces, and `routing` sends telemetry to pipelines. A component controls its available
       contexts and functions.
    2. **Choose the lowest usable context.** Lower contexts can read their parents (for example, a span
       can read `resource.attributes`), but parents cannot read children. Use `datapoint` for point
       attributes instead of traversing `metric.data_points`.
    3. **Verify every emitted function, then write the statement.** Confirm each function's exact
       identifier and signature in [references/functions.md](references/functions.md). If an identifier
       is absent, treat it as unsupported instead of deriving or substituting a plausible name. An
       editor such as `set` or `delete_key` mutates data and may have a `where` condition. Converters
       such as `ParseJSON` and `IsMatch` return values; they do not mutate.
    4. **Set error behavior deliberately.** `ignore` logs statement errors and continues; `silent`
       continues without logging; `propagate` returns the error and can cause the component to drop the
       payload. Transform and filter default to `ignore`; the transform default-error gate was removed in
       v0.162.0, so naming it in `--feature-gates` fails startup (history: the `otel-collector` skill's
       transform quirks), and the filter gate is stable and cannot be disabled. Routing also
       defaults to `ignore` while its beta default-error feature gate is enabled. For routing, `ignore`
       sends an errored payload to `default_pipelines`; configure that fallback or the payload is dropped.
    5. **Verify end to end.** Validate the exact Collector version, then send known telemetry and inspect
       file-exporter output. Use the
       [telemetrygen recipe](../otel-telemetrygen/SKILL.md#verify-collector-behavior).
    
    ```ottl
    set(span.attributes["env"], "prod") where resource.attributes["env"] == nil
    ```
    
    ## Load only what the task needs
    
    - [Contexts](references/contexts.md) — exact paths, hierarchy, enums, and request metadata.
    - [Functions](references/functions.md) — editor/converter signatures and release availability.
    - [Quick reference](references/quick-reference.md) — component YAML, recipes, escaping,
      troubleshooting, and safe skeletons.
    
    For a single path or function, read only the relevant section instead of loading the full catalogs.
    
    ## Safety and correctness gates
    
    - Guard optional or polymorphic input before conversion: `where x != nil`, `IsString(x)`, or the
      appropriate type check.
    - For JSON-object-only work, guard both the type and shape before calling `ParseJSON`, for example
      `IsString(log.body) and IsMatch(log.body.string, "(?s)^\\s*\\{.*\\}\\s*$")`. The RE2 `(?s)`
      flag admits pretty-printed objects containing newlines. Checking `IsMap` after
      parsing does not prevent arrays or scalar JSON from being parsed.
    - On a version-pinned request, confirm every chosen path and function against that release tag;
      do not assume a function listed for this skill's v0.162 anchor exists in an older release.
      For v0.156 JSON-object parsing, `ParseJSON`, `IsString`, and `IsMatch` are available without the
      v0.157 alpha lambda feature gate.
    - Request metadata is read-only and may contain credentials. Copy only explicitly allowlisted,
      non-sensitive keys. OTLP metadata routing requires `include_metadata: true` on the receiver.
      HTTP/client header spelling may retain its form (`otelcol.client.metadata["X-Tenant"][0]`);
      gRPC metadata keys are lowercase (`otelcol.grpc.metadata["x-tenant"][0]`).
    - The routing `request` context is deprecated as of v0.156; use `otelcol.client.metadata` or
      `otelcol.grpc.metadata`.
    - Log-record-specific rewrites of shared resource or scope data require `flatten_data: true` and the
      alpha `transform.flatten.logs` gate. This copies and regroups data; do not enable it accidentally.
    - Since v0.161, `set(target, nil)` is no longer a no-op: the `ottl.set.allowNil` gate became beta in
      v0.161 and stable (cannot be disabled) in v0.162. A nil value empties a map, slice, or
      `pcommon.Value` target (an attribute key is created with an empty value) and errors on scalar
      targets. Use a `where source != nil` guard when the destination must remain unchanged for missing input.
    - Hashing an identifier does not necessarily anonymize it. Apply the organization's data-handling
      policy before retaining deterministic hashes of personal data.
    
    ## Frequent syntax traps
    
    - In Collector YAML, write an OTTL replacement backreference `${1}` as `$${1}`. A replacement such
      as `$1REDACTED` is literal and silently fails to substitute the capture.
    - Go RE2 rejects large counted repetitions such as `(.{1024}).*`; use `Substring` with a nil/type
      guard and `Len`, or `truncate_all` for a map.
    - Current span-event paths use `spanevent.*`, not `span_event.*`. Cache paths are context-qualified,
      such as `span.cache["parsed"]`.
    - Quote any OTTL statement containing a map literal when YAML includes a space after `:`, for
      example `'set(log.attributes["a"], {"foo": "bar"})'`; otherwise YAML parses `: ` as a mapping.
    - Since v0.159, polymorphic `pcommon.Value` paths compare by their underlying type; maps and slices
      support only equality and inequality, while primitive values also support ordering.
    - Use `Decode(value, "base64")`; `Base64Decode` was removed in v0.161.
    - Regex escapes inside OTTL strings are doubled (`\\d`, `\\s`, `\\.`).
    
    ## Upstream sources
    
    - [OTTL package](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.162.0/pkg/ottl)
    - [Transform processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.162.0/processor/transformprocessor)
    - [Filter processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.162.0/processor/filterprocessor)
    - [Routing connector](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.162.0/connector/routingconnector)
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related