otel-dotnet
OpenTelemetry in .NET — DI/builder SDK setup (OpenTelemetry.Extensions.Hosting, AddOpenTelemetry, UseOtlpExporter, OpenTelemetrySdk.Create), native .NET instrumentation APIs (ActivitySource, System.Diagnostics.Metrics Meter) plus logging abstractions (ILogger), zero-code CLR-prof
Install
npx skills add https://github.com/ollygarden/opentelemetry-agent-skills/tree/main/skills/otel-dotnet
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install ollygarden-opentelemetry-agent-skills@llmmart
git clone https://github.com/ollygarden/opentelemetry-agent-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole ollygarden/opentelemetry-agent-skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
OpenTelemetry in .NET
Entry point for OpenTelemetry mechanics in .NET services. Load a reference below based on the task; each reference is self-contained.
References
| File | Use when |
|---|---|
references/setup.md |
Setting up the SDK via the DI/builder path (AddOpenTelemetry().WithTracing/WithMetrics/WithLogging, UseOtlpExporter, OpenTelemetrySdk.Create, OpenTelemetry.Extensions.Hosting), exporter wiring, env-var / IConfiguration inputs, and why there is no released declarative YAML config in .NET. |
references/api.md |
Instrumenting with native BCL APIs (ActivitySource/Activity, System.Diagnostics.Metrics.Meter, ILogger), how the SDK subscribes (AddSource/AddMeter), attributes, propagation, and the optional OTel API shim. |
references/instrumentation-libraries.md |
Zero-code (the opentelemetry-dotnet-instrumentation CLR-profiler agent), the contrib instrumentation-package catalog, and manual instrumentation following semconv. |
references/performance.md |
Tuning sampling (built-in samplers incl. AlwaysRecordSampler), batch export processor, periodic metric reader, views, exporter choice, async context, graceful shutdown. |
references/breaking-changes.md |
Auditing existing code for deprecated/renamed APIs and semconv changes across recent core/contrib releases. |
Sources of Truth
For YAML schema details (language-agnostic), see the otel-declarative-config skill — but note
.NET does not yet have a released declarative-config implementation (see references/setup.md).
For .NET-specific facts:
| Fact | Fetch |
|---|---|
Latest OpenTelemetry core / core-* tag |
gh api repos/open-telemetry/opentelemetry-dotnet/releases/latest -q '.tag_name' |
Latest NuGet package (OpenTelemetry, OpenTelemetry.Extensions.Hosting, exporters, OpenTelemetry.Instrumentation.<pkg>) |
WebFetch https://www.nuget.org/packages/<PackageId> (Version) |
| Latest auto-instrumentation agent | gh api repos/open-telemetry/opentelemetry-dotnet-instrumentation/releases/latest -q '.tag_name' |
| Core CHANGELOG (per package, at the release being audited) | WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet/<core-tag>/src/<Package>/CHANGELOG.md |
| Contrib CHANGELOG (per package, at that package's release tag) | WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet-contrib/<package-tag>/src/<Package>/CHANGELOG.md |
| .NET getting-started / instrumentation docs | WebFetch https://opentelemetry.io/docs/languages/dotnet/instrumentation/ |
| Zero-code agent docs | WebFetch https://opentelemetry.io/docs/zero-code/dotnet/ |
Cross-References
- Schema-level facts:
otel-declarative-configskill (language-agnostic YAML schema sources; .NET not yet implemented). - SDK version selection across languages:
otel-sdk-versionsskill. - Semantic conventions lookup:
otel-semantic-conventionsskill.
Files (opentelemetry-agent-skills)
-
references
-
api.md 11.2 KB
# .NET Instrumentation API — BCL-First Reference .NET instruments telemetry through native Base Class Library (BCL) APIs in `System.Diagnostics` and `Microsoft.Extensions.Logging`. The OpenTelemetry SDK **subscribes** to these APIs at startup; application code does not depend on OTel packages at compile time. ## Sources of Truth | Fact | Fetch | |---|---| | Latest `OpenTelemetry` core / `core-*` tag | `gh api repos/open-telemetry/opentelemetry-dotnet/releases/latest -q '.tag_name'` | | Latest NuGet package versions | `WebFetch https://www.nuget.org/packages/<PackageId>` | | Core CHANGELOG (per package, at the release being audited) | `WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet/<core-tag>/src/<Package>/CHANGELOG.md` | | Instrumentation docs | `WebFetch https://opentelemetry.io/docs/languages/dotnet/instrumentation/` | --- ## Traces — `System.Diagnostics.ActivitySource` / `Activity` `ActivitySource` and `Activity` are the BCL tracing primitives, part of `System.Diagnostics` (no extra package needed). The OTel SDK collects spans from any source registered via `.AddSource(name)`. ### Creating a source ```csharp // Declare once — typically as a static or instance field. // Name must match what you pass to .AddSource() in SDK setup. var activitySource = new ActivitySource("my-service"); ``` ### Starting a span ```csharp // StartActivity returns null if no listener is subscribed (e.g. in test environments). // Use a null-conditional or 'using var' — both patterns are idiomatic. using var span = activitySource.StartActivity("manual-work"); // With explicit kind (default is Internal) using var span = activitySource.StartActivity( "my-operation", ActivityKind.Server); ``` > **Verified spike:** `activitySource.StartActivity("manual-work")` produced the > `manual-work` span in console output with `Activity.Kind: Internal` and correct > `TraceId`/`SpanId`. ### Setting tags (attributes) ```csharp span?.SetTag("http.request.method", "GET"); span?.SetTag("db.system.name", "postgresql"); ``` Tags become span attributes in the OTLP export. For key names, follow semantic conventions — see the `otel-semantic-conventions` skill. ### Setting status ```csharp // Success (implicit when span ends without error) span?.SetStatus(ActivityStatusCode.Ok); // Error span?.SetStatus(ActivityStatusCode.Error, "something went wrong"); ``` ### Adding events ```csharp span?.AddEvent(new ActivityEvent("cache-miss")); // Event with attributes var tags = new ActivityTagsCollection { { "cache.key", "user:42" }, }; span?.AddEvent(new ActivityEvent("cache-miss", DateTimeOffset.UtcNow, tags)); ``` ### Adding links ```csharp // Link to another span by its ActivityContext var link = new ActivityLink(otherActivity.Context); using var span = activitySource.StartActivity( "batch-process", ActivityKind.Internal, parentContext: default, links: new[] { link }); ``` ### `Activity.Current` — ambient context `Activity.Current` is an `AsyncLocal`-backed property. The runtime maintains it automatically across `await` boundaries; you do not need to pass context explicitly in most cases. ```csharp // Read the current active span from anywhere in the call stack var current = Activity.Current; var traceId = current?.TraceId.ToString(); var spanId = current?.SpanId.ToString(); ``` ### SDK subscription The SDK collects spans only from sources you name at startup: ```csharp .WithTracing(t => t .AddSource("my-service") // matches ActivitySource name .AddAspNetCoreInstrumentation() // subscribes to Microsoft.AspNetCore source ... ) ``` Spans from un-subscribed sources are silently dropped — this is intentional, not a bug. --- ## Metrics — `System.Diagnostics.Metrics.Meter` `Meter` and its instrument types are part of `System.Diagnostics.Metrics` (BCL, no extra package). The OTel SDK collects metrics from any meter registered via `.AddMeter(name)`. ### Creating a meter ```csharp // Declare once — name must match what you pass to .AddMeter() in SDK setup. var meter = new Meter("my-service"); ``` ### Counter Monotonically increasing. Use for counts of events, requests, errors. ```csharp var hits = meter.CreateCounter<long>("validation.hits"); // Record with attributes hits.Add(1, new KeyValuePair<string, object?>("route", "/work")); ``` > **Verified spike:** `meter.CreateCounter<long>("validation.hits")` + `.Add(1, ...)` was > confirmed in console output as `Metric Name: validation.hits, Metric Type: LongSum`. ### Histogram Distribution of values. Use for durations, sizes, latencies. ```csharp var duration = meter.CreateHistogram<double>("request.duration", unit: "ms"); duration.Record(123.4, new KeyValuePair<string, object?>("route", "/work")); ``` ### UpDownCounter Non-monotonic sum. Use for queue depths, active connections, in-flight requests. ```csharp var queueDepth = meter.CreateUpDownCounter<long>("queue.depth"); queueDepth.Add(1); // item enqueued queueDepth.Add(-1); // item dequeued ``` ### Observable instruments (asynchronous / pull-based) Observable instruments report their value via a callback, invoked by the SDK when it collects metrics (e.g. on each periodic-reader interval). Use them for values you poll (memory, CPU, connection pool size) rather than values you increment on events. ```csharp // Observable gauge — reports the current value on each collection meter.CreateObservableGauge<long>( "process.memory.bytes", observeValue: () => GC.GetTotalMemory(forceFullCollection: false)); // Observable counter — cumulative monotonic value (e.g. total GC collections) meter.CreateObservableCounter<long>( "gc.collections", observeValue: () => GC.CollectionCount(0)); // Observable up-down counter — non-monotonic cumulative value meter.CreateObservableUpDownCounter<long>( "thread.pool.queue.length", observeValue: () => ThreadPool.PendingWorkItemCount); ``` ### SDK subscription ```csharp .WithMetrics(m => m .AddMeter("my-service") // matches Meter name ... ) ``` Instruments from un-subscribed meters are silently dropped. --- ## Logs — `Microsoft.Extensions.Logging.ILogger` Logging in .NET goes through `ILogger` / `ILogger<T>` from `Microsoft.Extensions.Logging`. The OTel SDK captures log records from the standard logging pipeline — no separate "OTel logger" is needed in application code. ### Injecting and using `ILogger` ```csharp // In a minimal-API handler (injected by the DI container) app.MapGet("/work", (ILogger<Program> logger) => { logger.LogInformation("handled /work"); return Results.Ok(new { ok = true }); }); // In a class (constructor injection) public class OrderService(ILogger<OrderService> logger) { public void Process(Order order) { logger.LogInformation("Processing order {OrderId}", order.Id); } } ``` Structured log parameters (`{OrderId}`) are captured as log attributes. > **Verified spike:** `logger.LogInformation("handled /work")` appeared in console output > with `LogRecord.TraceId` and `LogRecord.SpanId` matching the active `manual-work` span, > confirming automatic trace-log correlation. ### How OTel captures logs The SDK attaches to the `ILoggingBuilder` pipeline: ```csharp // Option A — co-located with other OTel config (OpenTelemetry.Extensions.Hosting) builder.Services.AddOpenTelemetry() .WithLogging(l => l.AddOtlpExporter()); // Option B — via ILoggingBuilder directly builder.Logging.AddOpenTelemetry(o => { o.IncludeScopes = true; o.AddOtlpExporter(); }); ``` Both options work; Option A keeps all OTel wiring co-located. ### Trace-log correlation When a log statement executes inside an active `Activity` (span), the SDK automatically stamps `TraceId`, `SpanId`, and `TraceFlags` onto the log record. No extra code is needed. --- ## Attributes / Tags Both `Activity.SetTag` (traces) and metric instrument `.Add(...)` / `.Record(...)` accept key-value attributes. Use semantic-convention names for interoperability: - Traces: `http.request.method`, `db.system.name`, `server.address`, `error.type`, etc. - Metrics: `http.route`, `rpc.method`, `messaging.system`, etc. For the authoritative attribute registry, consult the `otel-semantic-conventions` skill. **Types supported in BCL instruments:** ```csharp // Tags on Activity span?.SetTag("key", "string-value"); span?.SetTag("key", 42); // int span?.SetTag("key", 3.14); // double span?.SetTag("key", true); // bool // Attributes on metric instruments — pass as KeyValuePair or TagList hits.Add(1, new KeyValuePair<string, object?>("route", "/work")); // TagList (struct, zero-allocation for small sets) var tags = new TagList { { "route", "/work" }, { "status", 200 }, }; hits.Add(1, tags); ``` --- ## Propagation ### How context flows `Activity.Current` is backed by `AsyncLocal<T>`. The runtime propagates it automatically across `await` boundaries, thread-pool callbacks, and `Task` continuations. You do not pass a context object explicitly — the active span is always available via `Activity.Current`. ### Default format: W3C TraceContext The SDK uses W3C TraceContext (`traceparent` / `tracestate` headers) by default. For ASP.NET Core apps with `AddAspNetCoreInstrumentation()`, incoming `traceparent` headers are extracted and the `Activity` is parented correctly without extra code. ### Manual inject/extract For transports not automatically instrumented (custom TCP protocols, message queues, background jobs via a custom dispatcher), use the `Propagators` API: ```csharp using System.Diagnostics; using OpenTelemetry; using OpenTelemetry.Context.Propagation; // --- Injecting (producer / outbound side) --- var propagator = Propagators.DefaultTextMapPropagator; var carrier = new Dictionary<string, string>(); propagator.Inject( new PropagationContext(Activity.Current?.Context ?? default, Baggage.Current), carrier, (dict, key, value) => dict[key] = value); // Serialize 'carrier' alongside your message/request. // --- Extracting (consumer / inbound side) --- var parentContext = propagator.Extract( default, carrier, (dict, key) => dict.TryGetValue(key, out var v) ? new[] { v } : Array.Empty<string>()); using var span = activitySource.StartActivity( "process-message", ActivityKind.Consumer, parentContext.ActivityContext); ``` Once the OpenTelemetry SDK is initialized, `Propagators.DefaultTextMapPropagator` defaults to W3C TraceContext + Baggage. The .NET SDK does not read `OTEL_PROPAGATORS`; replace the default programmatically with `Sdk.SetDefaultTextMapPropagator(...)` when a different propagator is required. --- ## Optional: OpenTelemetry API Shim > **Note:** An `OpenTelemetry.Api` tracing shim exists for developers who prefer > OTel trace terminology (`Tracer`/`TelemetrySpan`) over `ActivitySource`/`Activity`. > It wraps the native BCL tracing types under the hood; metrics still use > `System.Diagnostics.Metrics.Meter` and its instruments directly. > > The BCL path (`System.Diagnostics.ActivitySource`, `System.Diagnostics.Metrics.Meter`) > is the recommended and idiomatic choice for .NET. Prefer it unless you are porting > multi-language instrumentation code or an existing OTel-API-based library. > > See: `WebFetch https://opentelemetry.io/docs/languages/dotnet/instrumentation/` for > the shim's package and usage. -
breaking-changes.md 9.1 KB
# .NET OpenTelemetry Breaking Changes — Audit Workflow Use this reference to audit existing .NET OpenTelemetry code for deprecated or renamed APIs, semconv renames, and experimental feature flags. This is a **workflow document** — fetch the upstream CHANGELOGs rather than relying on any embedded snapshot here. --- ## 1. Understand the Versioning Model First OpenTelemetry .NET ships two kinds of releases: **Core packages** (`OpenTelemetry`, `OpenTelemetry.Api`, `OpenTelemetry.Exporter.*`, `OpenTelemetry.Extensions.Hosting`, etc.) are released together under a single umbrella `core-x.y.z` tag in the `opentelemetry-dotnet` repository. All packages under that tag share the same version number. **Instrumentation and contrib packages** (`OpenTelemetry.Instrumentation.AspNetCore`, `OpenTelemetry.Instrumentation.Http`, `OpenTelemetry.Instrumentation.GrpcNetClient`, etc.) live in the `opentelemetry-dotnet-contrib` repository and follow **their own independent versioning cadence**. A given contrib package may be at a completely different version than the core release released at the same time (for example, `OpenTelemetry.Instrumentation.AspNetCore` may be at `1.15.x` while the core is at `1.16.x`). **Stable vs pre-release packaging:** Core APIs (`OpenTelemetry.Api`) are stable. Instrumentation packages are often released as pre-release (e.g. `1.12.0-beta.1`) until the underlying semantic conventions they implement are finalized. Check the NuGet stability label before pinning a version: ```bash WebFetch https://www.nuget.org/packages/<PackageId> ``` Always fetch the latest `core-*` tag and per-package NuGet version as described in the Sources of Truth table in `SKILL.md` — do not rely on version numbers embedded in any skill file. --- ## 2. Fetch the CHANGELOGs Breaking changes, deprecations, and renames are documented per package in each repository. ### 2a. Core packages (`opentelemetry-dotnet`) ```bash # Discover current core release tag gh api repos/open-telemetry/opentelemetry-dotnet/releases/latest -q '.tag_name' # Fetch CHANGELOG for a specific core package — replace both placeholders WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet/<core-tag>/src/<Package>/CHANGELOG.md ``` Common core package names (match the directory under `src/`): - `OpenTelemetry` - `OpenTelemetry.Api` - `OpenTelemetry.Exporter.OpenTelemetryProtocol` - `OpenTelemetry.Exporter.Console` - `OpenTelemetry.Extensions.Hosting` ### 2b. Contrib / instrumentation packages (`opentelemetry-dotnet-contrib`) Resolve the package version from NuGet first. For the packages below, the release tag is the package ID without the `OpenTelemetry.` prefix, followed by `-<version>` (for example, `Instrumentation.Http-1.18.0`). Confirm that tag exists before fetching it. ```bash # Fetch CHANGELOG for a contrib package at that package's release tag WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet-contrib/<package-tag>/src/<Package>/CHANGELOG.md ``` Common contrib package names (match the directory under `src/`): - `OpenTelemetry.Instrumentation.AspNetCore` - `OpenTelemetry.Instrumentation.Http` - `OpenTelemetry.Instrumentation.GrpcNetClient` - `OpenTelemetry.Instrumentation.SqlClient` - `OpenTelemetry.Instrumentation.Runtime` - `OpenTelemetry.Instrumentation.Process` - `OpenTelemetry.Instrumentation.Cassandra` - `OpenTelemetry.Instrumentation.ElasticsearchClient` - `OpenTelemetry.Instrumentation.Kusto` - `OpenTelemetry.Instrumentation.Quartz` - `OpenTelemetry.Instrumentation.AWS` - `OpenTelemetry.Resources.AWS` - `OpenTelemetry.Resources.Azure` - `OpenTelemetry.Resources.Container` - `OpenTelemetry.Resources.Gcp` - `OpenTelemetry.Resources.Host` - `OpenTelemetry.Resources.OperatingSystem` - `OpenTelemetry.Resources.Process` - `OpenTelemetry.Resources.ProcessRuntime` Each CHANGELOG lists breaking changes, deprecations, and removals in reverse-chronological order. Look for `### Breaking changes`, `### Deprecated`, and `### Removed` sections. --- ## 3. Audit Protocol Run this sequence when upgrading any package: 1. **Identify the package type** (core or contrib) and its repository. 2. **Fetch the CHANGELOG** (Step 2 above) from the exact release tag being upgraded. Do not use `main`, whose `Unreleased` section may describe APIs that are not published. 3. **Scan from the previous pinned version to the target version** — read every release section in that range for breaking changes and deprecations. 4. **Cross-reference semconv renames** — if the CHANGELOG mentions attribute renames, fetch the current semantic convention definition via the `otel-semantic-conventions` skill. 5. **Check experimental flags** (Section 4 below). 6. **Repeat** for every upgraded package — core and contrib have independent changelogs. For a full service upgrade (e.g. upgrading all `OpenTelemetry.*` packages): ```bash # Step 1: pin down the current core tag gh api repos/open-telemetry/opentelemetry-dotnet/releases/latest -q '.tag_name' # Step 2: fetch changelogs for every package in use # Core examples: WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet/<core-tag>/src/OpenTelemetry/CHANGELOG.md WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet/<core-tag>/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/CHANGELOG.md WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet/<core-tag>/src/OpenTelemetry.Extensions.Hosting/CHANGELOG.md # Contrib examples: WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet-contrib/<aspnetcore-tag>/src/OpenTelemetry.Instrumentation.AspNetCore/CHANGELOG.md WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet-contrib/<http-tag>/src/OpenTelemetry.Instrumentation.Http/CHANGELOG.md # Zero-code / CLR-profiler agent (single-repo product — CHANGELOG is at the repo root, not per-package): WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet-instrumentation/<agent-tag>/CHANGELOG.md ``` --- ## 4. Experimental Feature Flags The .NET SDK exposes opt-in experimental behaviors via environment variables prefixed with `OTEL_DOTNET_EXPERIMENTAL_`. These are not stable API surface and may change or be removed without a major version bump. To discover which flags are in effect in a given release, search that release's CHANGELOG and source for `OTEL_DOTNET_EXPERIMENTAL_` after fetching: ```bash WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet/<core-tag>/src/OpenTelemetry/CHANGELOG.md ``` Then search the result for `OTEL_DOTNET_EXPERIMENTAL_` to enumerate flags introduced or changed in that release. Common patterns include flags that enable pre-stable metric views, proto format choices, or alternative propagation behaviors. Similarly, check the contrib CHANGELOG of any instrumentation package you use — instrumentation packages sometimes expose their own experimental flags gated on `OTEL_DOTNET_EXPERIMENTAL_*`. > Treat any `OTEL_DOTNET_EXPERIMENTAL_*` flag as unstable: re-audit it on every upgrade. --- ## 5. Semantic Convention Attribute Renames Instrumentation packages in `opentelemetry-dotnet-contrib` track upstream semconv versions. When semconv renames an attribute (e.g. HTTP, RPC, or database attributes), contrib packages follow, sometimes in pre-release versions first. **Audit workflow for semconv renames:** 1. Fetch the CHANGELOG for the affected instrumentation package (Section 2b). 2. Identify any attribute rename noted in the `### Breaking changes` section. 3. Cross-reference the current definition in the `otel-semantic-conventions` skill to confirm the canonical name and any opt-in migration flags the package may provide during a transition period. 4. Update your custom attribute keys, dashboards, and alert queries to match the new names. Packages will sometimes emit **both** old and new attribute names for one or more releases behind a feature flag (e.g. `OTEL_DOTNET_EXPERIMENTAL_*`) to allow migration without a hard cutover — check the CHANGELOG section for that detail. --- ## 6. Finding Deprecated or Removed APIs Search the fetched CHANGELOG text for: - `[Obsolete]` — C# obsolete annotations that become compile-time warnings - `Deprecated` — noted in prose, may not yet be `[Obsolete]` - `Removed` — already gone; a compile error if still referenced - `renamed` / `moved to` — indicates the replacement symbol For each deprecated or removed API, the CHANGELOG typically names the replacement. If not, fetch the current API docs: ```bash # Per-package CHANGELOG is the authoritative source — fetch it as shown in Section 2. # If you need a general API reference, search the .NET API browser: WebFetch https://learn.microsoft.com/en-us/dotnet/api/?term=OpenTelemetry ``` or the in-repo migration guide linked from the CHANGELOG entry. --- ## Cross-References - Semantic convention definitions: `otel-semantic-conventions` skill. - SDK version selection across languages: `otel-sdk-versions` skill. - SDK setup and exporter configuration: `references/setup.md`. - Instrumentation package catalog: `references/instrumentation-libraries.md`. -
instrumentation-libraries.md 14.5 KB
# .NET Instrumentation Libraries ## Detecting Existing Instrumentation Before adding instrumentation, check whether it is already in place. **In-process packages** — scan `.csproj` files for `OpenTelemetry.Instrumentation.*` references: ```bash grep -r "OpenTelemetry\.Instrumentation\." --include="*.csproj" . ``` **SDK wiring** — search startup code for `AddOpenTelemetry()`: ```bash grep -r "AddOpenTelemetry\(\)" --include="*.cs" . ``` **Zero-code agent** — check whether the CLR Profiler env vars are set in the process environment or the container/service definition: ```bash env | grep -E "CORECLR_ENABLE_PROFILING|CORECLR_PROFILER|OTEL_DOTNET_AUTO_HOME" ``` If the profiler vars are present, the `opentelemetry-dotnet-instrumentation` agent is active and is injecting instrumentation without code changes. --- ## Zero-Code Path: `opentelemetry-dotnet-instrumentation` The [`opentelemetry-dotnet-instrumentation`](https://github.com/open-telemetry/opentelemetry-dotnet-instrumentation) agent instruments .NET applications without source changes by loading a CLR Profiler at startup. **Source of Truth:** [https://opentelemetry.io/docs/zero-code/dotnet/](https://opentelemetry.io/docs/zero-code/dotnet/) Do not pin the agent version in prose; fetch the latest release from the repo or the zero-code docs. ### Install **Linux / macOS:** ```bash curl -sSfL https://github.com/open-telemetry/opentelemetry-dotnet-instrumentation/releases/latest/download/otel-dotnet-auto-install.sh \ | bash ``` The script downloads the agent to `$OTEL_DOTNET_AUTO_HOME` (defaults to `$HOME/.otel-dotnet-auto`). **Windows (PowerShell):** The Windows install uses a PowerShell module flow. For the exact current command, see the [zero-code .NET docs](https://opentelemetry.io/docs/zero-code/dotnet/) — the Source of Truth for Windows installation steps and script filenames. ### Activation (CLR Profiler) The install script generates an `instrument.sh` (Linux/macOS) or sets registry/env values (Windows). Source it before running the process: ```bash . $OTEL_DOTNET_AUTO_HOME/instrument.sh dotnet run ``` The script exports these required vars: | Variable | Purpose | |----------|---------| | `CORECLR_ENABLE_PROFILING` | Must be `1` to activate the profiler | | `CORECLR_PROFILER` | GUID identifying the OTel profiler | | `OTEL_DOTNET_AUTO_HOME` | Path to the agent installation directory | ### Configuration Configure via standard `OTEL_*` env vars: ```bash export OTEL_SERVICE_NAME=my-service export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 export OTEL_TRACES_EXPORTER=otlp export OTEL_METRICS_EXPORTER=otlp export OTEL_LOGS_EXPORTER=otlp ``` For the full list of env vars (including `OTEL_DOTNET_AUTO_*` agent-specific vars), see the [zero-code .NET docs](https://opentelemetry.io/docs/zero-code/dotnet/). --- ## In-Process Library Instrumentation The in-process alternative to the zero-code agent: add NuGet packages and register them in the OTel builder. These run inside the application process and give the most control over which instrumentation is active. The snippet below is the verified baseline from the spike (all three signals confirmed): ```csharp builder.Services.AddOpenTelemetry() .ConfigureResource(r => r.AddService(ServiceName, serviceVersion: "0.1.0")) .WithTracing(t => t .AddSource(ServiceName) .AddAspNetCoreInstrumentation() // OpenTelemetry.Instrumentation.AspNetCore .AddHttpClientInstrumentation() // OpenTelemetry.Instrumentation.Http .AddSqlClientInstrumentation() // OpenTelemetry.Instrumentation.SqlClient .AddOtlpExporter()) .WithMetrics(m => m .AddMeter(ServiceName) .AddAspNetCoreInstrumentation() // also emits HTTP server metrics .AddHttpClientInstrumentation() // also emits HTTP client metrics .AddOtlpExporter()) .WithLogging(l => l .AddOtlpExporter()); ``` See `references/setup.md` for the full `AddOpenTelemetry()` setup pattern. --- ## Contrib Catalog All packages below have a released package tag in [`open-telemetry/opentelemetry-dotnet-contrib`](https://github.com/open-telemetry/opentelemetry-dotnet-contrib). **Version rule:** instrumentation packages follow their own release cadence and are not in lockstep with the core SDK. Do not pin versions in prose — fetch the current version for each package from NuGet: ```text https://www.nuget.org/packages/<PackageId> ``` Or from the per-package CHANGELOG at that package's release tag (source of truth for released breaking changes). For these packages, form the tag by removing `OpenTelemetry.` from the package ID and appending `-<version>`; confirm that tag exists before fetching it. Exception: `OpenTelemetry.Instrumentation.AWSLambda` releases under `Instrumentation.AWS-<version>` tags (its `<MinVerTagPrefix>`). ```text https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet-contrib/<package-tag>/src/<Package>/CHANGELOG.md ``` ### Web / HTTP | Package | Builder extension | NuGet | |---------|-------------------|-------| | `OpenTelemetry.Instrumentation.AspNetCore` | `.AddAspNetCoreInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.AspNetCore) | | `OpenTelemetry.Instrumentation.Http` | `.AddHttpClientInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.Http) | | `OpenTelemetry.Instrumentation.AspNet` | `.AddAspNetInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.AspNet) — .NET Framework only | ### Data | Package | Builder extension | NuGet | |---------|-------------------|-------| | `OpenTelemetry.Instrumentation.SqlClient` | `.AddSqlClientInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.SqlClient) | | `OpenTelemetry.Instrumentation.EntityFrameworkCore` | `.AddEntityFrameworkCoreInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.EntityFrameworkCore) | | `OpenTelemetry.Instrumentation.StackExchangeRedis` | `.AddRedisInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.StackExchangeRedis) | | `OpenTelemetry.Instrumentation.Cassandra` | `.AddCassandraInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.Cassandra) | | `OpenTelemetry.Instrumentation.ElasticsearchClient` | `.AddElasticsearchClientInstrumentation()` for NEST/Elasticsearch.Net; deprecated upstream (no new versions planned) — `Elastic.Clients.Elasticsearch` emits `ActivitySource` telemetry directly, register it with `.AddSource("Elastic.Transport")` (8.10.0+) | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.ElasticsearchClient) | | `OpenTelemetry.Instrumentation.Kusto` | `.AddKustoInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.Kusto) | ### RPC | Package | Builder extension | NuGet | |---------|-------------------|-------| | `OpenTelemetry.Instrumentation.GrpcNetClient` | `.AddGrpcClientInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.GrpcNetClient) | | `OpenTelemetry.Instrumentation.GrpcCore` | `.AddGrpcCoreInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.GrpcCore) | | `OpenTelemetry.Instrumentation.Wcf` | `.AddWcfInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.Wcf) | | `OpenTelemetry.Instrumentation.Remoting` | `.AddRemotingInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.Remoting) | | `OpenTelemetry.Instrumentation.ServiceFabricRemoting` | `.AddServiceFabricRemotingInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.ServiceFabricRemoting) | ### Messaging | Package | Builder extension | NuGet | |---------|-------------------|-------| | `OpenTelemetry.Instrumentation.ConfluentKafka` | `.AddKafkaConsumerInstrumentation<TKey,TValue>()` / `.AddKafkaProducerInstrumentation<TKey,TValue>()` (both available on `TracerProviderBuilder` and `MeterProviderBuilder`) | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.ConfluentKafka) | | `OpenTelemetry.Instrumentation.MassTransit` | `.AddMassTransitInstrumentation()` for MassTransit 7 and earlier; deprecated — MassTransit 8+ emits `ActivitySource` telemetry directly, register it with `.AddSource("MassTransit")` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.MassTransit) | | `OpenTelemetry.Instrumentation.Hangfire` | `.AddHangfireInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.Hangfire) | | `OpenTelemetry.Instrumentation.Quartz` | `.AddQuartzInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.Quartz) | ### Runtime / System | Package | Builder extension | NuGet | |---------|-------------------|-------| | `OpenTelemetry.Instrumentation.Runtime` | `.AddRuntimeInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.Runtime) — emits GC/thread metrics | | `OpenTelemetry.Instrumentation.Process` | `.AddProcessInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.Process) — CPU/memory metrics | | `OpenTelemetry.Instrumentation.EventCounters` | `.AddEventCountersInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.EventCounters) | | `OpenTelemetry.Instrumentation.Owin` | `.AddOwinInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.Owin) — OWIN / .NET Framework | ### Cloud / Infrastructure | Package | Builder extension | NuGet | |---------|-------------------|-------| | `OpenTelemetry.Instrumentation.AWS` | `.AddAWSInstrumentation()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.AWS) | | `OpenTelemetry.Instrumentation.AWSLambda` | `.AddAWSLambdaConfigurations()` for provider setup; `AWSLambdaWrapper.Trace()` / `.TraceAsync()` for handlers | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.AWSLambda) | | `OpenTelemetry.Resources.AWS` | `.AddAWSEBSDetector()` / `.AddAWSEC2Detector()` / `.AddAWSECSDetector()` / `.AddAWSEKSDetector()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Resources.AWS) | | `OpenTelemetry.Resources.Azure` | `.AddAzureAppServiceDetector()` / `.AddAzureVMDetector()` / `.AddAzureContainerAppsDetector()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Resources.Azure) | | `OpenTelemetry.Resources.Container` | `.AddContainerDetector()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Resources.Container) | | `OpenTelemetry.Resources.Gcp` | `.AddGcpDetector()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Resources.Gcp) | | `OpenTelemetry.Resources.Host` | `.AddHostDetector()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Resources.Host) | | `OpenTelemetry.Resources.OperatingSystem` | `.AddOperatingSystemDetector()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Resources.OperatingSystem) | | `OpenTelemetry.Resources.Process` | `.AddProcessDetector()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Resources.Process) | | `OpenTelemetry.Resources.ProcessRuntime` | `.AddProcessRuntimeDetector()` | [NuGet](https://www.nuget.org/packages/OpenTelemetry.Resources.ProcessRuntime) | --- ## Manual Instrumentation Patterns Use these when no contrib package covers the target library. Attributes must follow semantic conventions — load the `otel-semantic-conventions` skill for the authoritative attribute names. These patterns use the native BCL `ActivitySource` API (the primary path for .NET), not the OTel API shim (`tracerProvider.GetTracer(...)`). Declare a module-level source and start activities from it directly. ### HTTP Client Call ```csharp using System.Diagnostics; // Module-level — one instance per library/component static readonly ActivitySource ActivitySource = new("my-service"); // At call site: using var activity = ActivitySource.StartActivity("GET", ActivityKind.Client); activity?.SetTag("http.request.method", "GET"); activity?.SetTag("url.full", requestUrl); var response = await httpClient.GetAsync(requestUrl); activity?.SetTag("http.response.status_code", (int)response.StatusCode); ``` Semconv reference: `otel-semantic-conventions` → HTTP client spans. ### Database Call ```csharp using var activity = ActivitySource.StartActivity("SELECT users", ActivityKind.Client); activity?.SetTag("db.system.name", "postgresql"); activity?.SetTag("db.namespace", databaseName); activity?.SetTag("db.operation.name", "SELECT"); activity?.SetTag("db.collection.name", "users"); // execute query … if (ex != null) { activity?.SetStatus(ActivityStatusCode.Error, ex.Message); activity?.SetTag("error.type", ex.GetType().Name); } ``` Semconv reference: `otel-semantic-conventions` → database spans. ### Background Job ```csharp using var activity = ActivitySource.StartActivity("process batch", ActivityKind.Internal); activity?.SetTag("batch.id", batchId); activity?.SetTag("worker.id", workerId); var items = await repo.GetBatchItems(batchId); activity?.SetTag("batch.size", items.Count); int processed = 0; foreach (var item in items) { try { await ProcessItem(item); processed++; } catch (Exception ex) { logger.LogWarning(ex, "item {ItemId} failed", item.Id); } } activity?.SetTag("batch.processed", processed); activity?.SetTag("batch.failed", items.Count - processed); ``` Semconv reference: `otel-semantic-conventions` → messaging/general spans (use `messaging.system`, `messaging.operation.name`, `messaging.operation.type`, etc. for queue-backed jobs; omit if purely internal). --- ## Enriching Auto-Instrumented Spans When `AddAspNetCoreInstrumentation()` or the zero-code agent has already created a span, add business attributes to it via `Activity.Current` rather than starting a new span: ```csharp app.MapGet("/orders/{id}", async (string id, IOrderService svc) => { // Span already created by AddAspNetCoreInstrumentation() Activity.Current?.SetTag("business.operation", "order_lookup"); Activity.Current?.SetTag("order.id", id); var order = await svc.GetOrder(id); if (order is null) { Activity.Current?.SetStatus(ActivityStatusCode.Error, "order not found"); return Results.NotFound(); } Activity.Current?.SetTag("customer.tier", order.CustomerTier); return Results.Ok(order); }); ``` `Activity.Current` is the ambient span set by the instrumentation library. Calling `SetTag` on it adds attributes to the existing span without creating additional spans. -
performance.md 19.6 KB
# OpenTelemetry .NET Performance Tuning Performance tuning reference for OpenTelemetry .NET SDK. Covers sampling, batch processing, metric readers, Views for cardinality control, exporter configuration, async context, and graceful shutdown. --- ## Performance Impact by Signal | Signal | Unsampled Overhead | Sampled Overhead | Primary Cost | |--------|--------------------|------------------|--------------| | Traces | Low (null or propagation-only Activity) | Moderate | Allocations, export I/O | | Metrics | N/A (always collected) | N/A | Aggregation, cardinality | | Logs | Low (check enabled before building record) | Low-moderate | Serialization, export I/O | --- ## Default Configuration Values Exact numeric defaults can shift between releases. For authoritative values, check the CHANGELOG or source constants: - `BatchExportProcessor<T>` defaults: `src/OpenTelemetry/BatchExportProcessor.cs` - `PeriodicExportingMetricReader` defaults: `src/OpenTelemetry/Metrics/Reader/PeriodicExportingMetricReader.cs` - `OtlpExporterOptions` defaults: `src/OpenTelemetry.Exporter.OpenTelemetryProtocol/OtlpExporterOptions.cs` | Parameter | Environment Variable | |-----------|---------------------| | Batch span queue size | `OTEL_BSP_MAX_QUEUE_SIZE` | | Batch span export size | `OTEL_BSP_MAX_EXPORT_BATCH_SIZE` | | Batch scheduled delay | `OTEL_BSP_SCHEDULE_DELAY` | | Batch exporter timeout | `OTEL_BSP_EXPORT_TIMEOUT` | | Batch log queue size | `OTEL_BLRP_MAX_QUEUE_SIZE` | | Batch log export size | `OTEL_BLRP_MAX_EXPORT_BATCH_SIZE` | | Batch log scheduled delay | `OTEL_BLRP_SCHEDULE_DELAY` | | Batch log exporter timeout | `OTEL_BLRP_EXPORT_TIMEOUT` | | Metric export interval | `OTEL_METRIC_EXPORT_INTERVAL` | | Metric export timeout | `OTEL_METRIC_EXPORT_TIMEOUT` | | OTLP export timeout | `OTEL_EXPORTER_OTLP_TIMEOUT` | | OTLP compression | `OTEL_EXPORTER_OTLP_COMPRESSION` | | OTLP metrics default histogram aggregation | `OTEL_EXPORTER_OTLP_METRICS_DEFAULT_HISTOGRAM_AGGREGATION` | --- ## Sampling Sampling is the single most impactful performance lever for traces. Unsampled activities are not recorded or exported, but the SDK may still create a propagation-only `Activity` for a root or remote-parent span so that trace context continues downstream. A dropped local child may instead make `StartActivity` return `null`. ### Head Sampling Configure a sampler at `TracerProviderBuilder` level: ```csharp using var tracerProvider = Sdk.CreateTracerProviderBuilder() // Sample 10% of new traces; respect parent sampling decision for in-flight traces .SetSampler(new ParentBasedSampler(new TraceIdRatioBasedSampler(0.1))) .AddOtlpExporter() .Build(); ``` `TraceIdRatioBasedSampler` makes sampling decisions based on the trace ID, ensuring consistent sampling across a distributed trace when applied without a `ParentBasedSampler` wrapper. `ParentBasedSampler` defers to the upstream decision when a parent context is present and applies the root sampler otherwise. ### Via Environment Variable ```bash # "traceidratio" with ratio, "always_on", "always_off", "parentbased_traceidratio", etc. OTEL_TRACES_SAMPLER=parentbased_traceidratio OTEL_TRACES_SAMPLER_ARG=0.1 ``` See the [OpenTelemetry spec sampler names](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#general-sdk-configuration) for valid values. ### Sampling Decision Impact ```text AlwaysOnSampler -> Full Activity lifecycle: allocation + recording + export ParentBasedSampler(new TraceIdRatioBasedSampler(0.1)) -> about 90% non-recording, 10% fully recorded TraceIdRatioBasedSampler(0.1) -> Consistent 10% sampling ignoring parent AlwaysOffSampler -> No recording/export; context may still use propagation-only Activities ``` Core 1.19.0 added `AlwaysRecordSampler(rootSampler)`, which turns the wrapped sampler's `Drop` decisions into `RecordOnly`: every span is created and passed to processors, but only sampled spans are exported. This removes the sampling saving on in-process cost, so use it only when a processor must see all spans. Core 1.17.0 also emits a verbose `OpenTelemetry-Sdk` self-diagnostics event when `ParentBasedSampler` drops an activity because its local parent is not recorded. > **Tail sampling**: For decisions based on complete traces (error rate, latency thresholds), use the OpenTelemetry Collector's tail sampling processor. Combine SDK head sampling with Collector tail sampling for a common production pattern. --- ## Batch Processor Tuning `BatchActivityExportProcessor` buffers completed `Activity` objects in a queue and exports them asynchronously in batches. It wraps `BatchExportProcessor<Activity>`. ### How It Works ```text Application thread BatchActivityExportProcessor thread | | activity.Stop() --enqueue--> queue (MaxQueueSize) | | | timer fires (ScheduledDelayMilliseconds) | OR batch full (MaxExportBatchSize) | | | --export batch--> Exporter ``` ### Configuration via OtlpExporterOptions When using `AddOtlpExporter()`, the batch options are accessible through `OtlpExporterOptions.BatchExportProcessorOptions`: ```csharp using var tracerProvider = Sdk.CreateTracerProviderBuilder() .AddSource("MyApp") .AddOtlpExporter(options => { options.ExportProcessorType = ExportProcessorType.Batch; options.BatchExportProcessorOptions.MaxQueueSize = 4096; options.BatchExportProcessorOptions.MaxExportBatchSize = 1024; options.BatchExportProcessorOptions.ScheduledDelayMilliseconds = 10000; }) .Build(); ``` Check `BatchExportActivityProcessorOptions` in source for current defaults; they are also configurable via the `OTEL_BSP_*` environment variables. ### Tuning for Throughput For high-volume services (many spans per second), increase queue and batch sizes to absorb bursts and reduce the number of export calls per unit time. ### Tuning for Latency For services where trace delivery speed matters (live debugging, alerting), decrease `ScheduledDelayMilliseconds` and `MaxExportBatchSize` to export smaller batches more frequently. ### Queue Full Behavior When the queue fills, new spans are dropped instead of blocking the application. The processor tracks the drop count and emits an `OpenTelemetry-Sdk` `EventSource` summary when it shuts down. ### SimpleActivityExportProcessor `SimpleActivityExportProcessor` exports activities synchronously on `activity.Stop()`. It adds latency to every activity ending. Use it for: - Tests and local development (deterministic behavior) - Short-lived CLI tools that must flush before exit ```csharp // Direct use (low-level) var processor = new SimpleActivityExportProcessor(exporter); ``` When using `AddOtlpExporter()`, set `ExportProcessorType = ExportProcessorType.Simple`. --- ## Metric Reader Tuning ### PeriodicExportingMetricReader `PeriodicExportingMetricReader` drives metric collection and export on a configurable interval. The metrics console exporter, for instance, flushes on this cycle. ```csharp using var meterProvider = Sdk.CreateMeterProviderBuilder() .AddMeter("MyApp") .AddReader(new PeriodicExportingMetricReader( exporter: new OtlpMetricExporter(new OtlpExporterOptions()), exportIntervalMilliseconds: 30000, // More frequent than the default exportTimeoutMilliseconds: 15000)) .Build(); ``` Check `PeriodicExportingMetricReader.cs` for the current defaults (`DefaultExportIntervalMilliseconds`, `DefaultExportTimeoutMilliseconds`). Interval tradeoffs: - Default: standard for dashboards and alerting - Shorter intervals: near-real-time visibility, higher export overhead - Longer intervals: reduced overhead, coarser freshness --- ## Views for Cardinality Control Views filter or transform metric streams before aggregation, reducing the number of unique time series stored and exported. They are the primary lever for cardinality control in .NET. ### Drop High-Cardinality Attributes Use `TagKeys` on `MetricStreamConfiguration` to allowlist the attributes to retain. Attributes not in the list are dropped before aggregation: ```csharp using var meterProvider = Sdk.CreateMeterProviderBuilder() .AddMeter("MyApp") .AddView( instrumentName: "http.server.request.duration", metricStreamConfiguration: new MetricStreamConfiguration { // Keep only these attributes; drop everything else (e.g. user.id) TagKeys = new[] { "http.request.method", "http.response.status_code", "url.scheme" }, }) .AddOtlpExporter() .Build(); ``` ### Exclude Specific Attributes Core 1.17.0 added `ExcludedTagKeys` for the inverse case: preserve all attributes except a small denylist. ```csharp .AddView( instrumentName: "http.server.request.duration", metricStreamConfiguration: new MetricStreamConfiguration { ExcludedTagKeys = new[] { "user.id", "request.id" }, }) ``` `TagKeys` and `ExcludedTagKeys` are mutually exclusive. Setting both causes the view to be ignored. ### Drop a Metric Entirely ```csharp .AddView( instrumentName: "runtime.dotnet.gc.collections", metricStreamConfiguration: MetricStreamConfiguration.Drop) ``` ### Wildcard / Predicate-Based Views ```csharp .AddView(viewConfig: instrument => { // Drop all metrics whose name starts with "runtime." if (instrument.Name.StartsWith("runtime.", StringComparison.OrdinalIgnoreCase)) { return MetricStreamConfiguration.Drop; } return null; // null = use default configuration }) ``` ### Per-Instrument Cardinality Limit ```csharp .AddView( instrumentName: "http.server.request.duration", metricStreamConfiguration: new MetricStreamConfiguration { CardinalityLimit = 500, // Max unique tag combinations for this metric }) ``` Metric attribute cardinality is the product of unique values across all recorded attributes. Attributes like user IDs or request IDs are unbounded. Attributes like HTTP method or status code are bounded. Always apply `TagKeys` allowlists to instruments that record unbounded attributes. --- ## Exporter Configuration ### Protocol Choice | Aspect | gRPC (`OtlpExportProtocol.Grpc`) | HTTP/Protobuf (`OtlpExportProtocol.HttpProtobuf`) | |--------|----------------------------------|---------------------------------------------------| | Default port | 4317 | 4318 | | .NET target | .NET only (deprecated for .NET Framework / .NET Standard) | All targets | | Connection | Single long-lived connection | HTTP/1.1 or HTTP/2 | | Best for | Persistent connections, high throughput | Firewalls, load balancers, .NET Framework | > **Note**: `OtlpExportProtocol.Grpc` is marked `[Obsolete]` for .NET Framework / .NET Standard targets. For cross-target libraries, prefer `HttpProtobuf`. The default protocol on .NET is `Grpc`; on .NET Framework / .NET Standard it is `HttpProtobuf`. See `OtlpExporterOptions.cs` source for the compile-time conditional. ```csharp .AddOtlpExporter(options => { options.Protocol = OtlpExportProtocol.HttpProtobuf; options.Endpoint = new Uri("http://collector:4318"); }) ``` ### Timeout The OTLP exporter's `TimeoutMilliseconds` property controls how long an individual export attempt waits before being cancelled. Route the default value to `OtlpExporterOptions.cs` source rather than asserting it here; it is controlled by `OTEL_EXPORTER_OTLP_TIMEOUT` (milliseconds). ```csharp .AddOtlpExporter(options => { options.TimeoutMilliseconds = 5000; // Override the default }) ``` ### Request and Response Size Limits Core 1.18.0 added `MaxRequestSizeBytes` and `MaxResponseSizeBytes` to `OtlpExporterOptions`. The request limit defaults to 64 MiB (reduced from 128 MiB in a breaking behavior change), and the response limit defaults to 4 MiB. An oversized request batch is dropped in its entirety; an oversized response is discarded and treated as a non-retryable failure. ```csharp .AddOtlpExporter(options => { options.MaxRequestSizeBytes = 128 * 1024 * 1024; options.MaxResponseSizeBytes = 8 * 1024 * 1024; }) ``` The maximum supported request limit is 256 MiB. Size is measured before compression, and the receiver may enforce its own request-size limit. ### Compression The OTLP exporter supports `none` and `gzip` compression. Configure it with `OtlpExporterOptions.Compression` or `OTEL_EXPORTER_OTLP_COMPRESSION=gzip`; when using `UseOtlpExporter`, signal-specific compression env vars such as `OTEL_EXPORTER_OTLP_TRACES_COMPRESSION` are also supported. ### Retry The OTLP exporter supports opt-in retry behavior via the experimental feature flag `OTEL_DOTNET_EXPERIMENTAL_OTLP_RETRY`. Valid values: - `in_memory`: retry failed exports in memory - `disk`: persist failed payloads to disk and retry later (requires `OTEL_DOTNET_EXPERIMENTAL_OTLP_DISK_RETRY_DIRECTORY_PATH`) Retry is not enabled by default. For production resilience, consider placing an OpenTelemetry Collector on localhost and delegating retry/buffering to it. ### Exporter Processor Type The OTLP exporter defaults to `ExportProcessorType.Batch`. Switch to `Simple` only for development or CLI tools: ```csharp .AddOtlpExporter(options => { options.ExportProcessorType = ExportProcessorType.Simple; // Development only }) ``` --- ## Async Context and Activity Propagation ### Automatic Flow via AsyncLocal `Activity.Current` is backed by `AsyncLocal<Activity>`, part of the .NET runtime's `ActivitySource` implementation. This means the current span flows automatically across `await` points within a single logical async chain: ```csharp // Requires an ActivityListener, or an OTel provider registered with AddSource("MyApp"). // Activity.Current flows automatically into all awaited calls when the activity is created. using var activity = activitySource.StartActivity("outer"); await DoSomethingAsync(); // Activity.Current == activity inside here await DoSomethingElseAsync(); // Still activity ``` No manual propagation is needed for standard `async`/`await` code. ### Non-Awaited Boundaries Require Manual Propagation `Activity.Current` normally flows into `Task.Run` through `ExecutionContext`, but it may not flow when context propagation is suppressed, when work is scheduled independently, or when callbacks run after the originating scope has exited: ```csharp using var activity = activitySource.StartActivity("producer"); var parentContext = activity?.Context ?? default; // Capture before forking // Fire-and-forget: capture the parent explicitly so the work item is deterministic. _ = Task.Run(async () => { // Restore or propagate the captured parent context explicitly. using var childActivity = activitySource.StartActivity( "consumer", ActivityKind.Consumer, parentContext); await ProcessAsync(); }); ``` For background workers, message consumers, and queued work items, extract the propagation context at the producer boundary and inject it into the work item payload (e.g., as W3C Trace Context headers), then restore it at the consumer boundary. ### Context Propagation Across Service Boundaries Always extract context at ingress and inject at egress: ```csharp // At ingress (e.g., ASP.NET Core middleware handles this automatically) var propagator = Propagators.DefaultTextMapPropagator; var context = propagator.Extract(default, request.Headers, (headers, name) => headers.TryGetValue(name, out var value) ? new[] { value.ToString() } : []); // At egress propagator.Inject(new PropagationContext(Activity.Current?.Context ?? default, Baggage.Current), request.Headers, (headers, name, value) => headers[name] = value); ``` --- ## Graceful Shutdown Proper shutdown ensures buffered telemetry is flushed before the process exits. ### With .NET Generic Host (Recommended) When using `services.AddOpenTelemetry()`, the SDK registers a hosted service (`TelemetryHostedService`) that initializes providers on startup. Providers are registered as singletons and disposed by the DI container on host shutdown — no manual shutdown code is required: ```csharp builder.Services.AddOpenTelemetry() .WithTracing(b => b.AddSource("MyApp").AddOtlpExporter()) .WithMetrics(b => b.AddMeter("MyApp").AddOtlpExporter()); // On host.StopAsync() or Ctrl+C, providers are disposed automatically var app = builder.Build(); await app.RunAsync(); ``` ### Without Generic Host For SDK 1.10.0 and newer, prefer `OpenTelemetrySdk.Create(...)` when a non-hosted process needs one lifecycle boundary for multiple signals: ```csharp using var sdk = OpenTelemetrySdk.Create(builder => builder .WithTracing(b => b.AddSource("MyApp").AddOtlpExporter()) .WithMetrics(b => b.AddMeter("MyApp").AddOtlpExporter())); // Dispose flushes and shuts down all configured providers. ``` For separate provider lifetimes, dispose each provider explicitly. `ForceFlush` is an optional pre-exit checkpoint to drain buffered telemetry before disposal; it is not a shutdown substitute. ```csharp using var tracerProvider = Sdk.CreateTracerProviderBuilder() .AddSource("MyApp") .AddOtlpExporter() .Build(); using var meterProvider = Sdk.CreateMeterProviderBuilder() .AddMeter("MyApp") .AddOtlpExporter() .Build(); // ... application runs ... // On exit: Dispose flushes buffered data and shuts down exporters. // The `using` declarations above handle this automatically at scope end. ``` ### ForceFlush `ForceFlush` synchronously flushes all buffered telemetry without shutting down the provider. Use it when you need to guarantee delivery before a checkpoint (e.g., before a blue/green swap, before a long sleep): ```csharp // Traces bool flushed = tracerProvider.ForceFlush(timeoutMilliseconds: 5000); // Metrics bool flushed = meterProvider.ForceFlush(timeoutMilliseconds: 5000); ``` `ForceFlush` returns `true` if all data was flushed within the timeout, `false` if the timeout was exceeded or a processor failed. Both `TracerProvider` and `MeterProvider` expose `ForceFlush` as extension methods in `OpenTelemetry`. > **Hot path warning**: Calling `ForceFlush` on the request hot path synchronously exports all buffered data, adding latency to that request. Reserve it for shutdown paths or administrative endpoints. --- ## Pipeline Reliability The OpenTelemetry .NET SDK is designed so that telemetry failures do not crash or block the application: - **Dropped activities avoid recording/export work** — `StartActivity` may return `null` or a propagation-only `Activity`, depending on the parent context - **Metric recording never throws** — measurements are silently dropped on failure - **Export failures are logged internally** — via `EventSource`; check ETW/dotnet-trace for diagnostics - **Queue overflow drops spans** — the application is not blocked Export retries are opt-in (see the Retry section above). For reliable delivery in production, route telemetry through a local OpenTelemetry Collector that handles buffering and retry on your behalf. --- ## Sources of Truth Inspect these paths at the exact `core-*` release tag being audited, not at `main`: - Batch processor defaults: `src/OpenTelemetry/BatchExportProcessor.cs` and `src/OpenTelemetry/Trace/Processor/BatchExportActivityProcessorOptions.cs` - Metric reader defaults: `src/OpenTelemetry/Metrics/Reader/PeriodicExportingMetricReader.cs` - OTLP exporter options and protocol defaults: `src/OpenTelemetry.Exporter.OpenTelemetryProtocol/OtlpExporterOptions.cs` - OTLP retry (experimental): `src/OpenTelemetry.Exporter.OpenTelemetryProtocol/Implementation/ExperimentalOptions.cs` - Hosting lifecycle: `src/OpenTelemetry.Extensions.Hosting/Implementation/TelemetryHostedService.cs` - Environment variable names: `OTEL_BSP_*`, `OTEL_METRIC_EXPORT_*`, `OTEL_EXPORTER_OTLP_*` -
setup.md 8.5 KB
# .NET SDK Setup — DI/Builder Path The idiomatic .NET setup is the DI/builder path: call `AddOpenTelemetry()` on the service collection, then chain `.ConfigureResource(...)`, `.WithTracing(...)`, `.WithMetrics(...)`, and `.WithLogging(...)` via `OpenTelemetry.Extensions.Hosting`. For non-hosted multi-signal processes on SDK 1.10.0 or newer, use `OpenTelemetrySdk.Create(...)`. ## Sources of Truth | Fact | Fetch | |---|---| | Latest `OpenTelemetry` core / `core-*` tag | `gh api repos/open-telemetry/opentelemetry-dotnet/releases/latest -q '.tag_name'` | | Latest NuGet package (`OpenTelemetry`, `OpenTelemetry.Extensions.Hosting`, exporters) | `WebFetch https://www.nuget.org/packages/<PackageId>` (Version) | | Core CHANGELOG (per package, at the release being audited) | `WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet/<core-tag>/src/<Package>/CHANGELOG.md` | | .NET getting-started / instrumentation docs | `WebFetch https://opentelemetry.io/docs/languages/dotnet/instrumentation/` | ## Hosted Setup For ASP.NET Core or any service using the .NET generic host, install `OpenTelemetry.Extensions.Hosting` plus the exporter package you use (for example `OpenTelemetry.Exporter.OpenTelemetryProtocol` for `AddOtlpExporter()`), and call `AddOpenTelemetry()` on `builder.Services`. The snippet below is the verified baseline from the spike (all three signals produced working output): ```csharp using System.Diagnostics; using System.Diagnostics.Metrics; using OpenTelemetry.Logs; using OpenTelemetry.Metrics; using OpenTelemetry.Resources; using OpenTelemetry.Trace; var builder = WebApplication.CreateBuilder(args); const string ServiceName = "my-service"; var activitySource = new ActivitySource(ServiceName); var meter = new Meter(ServiceName); builder.Services.AddOpenTelemetry() .ConfigureResource(r => r.AddService(ServiceName, serviceVersion: "0.1.0")) .WithTracing(t => t .AddSource(ServiceName) .AddAspNetCoreInstrumentation() .AddOtlpExporter()) .WithMetrics(m => m .AddMeter(ServiceName) .AddOtlpExporter()) .WithLogging(l => l .AddOtlpExporter()); var app = builder.Build(); // ... routes, app.Run() ``` Replace `AddOtlpExporter()` with `AddConsoleExporter()` during local development (requires `OpenTelemetry.Exporter.Console`). The console exporter was used in the spike to confirm all three signals. **OTLP shortcut:** If the process exports all enabled signals through the same OTLP destination, `OpenTelemetry.Exporter.OpenTelemetryProtocol` also exposes `.UseOtlpExporter()` on the `AddOpenTelemetry()` builder. It automatically enables the OTLP exporter for logs, metrics, and traces, supports signal-specific `OTEL_EXPORTER_OTLP_*` overrides, and must not be mixed with signal-specific `.AddOtlpExporter()` calls. **Logging alternative — `ILoggingBuilder`:** Instead of `.WithLogging(...)` on the OTel builder, you can wire logging through the standard host logging builder: ```csharp builder.Logging.AddOpenTelemetry(o => { o.IncludeScopes = true; o.AddOtlpExporter(); }); ``` Both approaches work; `WithLogging(...)` keeps all OTel config co-located. **`IHostApplicationBuilder` overload (core 1.19.0+):** `builder.AddOpenTelemetry()` (instead of `builder.Services.AddOpenTelemetry()`) returns the same `OpenTelemetryBuilder`, and also seeds `service.name` from `IHostEnvironment.ApplicationName` and `deployment.environment.name` from `IHostEnvironment.EnvironmentName` as low-priority resource defaults. `OTEL_SERVICE_NAME`, `OTEL_RESOURCE_ATTRIBUTES`, or an explicit `ConfigureResource` call override them. See the `OpenTelemetry.Extensions.Hosting` README at the core tag for details. **Resource:** `ConfigureResource` accepts any `ResourceBuilder` callback. The resource is shared across all three signals, which is why `service.name`, `service.version`, and `telemetry.sdk.*` appear on every export. Core 1.17.0 added resource schema URLs. Use `ResourceBuilder.AddAttributes(attributes, schemaUrl)` when the supplied attributes follow a specific schema; the OTLP exporter carries `Resource.SchemaUrl` on traces, metrics, and logs. Merging resources with different non-empty schema URLs clears the resulting schema URL and emits a self-diagnostics warning. Core 1.18.0 added `ITelemetryHostInitializer` for DI-based applications that do not run hosted services, such as Blazor. Resolve it from the built service provider and initialize telemetry during application startup: ```csharp app.Services.GetRequiredService<ITelemetryHostInitializer>().Initialize(); ``` Regular generic-host applications do not need this call: the registered `TelemetryHostedService` initializes the providers when the host starts. ## Non-Hosted Setup For console apps or worker processes that do not use the generic host, SDK 1.10.0 and newer support `OpenTelemetrySdk.Create(...)` as the single lifecycle boundary for all enabled signals: ```csharp using OpenTelemetry; using OpenTelemetry.Logs; using OpenTelemetry.Metrics; using OpenTelemetry.Resources; using OpenTelemetry.Trace; using var sdk = OpenTelemetrySdk.Create(b => b .ConfigureResource(r => r.AddService("my-console-app")) .WithTracing(t => t .AddSource("my-source") .AddOtlpExporter()) .WithMetrics(m => m .AddMeter("my-meter") .AddOtlpExporter()) .WithLogging(l => l .AddOtlpExporter())); ``` Disposing the returned `OpenTelemetrySdk` flushes and shuts down all configured signals. For older SDKs or when you need separate provider lifetimes, build and own the providers manually: ```csharp using OpenTelemetry; using OpenTelemetry.Trace; using OpenTelemetry.Metrics; // Tracing using var tracerProvider = Sdk.CreateTracerProviderBuilder() .AddSource("my-source") .AddOtlpExporter() .Build(); // Metrics using var meterProvider = Sdk.CreateMeterProviderBuilder() .AddMeter("my-meter") .AddOtlpExporter() .Build(); // Logging (host-free): use LoggerFactory.Create(b => b.AddOpenTelemetry(...)) // or OpenTelemetryLoggerProvider directly. The ILoggingBuilder route shown in // the Hosted section above requires the generic host and does not apply here. ``` `using var` ensures `Dispose()` is called on exit, which flushes the batch processors and shuts down the providers. For explicit control, call `tracerProvider.Shutdown()` / `meterProvider.Shutdown()` before the process exits. ## Configuration Inputs The .NET SDK reads configuration from two sources; they can be combined: **`OTEL_*` environment variables** — the standard cross-language signal. Examples: ```sh export OTEL_SERVICE_NAME=my-service export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 export OTEL_TRACES_SAMPLER=parentbased_traceidratio export OTEL_TRACES_SAMPLER_ARG=0.1 ``` Env vars are read automatically by the SDK; no extra code is needed once `AddOpenTelemetry()` is called. **`IConfiguration` / `appsettings.json`** — the .NET-native config system. Pass configuration values into the builder lambdas as you would any other config: ```json // appsettings.json { "OTel": { "ServiceName": "my-service", "Endpoint": "http://localhost:4317" } } ``` ```csharp var cfg = builder.Configuration; builder.Services.AddOpenTelemetry() .ConfigureResource(r => r.AddService(cfg["OTel:ServiceName"]!)) .WithTracing(t => t .AddOtlpExporter(o => o.Endpoint = new Uri(cfg["OTel:Endpoint"]!))); ``` `IConfiguration` is the standard .NET way to surface config from JSON files, env vars, Azure Key Vault, and other providers — it is not the OTel declarative YAML format (see below). ## No Released Declarative Config in .NET > **Note:** .NET does not have a released implementation of the OpenTelemetry declarative > YAML (`file_format`) specification in the SDK. Verified 2026-10-15: `OpenTelemetry.Configuration.Declarative` > exists on `main`, but no NuGet package is published. The separate zero-code auto-instrumentation > (`opentelemetry-dotnet-instrumentation` v1.17.0) has experimental file-based config behind > `OTEL_EXPERIMENTAL_FILE_BASED_CONFIGURATION_ENABLED=true`; see the `otel-declarative-config` skill. > > Upstream tracking issue: > [open-telemetry/opentelemetry-dotnet#7658](https://github.com/open-telemetry/opentelemetry-dotnet/issues/7658) > > To check current status, fetch the issue or the latest core CHANGELOG (see Sources of > Truth above). For the language-agnostic YAML schema, load the `otel-declarative-config` > skill. Use `OTEL_*` env vars and `IConfiguration` (described above) for runtime configuration until declarative config is released.
-
-
SKILL.md 3.9 KB
--- name: otel-dotnet description: OpenTelemetry in .NET — DI/builder SDK setup (OpenTelemetry.Extensions.Hosting, AddOpenTelemetry, UseOtlpExporter, OpenTelemetrySdk.Create), native .NET instrumentation APIs (ActivitySource, System.Diagnostics.Metrics Meter) plus logging abstractions (ILogger), zero-code CLR-profiler agent, contrib instrumentation packages, performance tuning, and breaking-change audits. Use when adding, reviewing, or configuring OpenTelemetry in a .NET or ASP.NET Core service. Triggers on "setup otel in dotnet", "dotnet telemetry", ".net tracing", "ASP.NET Core opentelemetry", "AddOpenTelemetry", "UseOtlpExporter", "OpenTelemetrySdk.Create", "ActivitySource", "System.Diagnostics.Metrics Meter", "ILogger opentelemetry", "OpenTelemetry.Extensions.Hosting", "opentelemetry-dotnet-instrumentation", or any C# OTel question. --- # OpenTelemetry in .NET Entry point for OpenTelemetry mechanics in .NET services. Load a reference below based on the task; each reference is self-contained. ## References | File | Use when | |---|---| | [`references/setup.md`](references/setup.md) | Setting up the SDK via the DI/builder path (`AddOpenTelemetry().WithTracing/WithMetrics/WithLogging`, `UseOtlpExporter`, `OpenTelemetrySdk.Create`, `OpenTelemetry.Extensions.Hosting`), exporter wiring, env-var / `IConfiguration` inputs, and why there is no released declarative YAML config in .NET. | | [`references/api.md`](references/api.md) | Instrumenting with native BCL APIs (`ActivitySource`/`Activity`, `System.Diagnostics.Metrics.Meter`, `ILogger`), how the SDK subscribes (`AddSource`/`AddMeter`), attributes, propagation, and the optional OTel API shim. | | [`references/instrumentation-libraries.md`](references/instrumentation-libraries.md) | Zero-code (the `opentelemetry-dotnet-instrumentation` CLR-profiler agent), the contrib instrumentation-package catalog, and manual instrumentation following semconv. | | [`references/performance.md`](references/performance.md) | Tuning sampling (built-in samplers incl. `AlwaysRecordSampler`), batch export processor, periodic metric reader, views, exporter choice, async context, graceful shutdown. | | [`references/breaking-changes.md`](references/breaking-changes.md) | Auditing existing code for deprecated/renamed APIs and semconv changes across recent core/contrib releases. | ## Sources of Truth For YAML schema details (language-agnostic), see the `otel-declarative-config` skill — but note .NET does not yet have a released declarative-config implementation (see `references/setup.md`). For .NET-specific facts: | Fact | Fetch | |---|---| | Latest `OpenTelemetry` core / `core-*` tag | `gh api repos/open-telemetry/opentelemetry-dotnet/releases/latest -q '.tag_name'` | | Latest NuGet package (`OpenTelemetry`, `OpenTelemetry.Extensions.Hosting`, exporters, `OpenTelemetry.Instrumentation.<pkg>`) | `WebFetch https://www.nuget.org/packages/<PackageId>` (Version) | | Latest auto-instrumentation agent | `gh api repos/open-telemetry/opentelemetry-dotnet-instrumentation/releases/latest -q '.tag_name'` | | Core CHANGELOG (per package, at the release being audited) | `WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet/<core-tag>/src/<Package>/CHANGELOG.md` | | Contrib CHANGELOG (per package, at that package's release tag) | `WebFetch https://raw.githubusercontent.com/open-telemetry/opentelemetry-dotnet-contrib/<package-tag>/src/<Package>/CHANGELOG.md` | | .NET getting-started / instrumentation docs | `WebFetch https://opentelemetry.io/docs/languages/dotnet/instrumentation/` | | Zero-code agent docs | `WebFetch https://opentelemetry.io/docs/zero-code/dotnet/` | ## Cross-References - Schema-level facts: `otel-declarative-config` skill (language-agnostic YAML schema sources; .NET not yet implemented). - SDK version selection across languages: `otel-sdk-versions` skill. - Semantic conventions lookup: `otel-semantic-conventions` skill.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.