# Decorator placement

Tier 3 reference for the `instrument-agent` skill. Single concern: which decorator goes where, and how to propose placements safely.

## 1. CRITICAL — never apply edits without user approval

> **CRITICAL — Never modify the customer's source files without explicit per-file user approval.** The skill **proposes** decorator diffs; the customer accepts or rejects them. Apply each diff one file at a time, wait for `yes` per file. This rule mirrors `SKILL.md` and applies to every decorator placement.

**IMPORTANT:** "Apply all" is not a substitute for per-file confirmation. If the customer says "looks good, apply all of them," confirm explicitly: *"I'll apply these N diffs now — confirm?"* Wait for `yes` before doing anything.

## 2. The two decorators in scope

Only two decorators are in scope for v1. Anything else is out of scope (see section 3).

### `@mc.trace_with_workflow(span_name, workflow_name)` — orchestration / entry functions

A function counts as **orchestration** when:

- It coordinates a sequence of operations across one or more LLM calls, tool calls, or task functions.
- It's the entry point of a logical agent flow (e.g., a LangGraph graph node or API handler that calls downstream functions).
- It's a router or controller function that decides which downstream functions to call.

Every instrumented agent should have a workflow decorator on the top-level entry or enclosing function, even when the flow contains only one LLM-calling task.

Examples:

- A `chat_agent(message)` function that coordinates retrieval + LLM call + tool dispatch.
- A `run_agent(message)` function that validates input, calls one LLM task function, and returns the response.
- A LangGraph node function that runs `should_continue → call_model → validate`.
- A planner function that loops over an LLM until success.

### `@mc.trace_with_task(span_name, task_name)` — LLM-calling task functions

A function counts as a **task** when:

- It makes an LLM API call (directly or via the AI library: `ChatOpenAI(...).invoke(...)`, `Anthropic().messages.create(...)`, `bedrock.invoke_model(...)`, etc.).
- It performs a discrete unit of work that's interesting to evaluate (e.g., regex generation, summary, classification).

Examples:

- A `summarize(text)` function that calls an LLM with a summarization prompt.
- A `extract_entities(doc)` function that calls an LLM and parses structured output.
- A `book_flight()` function that calls an LLM to choose flight options.

### Why workflow vs task matters

Tasks are nested within workflows. Both labels propagate down the trace tree, so spans automatically inherit the workflow attribute when called from a workflow-decorated function.

Workflow + task are used at **evaluation time** to filter and differentiate parts of the agent — e.g., *"show me all `chat` task spans inside the `customer-support` workflow."* Without these labels, the trace tree is just a span hierarchy with no semantic meaning to MC's evaluation pipeline.

## 3. Only the two decorators above

> **CRITICAL — `@trace_with_workflow` and `@trace_with_task` are the only decorators this skill proposes.** Tasks-nested-in-workflows is the entire decorator surface for v1; that pair already provides the filtering and propagation surface MC's evaluation pipeline needs.

If the customer asks about other SDK tracing primitives, redirect them to the live SDK docs on PyPI / GitHub for manual usage — but the skill does not scaffold them.

## 4. Placement guidance

Walk the customer's source files (after they've approved which files to inspect — the skill is read-only on source until a diff is approved). For each candidate function:

1. **Identify the entry point.** What's the top-level function the user calls to run the agent? That's the workflow candidate and should always be proposed.
2. **Identify the LLM call sites.** Functions that call `OpenAI().chat.completions.create(...)`, `Anthropic().messages.create(...)`, `LangchainInstrumentor`-instrumented chains, etc. Those are task candidates.
3. **Identify intermediate orchestration nodes.** Multi-step functions between the entry and the LLM call sites are additional workflow candidates only when they represent a distinct logical agent flow.
4. **Match `workflow_name` to a meaningful concept.** Use the customer's domain language: `"customer-support"`, `"travel-planner"`, `"regex-bootstrap"`, etc. Not technical names like `"main"` or `"agent"`.
5. **Match `task_name` to the LLM call's purpose.** `"summarize"`, `"classify"`, `"plan-flight"`, `"create_regex"`. Not `"call_llm"`.

**IMPORTANT — Always propose both decorator types.** Aim for 1 workflow on the agent entry / enclosing function + 1 task per LLM call. Decorating every helper function adds span noise without semantic value, but producing only task spans or only workflow spans leaves the trace without the required workflow/task pairing.

## 5. The canonical placement example

A typical task placement looks like:

```python
@mc.trace_with_task(
    span_name="call_first_model",
    task_name="create_regex",
)
def call_first_model(state, config, logger: Logger):
    # ... function body that invokes the LLM
```

Notice:

- Task name should describe the call's purpose. The function name is fine when it's meaningful and distinct (e.g. `summarize`, `extract_entities`); generic names like `call_llm` are too opaque.
- `span_name` is fine to use the function name for.

The orchestration function that drives the graph (a few levels up the call stack) gets a workflow-level decorator instead. Use this pattern when proposing diffs.

## 6. Conservative defaults — both decorators, few placements

For the first pass, propose decorators on:

- The single highest-level entry function for the logical agent flow → `@trace_with_workflow`.
- The LLM-calling function(s) inside that flow → `@trace_with_task`.

Show these as diffs. Both decorator types should be present in the proposal before asking about additional helper functions or sub-orchestrators. **Don't propose 20 decorators on a first pass.**

## 7. Diff format for proposals

Show each placement as a unified diff so the customer sees the exact line where the decorator lands and the import that needs to be added (if not already present):

```
--- src/agent.py
+++ src/agent.py
@@ -1,3 +1,5 @@
+import montecarlo_opentelemetry as mc
+
 def chat_agent(message: str) -> str:
     ...
@@ -10,2 +12,6 @@
+@mc.trace_with_workflow(
+    span_name="chat_agent",
+    workflow_name="customer-support",
+)
 def chat_agent(message: str) -> str:
```

Wait for `yes` before applying. If the customer says "looks good, apply all of them" — confirm explicitly: *"I'll apply these N diffs now — confirm?"* before doing anything.

## Common mistakes

- **Scaffolding any decorator other than `@trace_with_workflow` or `@trace_with_task`** — those are the only two in scope for v1. Other SDK tracing primitives are not part of the surface this skill proposes.
- **Producing task-only or workflow-only traces** — always propose both `@trace_with_workflow` and `@trace_with_task`. Place `@trace_with_workflow` on the function that calls the LLM-calling function, and `@trace_with_task` on the LLM-calling function itself.
- **Decorating every helper function** — span noise. Aim for 1 workflow on the agent entry / enclosing function + 1 task per LLM call.
- **Using `task_name="call_llm"` or `workflow_name="main"`** — opaque. Match the customer's domain.
- **Applying diffs without explicit per-file confirmation** — violates `SKILL.md` guardrail.
- **Skipping the workflow decorator because the flow has only one LLM call** — wrong. The enclosing function is still the workflow boundary.
- **Treating "apply all" as blanket approval** — always re-confirm before bulk-applying multiple diffs.
- **Inferring `workflow_name` / `task_name` from a generic or non-descriptive function name** — match the domain purpose, not opaque names like `main` or `agent`. A meaningful, distinct function name (`summarize`, `extract_entities`) is fine to reuse.

# Library detection and runtime classification

This reference governs how the instrument-agent skill decides **which AI libraries to instrument** and **what runtime template to use**. The contract has two pieces: `scripts/detect_libraries.py` (a thin discovery layer over the customer's repo) and `scripts/fetch_sdk_docs.py` (the live PyPI lookup that names the SDK's currently-supported instrumentors). The matching between the two is the LLM's job — there is no static map in the skill.

## Inputs the skill works from

`scripts/detect_libraries.py` returns a single JSON document of this shape:

```json
{
  "dependencies": ["anthropic", "boto3", "fastapi", "langchain", "langgraph", "openai"],
  "runtime": "serverless",
  "serverless_signals": ["serverless.yml", "lambda_handler"],
  "existing_setup": {"found": true, "files": ["src/tracing.py"]}
}
```

Field meanings:

- `dependencies` — sorted list of normalized pip package names parsed from `requirements.txt` / `pyproject.toml` / `Pipfile`. Raw surface; the script does not classify which entries are AI-relevant. Everything the customer declared is here, lowercased.
- `runtime` — `serverless`, `long_running`, or `unknown`. `serverless` if any serverless signal is found. `long_running` if a dep manifest was found but no serverless signals. `unknown` when no dep manifest exists at all (so we can't reason about it).
- `serverless_signals` — what triggered the serverless classification (e.g. `lambda_handler`, `serverless.yml`, `mangum`).
- `existing_setup` — `{ found: bool, files: list[str] }` for any pre-existing `mc.setup()` call. `files` contains repo-relative paths.

`scripts/fetch_sdk_docs.py` returns the SDK's live supported-instrumentor list from PyPI:

```json
{
  "source": "pypi",
  "sdk": {"version": "...", "pypi_url": "https://pypi.org/project/montecarlo-opentelemetry/"},
  "supported_instrumentors": [
    {"library": "langchain", "package": "opentelemetry-instrumentation-langchain", "version_constraint": "<=0.53.4"},
    {"library": "openai", "package": "opentelemetry-instrumentation-openai", "version_constraint": "<=0.53.4"},
    {"library": "anthropic", "package": "opentelemetry-instrumentation-anthropic"}
  ]
}
```

That payload is the source of truth for what to install. If PyPI is unreachable, the script exits with `source: "error"` and a `guidance` field pointing at the PyPI page — surface that to the user rather than guessing.

## 1. The supported library set comes from PyPI

The Monte Carlo OpenTelemetry SDK supports a set of AI libraries that is published on PyPI: https://pypi.org/project/montecarlo-opentelemetry/. That page is the source of truth for what's currently supported — full stop.

`scripts/fetch_sdk_docs.py` queries PyPI live to retrieve that set and the per-instrumentor version pins. There is no offline fallback; on PyPI failure, `fetch_sdk_docs.py` exits with an error and the skill must surface that to the user rather than guessing.

A non-exhaustive subset of currently-supported libraries (examples — see PyPI for the current full list):

| Library | Instrumentor package |
|---|---|
| langchain (covers langgraph) | `opentelemetry-instrumentation-langchain` |
| openai | `opentelemetry-instrumentation-openai` |
| anthropic | `opentelemetry-instrumentation-anthropic` |
| crewai | `opentelemetry-instrumentation-crewai` |
| bedrock | `opentelemetry-instrumentation-bedrock` |
| sagemaker | `opentelemetry-instrumentation-sagemaker` |
| vertexai | `opentelemetry-instrumentation-vertexai` |

> **NEVER**: Hardcode a version constraint into a generated `requirements.txt` or `pyproject.toml` without first running `fetch_sdk_docs.py`. If PyPI is unreachable, surface the error to the user — don't invent a pin.

## 2. How matching works (LLM-driven)

There is one supported tier: whatever the SDK currently supports per PyPI. The matching flow:

1. Run `detect_libraries.py` against the target. It returns the raw `dependencies` list (every pip package the customer declared) plus `runtime`, `serverless_signals`, and `existing_setup`.
2. Run `fetch_sdk_docs.py` to get the SDK's `supported_instrumentors` list from PyPI.
3. **Match `dependencies` against `supported_instrumentors`.** The LLM does this — there's no static map. Walk the customer's deps and for each one decide whether an instrumentor covers it. Use the `library` slug in `supported_instrumentors` plus your knowledge of which pip packages each instrumentor wraps (e.g. `langchain-core` and `langchain-community` are part of the `langchain` instrumentor's surface; `langgraph` is also covered by `opentelemetry-instrumentation-langchain`).
4. **Ask the customer when a dep is ambiguous.** Some pip packages don't map cleanly to one instrumentor — see section 4. Always disambiguate explicitly rather than guessing.
5. **Use PyPI as the tiebreaker.** If you're unsure whether a particular dep maps to an instrumentor, the PyPI README (which `fetch_sdk_docs.py` parses) is canonical. If it doesn't appear there, there's no auto-instrumentor for it.

### Decorators and manual spans are independent of auto-instrumentors

`@trace_with_workflow`, `@trace_with_task`, and `mc.create_llm_span` are SDK-level affordances that work regardless of whether an auto-instrumentor exists for the underlying library. Do not present them as a *substitute* for auto-instrumentation — they serve different purposes:

- If an auto-instrumentor exists on PyPI for a customer's AI library, install it.
- Decorators and `mc.create_llm_span` are *additionally* available for orchestration spans and bespoke LLM calls.

## 3. Multi-library detection rules

When multiple AI libraries appear in `dependencies`, treat them as **additive** — install all matched instrumentors. A single `mc.setup()` call lists all of them:

```python
mc.setup(instrumentors=[
    LangchainInstrumentor(),
    OpenAIInstrumentor(),
])
```

> **IMPORTANT**: Multiple libraries can share one instrumentor package (e.g. `langchain` and `langgraph` both ship via `opentelemetry-instrumentation-langchain`). Deduplicate by package when building the install set and the `instrumentors=[...]` list — installing or instantiating the same instrumentor twice is a bug.

> **IMPORTANT**: When `dependencies` contains no AI libraries from the PyPI supported list AND `runtime: "unknown"` — there are no AI libraries to instrument. Exit cleanly. Do **not** scaffold a `mc.setup()` for nothing. See section 7.

## 4. Ambiguous-multipurpose-SDK rule (boto3, etc.)

Some pip packages cover a broad surface and don't tell us which AI service (if any) the customer is using:

- `boto3`, `botocore`, `aioboto3` — cover the entire AWS surface. Could mean Bedrock, SageMaker, or just S3 / DynamoDB / SQS / anything else.
- `google-cloud-aiplatform` — could be Vertex AI inference or Vertex AI Search.
- `azure-ai-*` — covers many distinct Azure AI products.

`detect_libraries.py` doesn't single these out — they appear in `dependencies` like any other package. **The LLM handles the disambiguation by asking the customer.** When `boto3` is present, ask "are you calling Bedrock or SageMaker through boto3, or is it just generic AWS work?". Don't install `opentelemetry-instrumentation-bedrock` until the customer confirms Bedrock usage.

> **NEVER**: Silently install `opentelemetry-instrumentation-bedrock` (or `-sagemaker`) just because `boto3` is in the dependency list. Always ask first.

## 5. Serverless framework detection

`detect_libraries.py` sets `runtime: "serverless"` when **ANY** of the following is present in the customer's project:

**Files**

- `serverless.yml`, `serverless.yaml` — Serverless Framework
- `template.yaml`, `template.yml` — AWS SAM
- `vercel.json` — Vercel
- `netlify.toml` — Netlify
- `wrangler.toml` — Cloudflare Workers
- `zappa_settings.json` — Zappa
- `modal.toml` — Modal

**Dependencies**

- `aws-lambda-powertools`, `mangum`, `chalice`, `zappa`
- `aws-cdk-lib`, `aws-sam-cli`
- `modal`, `sst`

**Code patterns**

- `def lambda_handler(`
- `from chalice import Chalice`
- `from mangum import Mangum`
- `app = Chalice(`

The matched signal name (file name or pattern) appears in `serverless_signals` in the JSON output. Use that list to explain the runtime classification when the user asks "why did you pick the serverless template?".

> **CRITICAL**: When `runtime: "serverless"`, the skill must use the **`SimpleSpanProcessor`** template — see `setup-template.md`. Without it, traces are silently dropped on Lambda when the batch processor is suspended before flushing the queue. This foot-gun is also documented in `troubleshooting.md`.

### Ask the user when serverless signals are ambiguous

Detection is intentionally broad — a single signal is enough to flip `runtime` to `serverless`. That's the right call when the project is clearly Lambda/Vercel/etc., but it's wrong for codebases where the serverless framework applies to only part of the project:

- A monorepo where `template.yaml` lives under one subdirectory and the rest of the code is a long-running service.
- A repo with `serverless.yml` for an auxiliary handler, but the AI code runs in a separate long-running worker.
- A single weak signal (e.g., `mangum` in deps) without any handler entry point or framework config file.

When the picture is borderline — one signal, or signals that don't obviously cover the code where the AI libraries are used — ask the user before committing to the serverless template. Show them `serverless_signals` and confirm whether the AI code actually runs in that serverless context. If only part of the codebase is serverless, the user may need different templates for different entry points.

### Other runtime values

- `runtime: "long_running"` — at least one dependency manifest exists and no serverless signal was observed. Use the standard batch-processor template.
- `runtime: "unknown"` — no dependency manifest found in the target. Ask the user where the agent code lives before scaffolding anything.

> **NEVER**: Auto-scaffold `mc.setup()` when `runtime: "unknown"`. Choosing the wrong span processor will silently drop traces (serverless) or add unnecessary memory pressure (long-running). Ask.

## 6. Existing-`mc.setup()` detection

If `existing_setup.found: true`, the customer already has Monte Carlo OpenTelemetry instrumentation in their codebase. The list under `existing_setup.files` shows where.

> **CRITICAL**: Do **not** scaffold a duplicate `mc.setup()`. Route to the existing-setup decision matrix in `setup-template.md` to walk through whether to update the existing call vs. leave it alone. A second `mc.setup()` will produce duplicate spans and confusing traces.

## 7. No-match exit

If after matching `dependencies` against `fetch_sdk_docs.py`'s `supported_instrumentors` you find no AI library that the SDK supports, exit cleanly with this message:

> "No supported AI libraries were detected in your dependency files. The Monte Carlo OpenTelemetry SDK supports a set of libraries that's published on PyPI: https://pypi.org/project/montecarlo-opentelemetry/. You can run `scripts/fetch_sdk_docs.py` to see the current supported set. If you'd like to share your `requirements.txt` / `pyproject.toml` / `Pipfile`, I'll re-check."

If the user names a specific library that isn't in `dependencies`, run `fetch_sdk_docs.py` to confirm whether PyPI currently lists an instrumentor for it, then proceed per section 2.

> **NEVER**: Scaffold `mc.setup()` against an empty instrumentor list unless the customer is manually reporting every LLM call with `mc.create_llm_span`. An empty setup call without manual spans is worse than no setup call — it implies instrumentation is in place when it isn't.

## 8. Version pinning

For each instrumentor you propose installing, take the `version_constraint` from `fetch_sdk_docs.py`'s `supported_instrumentors[*]` entry. That value is parsed live from the PyPI README's `pip install` lines (e.g. `<=0.53.4`). Apply it directly in the customer's dependency-file diff — never strip it.

Some instrumentor versions have transitive compatibility constraints that aren't expressed in PyPI metadata. The most common one in the current SDK release is the `wrapt<2` requirement for OpenLLMetry instrumentors (they pass `module=` to `wrap_function_wrapper`, which `wrapt` 2.x renamed to `target=`). The skill surfaces these as **symptom-driven fixes** in `troubleshooting.md` rather than baking them into every install diff — if a customer hits the symptom, the troubleshooting reference names the pin.

## Common mistakes

- **Installing the `bedrock` instrumentor when only `boto3` is detected.** Wrong — `boto3` is multi-purpose. Always ask the customer whether they're actually using Bedrock before installing.
- **Hardcoding a version constraint without running `fetch_sdk_docs.py`.** Wrong — PyPI live is the source of truth for instrumentor version pins. If PyPI is unreachable, surface the error rather than inventing a pin.
- **Skipping the disambiguation prompt for ambiguous deps.** Wrong — `boto3`, `google-cloud-aiplatform`, `azure-ai-*` all need explicit user confirmation before installing any instrumentor.
- **Silently auto-scaffolding when `runtime: "unknown"` or when serverless signals are weak/partial.** Wrong — ask the user before picking a template. The wrong span processor drops traces or wastes memory.
- **Treating decorators / `create_llm_span` as a *substitute* for an auto-instrumentor.** Wrong — they are independent. If an auto-instrumentor exists on PyPI, install it. Decorators and manual spans are additionally available for orchestration and bespoke LLM calls.
- **Trusting a stale memory of the supported set instead of `fetch_sdk_docs.py`.** Wrong — the supported set is whatever PyPI currently lists. Re-fetch.
- **Scaffolding a duplicate `mc.setup()` when `existing_setup.found: true`.** Wrong — duplicate setup produces duplicate spans. Route to the existing-setup decision matrix in `setup-template.md`.
- **Scaffolding `mc.setup()` against an empty instrumentor list with no manual reporting.** Wrong — empty `instrumentors=[]` is only useful when every LLM call is manually reported with `mc.create_llm_span`. Otherwise exit cleanly with the no-match message.
- **Editing `requirements.txt` / `pyproject.toml` / `Pipfile` without explicit user approval.** Wrong — always propose the diff and wait for confirmation. See SKILL.md's CRITICAL no-silent-edit guardrail.
- **Forgetting the `wrapt<2` pin and getting a `TypeError` at `mc.setup()` import.** Surface the symptom path in `troubleshooting.md` if the customer hits it — the fix is to pin `wrapt<2` alongside the OpenLLMetry instrumentors.

# Redaction guidance (V1)

Reference for the V1 redaction guidance supported by the Monte Carlo
instrument-agent skill. Read this before generating any `mc.setup()` snippet or
proposing instrumentation that touches LLM calls.

---

## 1. What the SDK captures by default — and where it lives

> **CRITICAL — capture-on is the value proposition, not a footgun.** The
> Monte Carlo OpenTelemetry SDK plus the `opentelemetry-instrumentation-*`
> auto-instrumentors capture full LLM **prompt** and **completion** content
> as span attributes whenever an instrumentor is loaded. This is the core use
> case: low-lift auto-instrumentation that records what the agent said and what
> the model said back.

The facts the customer needs to hear up front:

1. **SDK default.** When an OpenLLMetry instrumentor is loaded
   (`opentelemetry-instrumentation-langchain`, `-openai`, `-anthropic`,
   `-bedrock`, `-vertexai`, etc.), the auto-instrumentor wraps the LLM SDK
   call directly and records full prompt and completion content as span
   attributes. No decorator or manual span is required for capture.
2. **Transport.** Spans are sent over OTLP to whatever endpoint is
   passed to `mc.setup(otlp_endpoint=...)`. The customer always supplies
   the endpoint explicitly — the SDK has no built-in default. The
   templates in `setup-template.md` resolve it from an env var
   (`OTEL_ENDPOINT`). The MC-hosted collector also requires credentials
   or OTLP headers as shown in `setup-template.md`; a self-hosted
   collector handles auth at the collector.
3. **Data residency — traces live in the customer's environment.** The
   MC-hosted collector is a write-back pass-through. It routes spans back
   to the customer's storage and **does not persist trace content on
   Monte Carlo's side.** Trace content stays in the customer's environment.

For most customers, ship with full capture. **Prompt/completion content is
the most valuable thing the SDK records** — token counts and span shapes
alone don't answer "why did the agent say that?"

---

## 2. When customers want redaction

Most customers ship with full capture. A subset with stricter requirements
choose to redact. Examples of stricter situations:

- **HIPAA workloads** where prompts or completions can contain PHI, and
  the customer's policy is that PHI never enters any tracing or
  observability tool regardless of where it's stored.
- **Customers whose contracts forbid PII in tracing tools** — some
  enterprise contracts treat tracing systems as a separate data-handling
  surface, independent of where the underlying data lives.
- **Multi-tenant agents** where prompts contain another customer's content
  and the operator wants to scrub before it lands in their own trace store.
- **Credential / secret leakage risk** — agents that occasionally receive
  API keys or tokens in user input.

Redaction is a choice for these customers, not a default privacy posture.
The skill walks them through how to opt out of content capture (and
optionally substitute placeholders) when they ask.

---

## 3. Layer 1 for redaction: disable auto-instrumentor content capture

> **CRITICAL — disabling auto-capture is a hard prerequisite for ALL redaction.** If
> the customer keeps the auto-instrumentor with content capture on AND
> also calls `mc.create_llm_span` with redacted prompts, they end up with
> **duplicate spans** — one redacted (manual) and one with the full
> content (auto). That defeats the redaction. Any redaction story starts
> with disabling auto-capture.

**How.** The OpenLLMetry instrumentors all read a single env var:
`TRACELOOP_TRACE_CONTENT`. Set it to `"false"` to disable content capture
across the entire OpenLLMetry instrumentor family.

```bash
export TRACELOOP_TRACE_CONTENT=false
```

Or in code, **before any instrumentor imports**:

```python
import os
os.environ.setdefault("TRACELOOP_TRACE_CONTENT", "false")

# Only AFTER the env var is set can the instrumentor imports be safe:
import monte_carlo_observability_sdk as mc
mc.setup(...)
```

> **NEVER document `OTEL_INSTRUMENTATION_<lib>_TRACE_PROMPTS`** as the
> mechanism. Those env vars do not exist in the OpenLLMetry instrumentors.
> `TRACELOOP_TRACE_CONTENT` is the single source of truth.

**What is still captured with content capture disabled:**

- The full trace tree (workflow → task → span hierarchy).
- Span timings and latency.
- Token counts.
- Model identifiers.
- Tool call structure (which tools were called, in what order).

**What is dropped:**

- Prompt text.
- Completion text.
- Tool call argument values (depending on the instrumentor — verify
  per-instrumentor before promising the customer this).

For customers who want zero content but still want trace shape, this is
the complete answer. For customers who want some content with sensitive
fields scrubbed, layer manual redacted spans on top.

---

## 4. Layer 2 for selective content: manual `mc.create_llm_span` with placeholder-substituted `prompts_to_record`

**When to use.** The customer has already disabled auto-capture and wants
spans to record an audit trail of LLM calls with sensitive fields replaced
by placeholders — instead of having no content at all.

**How — the placeholder-substitution technique.** Keep **two sets of
prompts** in memory:

- One set with placeholder values where sensitive fields would go (e.g.,
  `"<SSN>"`, `"<EMAIL>"`, `"<CUSTOMER_NAME>"`). This set is what gets
  passed to `prompts_to_record`.
- One set with the real sensitive values. This set is what gets sent to
  the LLM.

The structure of the recorded prompt is preserved (role, shape,
non-sensitive context) while the sensitive fields are replaced with stable
placeholders that are useful for debugging without leaking content.

```python
# Build two prompt sets: one with placeholders for tracing, one real for the LLM.
redacted_messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": f"Look up account for customer <CUSTOMER_NAME>"},
]
full_messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": f"Look up account for customer {real_customer_name}"},
]

with mc.create_llm_span(
    span_name="anthropic.chat",
    provider="anthropic",
    model=model_name,
    operation="chat",
    prompts_to_record=redacted_messages,  # <- placeholder version recorded in span
) as span:
    # Send the FULL (un-redacted) version to the LLM:
    result = invoke_model(model, full_messages, logger, model_type)

    # Helpers populate response-side span attributes:
    mc.add_llm_response_model(span, model_config.bedrock_model)
    mc.add_llm_completions(
        span,
        # Redact the completion too if the response can contain sensitive content:
        [{"role": "assistant", "content": redact_completion(str(result.content))}],
    )
    mc.add_llm_tokens(
        span,
        prompt_tokens=result.usage.input_tokens,
        completion_tokens=result.usage.output_tokens,
        total_tokens=result.usage.total_tokens,
    )
```

The key idea: **`prompts_to_record`** is what gets stored in the span, and
it can differ from what is sent to the LLM. The customer builds the
placeholder-substituted version and passes it to `prompts_to_record`; the
real values go to the LLM separately.

Walk-through points to cover with the customer:

- `prompts_to_record` takes a list of `{"role": ..., "content": ...}`
  dicts. Shape it the same as `full_messages` so spans remain readable.
- The customer is responsible for the substitution logic. **The SDK does
  not redact.**
- Pick stable placeholder tokens (e.g., `<SSN>`, `<EMAIL>`) so future
  debuggers reading the trace can recognize the structure.
- Response-side helpers populate span attributes after the LLM call:
  - `mc.add_llm_response_model(span, ...)` — model identifier of the
    response.
  - `mc.add_llm_completions(span, [...])` — completion content (also
    accepts a placeholder-substituted list).
  - `mc.add_llm_tokens(span, prompt_tokens=..., completion_tokens=...,
    total_tokens=...)` — token counts (no content).

> **NEVER** pass the un-substituted messages as `prompts_to_record`. The
> whole point is that the placeholder version is what reaches the span.
> Mixing the two defeats redaction entirely.

> **IMPORTANT** — the same discipline applies to `mc.add_llm_completions`.
> If the response can contain sensitive content (e.g., a model that
> summarizes PHI), substitute placeholders in the completion before
> passing it to `add_llm_completions` too. A scrubbed prompt with a raw
> completion still leaks.
>
> Completion redaction is often harder than prompt redaction because model
> output is nondeterministic. Ask the customer what the expected output is
> and whether it can contain sensitive data. If the completion is
> unstructured or there is no reliable way to know which part is sensitive,
> redact the whole completion or omit completion content rather than
> recording a partial scrub that may leak.

> **IMPORTANT** — decorators (e.g., `@trace_with_task`) only add
> workflow/task metadata around a function. They do **not** gate what the
> auto-instrumentor captures inside that function. If auto-capture is on,
> the LLM SDK call is wrapped regardless of decorator presence. The
> `TRACELOOP_TRACE_CONTENT=false` env-var disable is the only way to stop
> auto-capture.

---

## 5. Choosing redaction configurations

| Scenario | Required setup |
|---|---|
| No redaction needed — capture everything (default) | Leave auto-instrumentor alone with full content capture. No env var change, no manual spans. |
| Want trace tree but **no** content at all | Disable auto-capture only — set `TRACELOOP_TRACE_CONTENT=false` before instrumentor imports. |
| Want trace tree + selective content with placeholders | Disable auto-capture, then call `mc.create_llm_span` with placeholder-substituted `prompts_to_record` (and `add_llm_completions`) at sensitive call sites. |

> **IMPORTANT — do not propose manual redacted spans without disabling auto-capture.** Without the
> env-var disable, the auto-instrumentor and the manual span both fire,
> producing duplicate spans (one redacted, one full-content). The
> redaction is silently undone.

---

## 6. What V1 does NOT do

> **OUT OF SCOPE for v1** — The skill does **not** auto-detect sensitive
> content (no automatic PII scanning) and does **not** scaffold redactor
> or placeholder-substitution functions for the customer.

The skill is *conversant* in the options above and walks the customer
through them. **The customer writes their substitution logic.** If a
customer asks the skill to "build me a redactor," the correct response is
to walk them through disabling auto-capture plus optional manual redacted
spans with their existing utilities (or to recommend they write the
substitution helpers themselves) — not to
scaffold one in their codebase.

---

## 7. NEVER edit any file without explicit user approval

When proposing a redaction change, the SKILL.md rule applies to every
single code change:

- **Disable auto-capture** → propose the env var setting in the relevant config
  (e.g., `.env.example`, deployment manifest, or the `mc.setup()` module
  with `os.environ.setdefault(...)` before imports). Wait for per-file
  approval.
- **Optional manual redacted spans** → propose the manual span wrap as a diff
  to the relevant function. Wait for per-file approval. Don't auto-apply.

> **NEVER** apply a redaction change in the customer's repo without
> their explicit approval for that specific file. Redaction changes
> touch the data plane; a wrong default here can leak sensitive content
> into traces or silently drop content the customer expected to see.

---

## Common mistakes

- **Treating env-var disable as optional when redaction is wanted.**
  Disabling auto-capture is mandatory for any redaction story. Without it,
  the auto-instrumentor still fires alongside the manual span and produces
  duplicate spans — one redacted, one full-content. The redaction is defeated.
- **Misstating data residency.** Trace content lives in the customer's
  environment. The MC-hosted collector routes spans back to the customer's
  storage without persisting content on the MC side. Don't tell customers
  "MC stores your prompts" — that's wrong.
- **Framing redaction as a privacy default.** Capture-on is the value
  proposition. Redaction is a choice for stricter customers, not a
  required privacy posture.
- **Recommending lossy fingerprints (e.g., hashing the prompt with its
  character count) as the primary technique.** Placeholder-substitution
  is the recommended
  structured technique — it preserves prompt shape and is useful for
  debugging. Hashes throw away the structure that makes the trace
  readable.
- **Assuming decorators gate auto-capture.** They don't. The
  auto-instrumentor wraps the LLM SDK call regardless of whether the
  surrounding function is decorated. Only `TRACELOOP_TRACE_CONTENT=false`
  stops auto-capture.
- **Setting `TRACELOOP_TRACE_CONTENT` *after* instrumentor imports.** Too
  late — the instrumentors read the env var at import/init time. Set it
  before any `mc.setup()` or instrumentor import runs.
- **Passing un-substituted messages to `prompts_to_record`.** Defeats
  the entire purpose of manual redacted spans. Confirm the placeholder
  version is what reaches `prompts_to_record`.
- **Forgetting that completions are content too.** Manual redacted spans apply to
  `mc.add_llm_completions` as well. A placeholder-substituted prompt with a
  raw completion still leaks. If the completion can contain sensitive data
  and cannot be scrubbed reliably, redact or omit the whole completion.
- **Auto-scaffolding a redactor or substitution helper.** Out of scope
  for v1. Walk the customer through the redaction options; they write the
  substitution logic.

# `mc.setup()` Template Reference

How to wire `mc.setup()` correctly for a customer's agent. This is a Tier 3 reference — use it once the workflow has classified runtime, picked an OTLP endpoint, and decided on prompt/completion capture.

Single concern: how the skill turns the workflow's answers into a correct, runnable `mc.setup()` snippet.

## SDK shape

```python
import montecarlo_opentelemetry as mc

mc.setup(
    agent_name=...,
    otlp_endpoint=...,
    instrumentors=[...],
    span_processor=...,  # optional; required for serverless
)
```

> **Source of truth for the `span_processor` kwarg contract:** the [`montecarlo-opentelemetry` PyPI page](https://pypi.org/project/montecarlo-opentelemetry/) (which mirrors the package README). When the SDK changes the kwarg's behavior, default, or auth-header injection rules, that page is the canonical source — re-read it before regenerating templates.

---

## 1. Choosing the template

Branch on `runtime` from `scripts/detect_libraries.py` **and** the collector / auth choices from workflow steps #2 and #9. Each combination has its own self-contained template below — pick one and paste it as-is (after substituting `agent_name`, endpoint resolution, and the instrumentor list). Do not mix-and-match between blocks.

| `runtime` value | Collector | Auth env vars | Template |
|---|---|---|---|
| `long_running` | any | any | [Long-running container](#long-running-container-default-batchspanprocessor) |
| `serverless` | MC-hosted | `MCD_DEFAULT_API_ID` / `MCD_DEFAULT_API_TOKEN` | [Serverless + MC-hosted + MCD_DEFAULT_*](#serverless--mc-hosted-collector--mcd_default_-env-vars) |
| `serverless` | MC-hosted | `OTEL_EXPORTER_OTLP_HEADERS` | [Serverless + MC-hosted + OTEL_EXPORTER_OTLP_HEADERS](#serverless--mc-hosted-collector--otel_exporter_otlp_headers) |
| `serverless` | Self-hosted | (auth at collector) | [Serverless + self-hosted collector](#serverless--self-hosted-collector) |
| `unknown` | — | — | Ask the user. Default to long-running, with an explicit note that the customer should switch to a serverless template if the agent runs on Lambda or another suspendable runtime. |

> **CRITICAL — serverless without `SimpleSpanProcessor` silently drops traces.** Lambda freezes the process between invocations. The default `BatchSpanProcessor` is suspended before its flush interval fires, and the spans never reach Monte Carlo. Symptom: customer instrumented their Lambda agent, ran it, and sees no traces in `get_agent_metadata`. Fix: switch to one of the serverless templates below. See `troubleshooting.md`.

> **CRITICAL — match the auth-path branch to the customer's actual setup before generating the snippet.** The `MCD_DEFAULT_*` template references `os.environ["MCD_DEFAULT_API_ID"]`; if the customer is on `OTEL_EXPORTER_OTLP_HEADERS` or self-hosted, that line raises `KeyError` at startup and tracing never initializes. Walk the customer through which auth path they're using *before* proposing the diff.

### Long-running container (default `BatchSpanProcessor`)

The default template. The SDK's built-in `BatchSpanProcessor` batches spans for efficient export, which is correct for any process that stays resident (containers, VMs, long-running workers).

```python
import os

import montecarlo_opentelemetry as mc
from opentelemetry.instrumentation.langchain import LangchainInstrumentor

# Resolve endpoint from env. If unset, skip setup so the agent runs uninstrumented.
otel_endpoint = os.getenv("OTEL_ENDPOINT")
if otel_endpoint:
    base_endpoint = otel_endpoint.rstrip("/")
    http_otel_endpoint = (
        base_endpoint
        if base_endpoint.endswith("/v1/traces")
        else f"{base_endpoint}/v1/traces"
    )

    mc.setup(
        agent_name="ai-agent",
        otlp_endpoint=http_otel_endpoint,
        instrumentors=[LangchainInstrumentor()],
    )
```

This template lets the OpenLLMetry instrumentors capture prompt/completion content (their default). For customers who want to suppress content capture, see the [prompts-disabled variant](#prompts-disabled-variant-opt-in-for-stricter-customers).

### Serverless + MC-hosted collector + `MCD_DEFAULT_*` env vars

The customer runs on Lambda (or another suspendable runtime), sends traces to Monte Carlo's hosted collector, and has `MCD_DEFAULT_API_ID` / `MCD_DEFAULT_API_TOKEN` set in their environment (workflow Step 9's preferred path).

`mc.setup()` only auto-injects `MCD_DEFAULT_*` headers when it builds the default exporter. With a custom `span_processor` we build the exporter ourselves, so we pass the auth headers explicitly.

```python
import logging
import os

import montecarlo_opentelemetry as mc
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.langchain import LangchainInstrumentor
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

AGENT_NAME = "monitoring-agent"


def init_tracing():
    otel_endpoint = os.getenv("OTEL_ENDPOINT")
    if not otel_endpoint:
        return  # tracing disabled

    # Use .get() (not os.environ[...]) so a partial-config window — endpoint set
    # but credentials missing — skips tracing instead of crashing the Lambda at
    # cold start with a KeyError.
    api_id = os.environ.get("MCD_DEFAULT_API_ID")
    api_token = os.environ.get("MCD_DEFAULT_API_TOKEN")
    if not api_id or not api_token:
        logging.warning(
            "Monte Carlo tracing disabled: OTEL_ENDPOINT is set but "
            "MCD_DEFAULT_API_ID / MCD_DEFAULT_API_TOKEN are missing."
        )
        return

    base_endpoint = otel_endpoint.rstrip("/")
    http_otel_endpoint = (
        base_endpoint
        if base_endpoint.endswith("/v1/traces")
        else f"{base_endpoint}/v1/traces"
    )

    mcd_headers = {"x-mcd-id": api_id, "x-mcd-token": api_token}

    # SimpleSpanProcessor flushes each span before the runtime can suspend the
    # process. BatchSpanProcessor would queue spans and lose them at freeze.
    exporter = OTLPSpanExporter(endpoint=http_otel_endpoint, headers=mcd_headers)
    simple_span_processor = SimpleSpanProcessor(exporter)

    mc.setup(
        agent_name=AGENT_NAME,
        otlp_endpoint=http_otel_endpoint,  # required by signature; ignored when span_processor is set
        instrumentors=[LangchainInstrumentor()],
        span_processor=simple_span_processor,
    )
```

### Serverless + MC-hosted collector + `OTEL_EXPORTER_OTLP_HEADERS`

The customer runs on Lambda, sends to Monte Carlo's hosted collector, and packs auth into the standard OTel env var (`OTEL_EXPORTER_OTLP_HEADERS=x-mcd-id=...,x-mcd-token=...`). `OTLPSpanExporter` reads that env var automatically, so the exporter takes no explicit `headers=` kwarg.

```python
import os

import montecarlo_opentelemetry as mc
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.langchain import LangchainInstrumentor
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

AGENT_NAME = "monitoring-agent"


def init_tracing():
    otel_endpoint = os.getenv("OTEL_ENDPOINT")
    if not otel_endpoint:
        return  # tracing disabled

    base_endpoint = otel_endpoint.rstrip("/")
    http_otel_endpoint = (
        base_endpoint
        if base_endpoint.endswith("/v1/traces")
        else f"{base_endpoint}/v1/traces"
    )

    # OTLPSpanExporter reads OTEL_EXPORTER_OTLP_HEADERS from the environment
    # automatically — no explicit `headers=` kwarg needed.
    exporter = OTLPSpanExporter(endpoint=http_otel_endpoint)
    simple_span_processor = SimpleSpanProcessor(exporter)

    mc.setup(
        agent_name=AGENT_NAME,
        otlp_endpoint=http_otel_endpoint,
        instrumentors=[LangchainInstrumentor()],
        span_processor=simple_span_processor,
    )
```

### Serverless + self-hosted collector

The customer runs on Lambda and sends to their own collector. Auth is handled at the collector — Monte Carlo never sees credentials. **Do not** reference `MCD_DEFAULT_*` anywhere in this template (not as a value read, not as a comment, not in a fallback).

```python
import os

import montecarlo_opentelemetry as mc
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.langchain import LangchainInstrumentor
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

AGENT_NAME = "monitoring-agent"


def init_tracing():
    otel_endpoint = os.getenv("OTEL_ENDPOINT")
    if not otel_endpoint:
        return  # tracing disabled

    base_endpoint = otel_endpoint.rstrip("/")
    http_otel_endpoint = (
        base_endpoint
        if base_endpoint.endswith("/v1/traces")
        else f"{base_endpoint}/v1/traces"
    )

    # Auth is enforced at the customer's collector; no headers from the exporter.
    exporter = OTLPSpanExporter(endpoint=http_otel_endpoint)
    simple_span_processor = SimpleSpanProcessor(exporter)

    mc.setup(
        agent_name=AGENT_NAME,
        otlp_endpoint=http_otel_endpoint,
        instrumentors=[LangchainInstrumentor()],
        span_processor=simple_span_processor,
    )
```

---

## 2. OTLP endpoint normalization

Customers provide either a collector base URL or a full `/v1/traces` endpoint. The skill must normalize **idempotently** — the same input run twice must produce the same output.

```python
base = customer_provided_url.rstrip("/")
if base.endswith("/v1/traces"):
    http_otel_endpoint = base
else:
    http_otel_endpoint = f"{base}/v1/traces"
```

Rules:

- If the URL already ends in `/v1/traces`, use as-is.
- Otherwise, append `/v1/traces` to the base.
- **NEVER double-append** (`https://collector/v1/traces/v1/traces` is broken).
- Strip trailing slashes before checking the suffix — `https://collector/v1/traces/` should not become `https://collector/v1/traces//v1/traces`.

> **IMPORTANT — show the resolved final URL to the user before generating any code.** Do not silently rewrite the customer's input. Ask: "I'll use `<resolved-url>` as the OTLP endpoint — is that correct?" and wait for confirmation. The customer needs to recognize the URL their collector is actually going to receive.

### Endpoint sources

| Source | Base URL | Resolved endpoint |
|---|---|---|
| MC-hosted collector | `https://integrations.getmontecarlo.com/otel` (per https://docs.getmontecarlo.com/docs/mcp-server) | `https://integrations.getmontecarlo.com/otel/v1/traces` |
| Self-hosted collector | Customer's own deploy | Ask the customer for the base URL, then normalize |

---

## 3. Prompt/completion capture — default on, opt-out for stricter customers

The default template the skill proposes **captures** prompt and completion content. That is the core value proposition: low-lift auto-instrumentation that records what the agent said and what the model said back. The OpenLLMetry instrumentors (`opentelemetry-instrumentation-langchain`, `-openai`, `-anthropic`, etc.) wrap the LLM SDK call and record full content as span attributes by default — no extra wiring needed.

**Data residency.** Whether the customer routes through the MC-hosted collector or a self-hosted one, trace content lives in the **customer's environment**. The MC-hosted collector is a write-back pass-through; it does not persist content on Monte Carlo's side. The decision about capturing content is a question of the customer's own risk tolerance and compliance posture, not about data leaving their network. See `redaction.md` for the full framing.

For most customers, ship the default templates in Section 1 unchanged.

### Prompts-disabled variant (opt-in for stricter customers)

A subset of customers (HIPAA workloads, regulated industries, company policy) prefer to suppress prompt/completion capture and rely on the structural value of traces (span shapes, latency, token counts, error rates) rather than the content itself.

When the workflow's redaction step (Step 3) returned "yes, redact," use this variant of whichever Section 1 template the runtime/collector branch picked. The only differences:

1. Set `TRACELOOP_TRACE_CONTENT=false` in code, before any instrumentor import. The OpenLLMetry instrumentors at `<=0.53.4` read this env var at span-emit time; setting it in code (rather than as a comment) is the only way to flip the default from inside the template.
2. Optionally wrap LLM calls with `mc.create_llm_span(...)` using placeholder-substitution to emit redacted prompt/completion attributes. See `redaction.md` for the substitution pattern.

The privacy default lives in code via `os.environ.setdefault(...)` rather than a comment because the instrumentors only honor an actual env var; a comment alone changes nothing at runtime.

```python
import os

# Stricter-customer variant: suppress prompt/completion content capture in the
# auto-instrumentors. Must be set before any opentelemetry.instrumentation.*
# import — the instrumentors read TRACELOOP_TRACE_CONTENT at span-emit time.
os.environ.setdefault("TRACELOOP_TRACE_CONTENT", "false")

import montecarlo_opentelemetry as mc
from opentelemetry.instrumentation.langchain import LangchainInstrumentor

otel_endpoint = os.getenv("OTEL_ENDPOINT")
if otel_endpoint:
    base_endpoint = otel_endpoint.rstrip("/")
    http_otel_endpoint = (
        base_endpoint
        if base_endpoint.endswith("/v1/traces")
        else f"{base_endpoint}/v1/traces"
    )

    mc.setup(
        agent_name="ai-agent",
        otlp_endpoint=http_otel_endpoint,
        instrumentors=[LangchainInstrumentor()],
    )
```

`os.environ.setdefault` preserves an explicit operator override (`TRACELOOP_TRACE_CONTENT=true`) while defaulting to off when unset.

> **CRITICAL — set `TRACELOOP_TRACE_CONTENT` at module scope, NOT inside `init_tracing()`.** The OpenLLMetry instrumentors read this env var when their package is *imported* (the `from opentelemetry.instrumentation.langchain import LangchainInstrumentor` line at the top of every serverless template). By the time `init_tracing()` runs the import has already happened and a `setdefault` call inside the function is a no-op. The splice point is **between `import os` and any `opentelemetry.instrumentation.*` import**.

#### Serverless splice — concrete example

For any of the Section 1 serverless templates (2, 3, or 4 — they share the same import layout), the patch is a single block inserted between `import os` and the first instrumentation import. Below is Template 2 with the splice applied; Templates 3 and 4 follow the same shape.

```python
import logging
import os

# Stricter-customer variant: suppress prompt/completion content capture in the
# auto-instrumentors. Must be set before any opentelemetry.instrumentation.*
# import — the instrumentors read TRACELOOP_TRACE_CONTENT at import time, so
# setting it inside init_tracing() below would be a no-op.
os.environ.setdefault("TRACELOOP_TRACE_CONTENT", "false")

import montecarlo_opentelemetry as mc
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.langchain import LangchainInstrumentor
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

AGENT_NAME = "monitoring-agent"


def init_tracing():
    # ... unchanged from Template 2 ...
    pass
```

The body of `init_tracing()` is identical to the unredacted Template 2 — the only difference is the three-line splice above the instrumentation imports.

> **IMPORTANT — `TRACELOOP_TRACE_CONTENT=false` is a prerequisite for any redaction under auto-instrumentation.** Manual `mc.create_llm_span` calls do not unwire the auto-instrumentor; if content capture is still on, the raw prompt/completion will be emitted alongside the redacted version. Set `TRACELOOP_TRACE_CONTENT=false` first, then layer manual spans on top if needed.

---

## 4. Env vars (only on the MC-hosted path)

Branch on the answer to workflow step #2 (collector source):

### MC-hosted collector

The customer needs auth credentials for the MC ingest endpoint. Either:

- `MCD_DEFAULT_API_ID` and `MCD_DEFAULT_API_TOKEN` (preferred), **or**
- `OTEL_EXPORTER_OTLP_HEADERS=x-mcd-id=...,x-mcd-token=...`

Confirm presence with a presence-only check:

```python
import os

assert os.environ.get("MCD_DEFAULT_API_TOKEN"), (
    "MCD_DEFAULT_API_TOKEN is not set. Configure it before running the agent."
)
```

> **CRITICAL — never log, echo, or include the value of `MCD_DEFAULT_API_TOKEN` or `OTEL_EXPORTER_OTLP_HEADERS` in any tool argument, diff, or transcript.** These are long-lived credentials. Editor transcripts get pasted into Slack and bug reports — a single `print(os.environ["MCD_DEFAULT_API_TOKEN"])` in a verify step would leak the token broadly without anyone realizing it. Use **only** presence-only checks (`bool(os.environ.get(...))`). Never read the value, never include it in a `Bash` command echo, never paste it into a confirmation message.

### Self-hosted collector

Auth is handled at the customer's collector — MC does not see the credentials. **Skip** the `MCD_*` prompt entirely. Do not generate env-var setup code, do not ask the customer for tokens, do not reference `MCD_DEFAULT_API_TOKEN` in the template.

---

## 5. Existing-`mc.setup()` decision matrix

When `scripts/detect_libraries.py` returns `existing_setup.found: true`, walk this decision tree. **Do not auto-scaffold a second `mc.setup()`** — the customer already has one, and silently adding another will create two agents with confusing telemetry.

| Scenario | What the skill does |
|---|---|
| Different `agent_name` (existing != intended) | Ask the user to confirm intent. Different names create different agents in MC. If the user wants a fresh one, propose **adding** the new `mc.setup()` next to the existing one (with explicit per-file approval). |
| Different `instrumentors` list | Propose merging the lists as a diff to the existing `mc.setup()` call. Wait for approval. Don't auto-merge. |
| Using `BatchSpanProcessor` but the runtime is now serverless (e.g. customer migrated to Lambda) | Propose switching to `SimpleSpanProcessor` (with `OTLPSpanExporter`) as a diff. Cite the matching serverless template from Section 1. Wait for approval. |
| Identical (same `agent_name`, same `instrumentors`, correct `span_processor` for the runtime) | No-op. Tell the user the existing setup is already correct and exit cleanly. |

> **IMPORTANT — read the existing `mc.setup()` source carefully before proposing a diff.** Do not assume the structure; the customer may have customizations the skill should preserve (custom resource attributes, conditional setup, env-var handling, logging hooks). The decision matrix above is the minimum — preserving customizations is also required.

---

## 6. CRITICAL — never edit any file without explicit user approval

This rule from `SKILL.md` applies to every file this reference covers:

> **CRITICAL — Never modify the customer's code without explicit per-file user approval.** Always propose the diff and wait for confirmation before writing the file. The skill is not a code generator that runs autonomously — it's an assistant that proposes changes for the customer to accept or reject.

This rule covers, at minimum:

- Source files where `mc.setup()` lands (the file the workflow proposes editing).
- Dependency files (`requirements.txt`, `pyproject.toml`, `Pipfile`, etc. — handled in `library-detection.md`).
- Env files (`.env`, `.env.example`, deployment manifests).

Surface every diff. Wait for `yes` per file. Never batch-approve across files.

---

## Common mistakes

- **Generating the default `BatchSpanProcessor` template for a Lambda agent.** Silent trace loss. Always check `runtime` first and pick a serverless template from Section 1.
- **Mixing-and-matching between Section 1 templates** (e.g., taking the `MCD_DEFAULT_*` block's `headers=` line and pasting it into the self-hosted template). Each template is self-contained — pick one and use it as-is.
- **Double-appending `/v1/traces`** (e.g. `https://collector/v1/traces/v1/traces`). Broken endpoint. Normalize idempotently — check the suffix before appending.
- **Forgetting to render the resolved final endpoint to the user** before generating code. Opaque magic. Always show the resolved URL and wait for confirmation.
- **Layering manual redaction on top of an auto-instrumentor without setting `TRACELOOP_TRACE_CONTENT=false`.** The raw content still gets emitted by the auto-instrumentor alongside the redacted version. The env var is a prerequisite for any redaction under auto-instrumentation — see `redaction.md`.
- **Reading or echoing `MCD_DEFAULT_API_TOKEN` to confirm it's set.** Credential leak. Use presence-only (`bool(os.environ.get(...))`).
- **Auto-scaffolding a duplicate `mc.setup()` when one already exists.** Confusing telemetry, two agents in MC. Walk the decision matrix instead.
- **Editing `requirements.txt` / `pyproject.toml` / source files without explicit per-file approval.** Violates the SKILL.md guardrail. Propose every diff, wait for `yes` per file.
- **Forgetting that `mc.setup()` does not auto-inject auth headers when `span_processor=` is set.** With the default exporter, `mc.setup()` injects `MCD_DEFAULT_*` from env vars automatically. With a custom `span_processor` the customer constructs the `OTLPSpanExporter`, so the auth path must be made explicit at exporter-construction time. Match the customer's setup: (a) MC-hosted with `MCD_DEFAULT_*` env vars → pass `headers={"x-mcd-id": ..., "x-mcd-token": ...}` to `OTLPSpanExporter`; (b) MC-hosted with `OTEL_EXPORTER_OTLP_HEADERS` → omit `headers=` (the exporter reads the env var); (c) self-hosted collector → omit `headers=` (auth is at the collector). Symptom of getting it wrong: traces emit but never appear in MC because the collector rejects them as unauthenticated, **or** `init_tracing()` raises `KeyError` at startup because the template references `MCD_DEFAULT_*` that the customer isn't using.

# Troubleshooting — diagnosing why traces aren't flowing

Tier 3 reference for the `instrument-agent` skill. Use this when verification has failed and traces aren't reaching the agent metadata endpoint. Walk the failure modes in priority order — order matters.

## 1. When to read this file

The workflow's verification step (step #10) calls `get_agent_metadata` and compares to the BEFORE snapshot. If the new agent doesn't appear (or appears but with no spans), branch here.

Walk the failure modes in priority order — the order matters because some are more common than others, and some have cheaper diagnostics. Resolve one cause at a time; don't change five things at once and re-test.

## 2. Diagnostic priority order

Given the symptom "traces aren't appearing in `get_agent_metadata`," check in this order:

| # | Failure mode | Cheap signal to look for first |
|---|---|---|
| 1 | Serverless `BatchSpanProcessor` foot-gun | Did `detect_libraries.py` flag `runtime: serverless`? Did the customer's `mc.setup()` use the `SimpleSpanProcessor` variant? |
| 2 | SDK init not running | Is `mc.setup()` actually called at agent startup? Or is it defined in a module that's never imported? |
| 3 | Missing credentials | MC-hosted collector path: are the selected auth env vars set in the runtime env (`MCD_DEFAULT_API_ID` / `MCD_DEFAULT_API_TOKEN` or `OTEL_EXPORTER_OTLP_HEADERS`)? |
| 4 | Wrong instrumentor versions | Did `pip install` succeed without resolver complaints? Are the installed versions compatible with the SDK? |
| 5 | Upstream pipeline not deployed | Did the customer's MC AO setup actually finish? Has anyone confirmed the collector endpoint accepts traffic? |

Most "no traces showing up" reports turn out to be #1 (serverless) or #2 (init not running). Walk through each in order.

## 3. Failure mode #1 — Serverless `BatchSpanProcessor` foot-gun

> **CRITICAL — incomplete or missing traces on Lambda are usually the `BatchSpanProcessor` foot-gun.** Lambda freezes the process between invocations; the default span processor is suspended before flushing, and the spans never leave the function. The fix is `SimpleSpanProcessor`.

This applies to any suspendable runtime — AWS Lambda, Google Cloud Functions, Vercel Functions, Cloudflare Workers, Azure Functions. Anywhere the process can be frozen mid-batch and resumed later (or never).

### Diagnostic

- Was `runtime: "serverless"` reported by `detect_libraries.py`?
- Does the customer's `mc.setup()` call include `span_processor=SimpleSpanProcessor(OTLPSpanExporter(endpoint=...))`?
- If serverless was detected but the customer used the default template (no `span_processor` kwarg), this is the bug.

### Fix

Propose a small diff switching to the serverless template from `setup-template.md`. Wait for per-file approval before applying.

## 4. Failure mode #2 — SDK init not running

The setup code exists in the codebase but is never executed at runtime. Common bug: `mc.setup()` lives in a module that nobody imports.

### Diagnostic

- Is `mc.setup()` defined in a module that's actually imported at agent startup? (Common bug: `mc.setup()` lives in `tracing.py` but `tracing.py` is never imported.)
- Is the import path correct? `import montecarlo_opentelemetry as mc` should not raise `ModuleNotFoundError`.
- Is `mc.setup()` called at the top level of the module (or in an `init_tracing()` function that's actually invoked)?
- If wrapped in a guard like `if otel_endpoint:`, is `OTEL_ENDPOINT` set? Print **presence** (NOT value) to confirm.

### Fix

- Add the missing import in the entry-point file.
- Or call `init_tracing()` explicitly at agent startup.
- Or set the missing env var.

Each fix is a per-file diff that needs approval.

> **IMPORTANT — never `print(os.environ["MCD_DEFAULT_API_TOKEN"])` to debug "is the value set."** Use `bool(os.environ.get(...))` or `"set" if os.environ.get(...) else "missing"`. Echoing token values into the agent's logs is a credential leak. Same for `OTEL_EXPORTER_OTLP_HEADERS`.

## 5. Failure mode #3 — Missing credentials (MC-hosted collector path only)

Only applies if the customer is using the MC-hosted collector (`https://integrations.getmontecarlo.com/otel`). Self-hosted collectors handle auth at the collector — skip this section in that branch.

### Diagnostic

- Is the customer using `MCD_DEFAULT_API_ID` / `MCD_DEFAULT_API_TOKEN`, and are both set in the runtime env (Lambda env vars, container env, dev shell, etc.)? Use presence-only checks.
- Or is the customer using `OTEL_EXPORTER_OTLP_HEADERS=x-mcd-id=...,x-mcd-token=...`, and is that env var set?
- Are the values current (not rotated)?
- **If the customer is on the serverless template (custom `span_processor`), is the auth path made explicit at exporter-construction time?** `mc.setup()` only auto-injects auth headers when it builds the default exporter; with a custom `span_processor` the customer constructs the `OTLPSpanExporter`, so the auth path must be picked explicitly. Three valid shapes: (a) `MCD_DEFAULT_*` env vars + `OTLPSpanExporter(endpoint=..., headers={"x-mcd-id": ..., "x-mcd-token": ...})`; (b) `OTEL_EXPORTER_OTLP_HEADERS` env var + `OTLPSpanExporter(endpoint=...)` with no explicit `headers=` (the exporter reads the env var); (c) self-hosted collector + `OTLPSpanExporter(endpoint=...)` with no headers (auth at the collector). If none of those match, env vars may exist but never reach the wire — symptom looks like missing credentials but the actual bug is the exporter is unauthenticated. See the SDK docs on PyPI (https://pypi.org/project/montecarlo-opentelemetry/) for the current auth-header guidance.

### Fix

- Set the missing auth env vars in the runtime (`MCD_DEFAULT_API_ID` / `MCD_DEFAULT_API_TOKEN`, or `OTEL_EXPORTER_OTLP_HEADERS` if the customer uses the standard OTel header path).
- For Lambda, that's the function's environment configuration (or, better, AWS Secrets Manager if the customer has a rotation policy).
- For containers, the deployment manifest.

> **NEVER include the actual token value in any diff, transcript, or log.** Walk the customer through setting the env var; don't echo it back.

## 6. Failure mode #4 — Wrong instrumentor versions

The instrumentor package version is incompatible with the SDK version, or with the AI library version it's instrumenting.

### Diagnostic

- What instrumentor version did `pip install` resolve? `pip show opentelemetry-instrumentation-<library>` or `pip freeze | grep instrumentation`.
- What does the live PyPI/SDK README compatibility table say? Run `python3 scripts/fetch_sdk_docs.py` and check the `supported_instrumentors` list — the `version_constraint` there is the current upper bound.
- Is the AI library itself a recent major version that breaks the instrumentor (e.g., LangChain 0.3 with an instrumentor pinned to LangChain 0.1.x)?

### Fix

- Re-run `pip install` with the current constraint from PyPI (via `python3 scripts/fetch_sdk_docs.py`).
- If the instrumentor doesn't yet support the AI library version, propose pinning the AI library to a compatible version, OR consult the instrumentor's PyPI page for upcoming compatibility.

> **CRITICAL — never edit `requirements.txt` / `pyproject.toml` / `Pipfile` without explicit per-file approval.** If a version pin needs to change, propose the diff and wait. See `SKILL.md`.

### Symptom: `TypeError: wrap_function_wrapper() got an unexpected keyword argument 'module'` at `mc.setup()` import

This is a transitive-dep collision between the OpenLLMetry instrumentors (langchain, openai, anthropic, bedrock, crewai, sagemaker, vertexai at `<=0.53.4`) and `wrapt` 2.x. The instrumentors call `wrap_function_wrapper(module=...)`; `wrapt` 2.x renamed that argument to `target=`. PyPI doesn't expose this transitive constraint, so a fresh `pip install opentelemetry-instrumentation-langchain` can resolve `wrapt` to 2.x and crash on first import.

**Fix:** pin `wrapt<2` alongside the OpenLLMetry instrumentor(s) in the customer's dependency file, then reinstall. For example:

```diff
 opentelemetry-instrumentation-langchain<=0.53.4
+wrapt<2
```

Then `pip install -r requirements.txt` (or the pyproject/Pipfile equivalent). The skill must propose this as a diff and wait for per-file approval — never edit dependency files autonomously.

## 7. Failure mode #5 — Upstream pipeline not deployed

The customer's MC AO infrastructure (collector, ingestion endpoint, workspace) isn't online or hasn't been provisioned. The agent code is correct but there's nothing on the other end.

### Symptoms

- All four other failure modes ruled out.
- The OTLP endpoint URL appears correct, env vars are set, code runs.
- `get_agent_metadata` still returns the old list with no new entries after running the agent multiple times.

### Diagnostic

- Has the customer actually completed their MC AO setup? The customer should have:
  - An MC AO workspace provisioned.
  - An ingestion endpoint configured (either MC-hosted or self-hosted collector reachable from the customer's runtime).
  - Outbound network access from the agent's runtime to the OTLP endpoint.
- Try `curl -v <otlp-endpoint>` from the agent's runtime — does the collector accept the connection?
- Does the customer see the workspace in `https://getmontecarlo.com/dashboard`?

### Fix

Customer needs to coordinate with their MC AO setup team. This is **out of scope for the instrument-agent skill** — it's a setup/infra concern. Point them at AO-product onboarding docs and exit cleanly. Don't try to fix infra from the skill.

## 8. Putting it together — the diagnostic loop

When the verification step shows the new agent isn't appearing:

1. Ask the user: "Is the agent runtime serverless (Lambda, Cloud Functions, Vercel, etc.)?" If yes, check #1 first.
2. Confirm `mc.setup()` is actually executed (#2).
3. If MC-hosted, confirm the env vars (#3).
4. Run `pip show` / `fetch_sdk_docs.py` to compare versions (#4).
5. If all four are clean, escalate to upstream pipeline (#5) — that's a setup-infra concern outside this skill's scope.

Walk through them one at a time, not all at once. Each step has a cheap diagnostic that either confirms or rules out the cause.

## Common mistakes

- **Jumping to credential issues first when the symptom is missing traces on Lambda** — usually it's the `SimpleSpanProcessor` foot-gun. Check runtime classification before chasing env vars.
- **Echoing env var values to "confirm" they're set** — credential leak. Presence-only checks (`bool(os.environ.get(...))`) only.
- **Editing `requirements.txt` to bump versions without per-file approval** — violates `SKILL.md` guardrail. Propose the diff; wait.
- **Trying to fix upstream pipeline issues from the skill** — out of scope. Escalate to the customer's AO setup team.
- **Polling `get_agent_metadata` while waiting for traces** — wasteful. Let the customer drive the cadence; don't loop on the metadata endpoint.
- **Changing multiple things at once and re-testing** — you lose the signal about which fix actually mattered. One change, one re-test.

# Verify Traces

How to confirm that an instrumented agent is actually emitting traces to Monte Carlo. This is the verification step at the end of the instrument-agent workflow.

The skill calls `get_agent_metadata` exactly **twice** per instrumentation flow — once to snapshot existing agents before any code changes, and once after the customer runs the instrumented agent. New traces are confirmed by diffing the two snapshots on `(agentName, traceTableMcon)`.

---

## 1. Pre-flight: `test_connection`

`test_connection` is the Step 0 pre-flight check — it runs once at the very start of the workflow, before any intake questions. Its job is to record whether MCP is available so Steps 4 and 10 know which verification path to take.

- **MCP available** — Step 4 captures a BEFORE snapshot via `get_agent_metadata` and Step 10 diffs against it. This is the canonical path.
- **MCP unavailable** — **degrade gracefully, don't exit.** Tell the user the Monte Carlo MCP server isn't reachable, link them to https://docs.getmontecarlo.com/docs/mcp-server for setup, and continue. Step 4 skips the BEFORE snapshot. Step 10 hands the customer off to verify the new agent appears in the Monte Carlo UI manually after they run the instrumented agent. Instrumentation can still proceed; only the in-skill verification step changes.

If MCP was reported available in Step 0 but a subsequent `get_agent_metadata` call fails (e.g. the session expired mid-workflow), re-run `test_connection`; if it still fails, flip `mcp_available = false` and proceed under the manual-UI verification path.

---

## 2. The before/after pattern

### BEFORE snapshot (workflow step #4)

Before any edits to the customer's code, call `get_agent_metadata` and save the full list of `(agentName, traceTableMcon)` pairs. This is the baseline.

The response shape:

```json
[
  {"agentName": "customer-support", "traceTableMcon": "MCON://...", "sourceType": "TRACE_TABLE", "backend_class": "customer_otel_trace_table"},
  {"agentName": "monitoring-agent", "traceTableMcon": "MCON://...", "sourceType": "PLATFORM_AGENT", "backend_class": "platform_agent"}
]
```

Each MCON is unique per ingestion source — it is the true identity of the trace stream. `agentName` is **not** unique on its own.

### AFTER snapshot (workflow step #10)

After the customer has approved the `mc.setup()` and decorator diffs **and** has run the instrumented agent end-to-end at least once, call `get_agent_metadata` again and diff against the BEFORE snapshot.

---

## 3. Why MCON, not name, is the identity

> **IMPORTANT — when the same `agent_name` reappears in the AFTER snapshot, compare MCONs.**

Common gotcha: a customer instruments the agent in dev, then later instruments the *same* agent in prod. Both report `agent_name="customer-support"` to MC. Comparing only on `agentName`, the AFTER snapshot looks identical to the BEFORE snapshot ("customer-support is still there"), and the skill would falsely conclude the prod instrumentation worked when it actually didn't.

A genuinely new agent has a **new MCON** not in the BEFORE list. If the MCON is unchanged from the BEFORE snapshot, no new traces have arrived — branch to `troubleshooting.md`.

---

## 4. Prompting the customer between snapshots

Verification is gated on the customer running the instrumented agent at least once. After the BEFORE snapshot and after they've approved the diffs, prompt them:

> "I've snapshotted your existing agents in Monte Carlo. Run your instrumented agent end-to-end against your environment (your dev or staging stack) at least once, then tell me when it's done. I'll re-check `get_agent_metadata` and confirm the new agent appears."

Then **wait for the customer to confirm** they ran it. Don't loop. Let them work and ping the skill when ready.

---

## 5. Don't poll

> **NEVER poll `get_agent_metadata` in a loop.** The skill calls it twice — once before edits, once after the user reports running the agent. Polling burns API quota and adds nothing. The trigger is the customer running the agent, not the passage of time. First-time visibility for low-traffic or dev agents can take 10 minutes or more.

If the customer says "I ran it but I don't see it yet," wait a couple of minutes and ask them to retry the check. If after ~10–15 minutes the new agent still isn't visible, branch to `troubleshooting.md`.

---

## 6. The AFTER call — what to check

When the user reports they've run the instrumented agent:

1. Call `get_agent_metadata`.
2. Filter for **new entries** — any `(agentName, traceTableMcon)` not in the BEFORE snapshot.
3. Look for an `agentName` matching what the customer put in `mc.setup(agent_name=...)`.
4. Confirm the MCON is genuinely new (not present in BEFORE).

### Success path

- New entry with the customer's chosen `agent_name` **and** a new MCON → traces are flowing. Tell the customer the instrumentation is verified, and recommend the `monte-carlo-monitoring-advisor` skill for setting up monitors.

### Failure paths

- **No new entries at all** → the instrumented agent hasn't sent any spans. Branch to `troubleshooting.md`.
- **New entry exists but with a different `agent_name` than expected** → likely a typo in `mc.setup()` or two `mc.setup()` calls in the codebase. Walk the customer through the decision matrix in `setup-template.md`.
- **New entry's MCON matches a BEFORE entry** → not actually new. Branch to `troubleshooting.md`.
- **Customer is on a serverless runtime (Lambda, Cloud Run, etc.) and traces aren't appearing** → highly likely the `SimpleSpanProcessor` is missing. Branch to `troubleshooting.md` with that hypothesis first.

---

## 7. Timing expectations

After the customer's agent runs and emits OTLP spans:

- A new `agentName` typically appears in `get_agent_metadata` within a few minutes. First-time visibility for low-traffic dev agents, or for agents emitting a small number of spans, can take 10 minutes or more — be patient, especially on a customer's first instrumentation pass.
- If after ~10–15 minutes the new agent still isn't visible, something is wrong (SDK init not running, wrong endpoint, missing credentials, batch processor suspended on Lambda, etc.). Branch to `troubleshooting.md`.

### Optional: local verification with a desktop OTLP receiver

When the customer wants to confirm the instrumentation produces valid OTLP spans *before* pointing at MC's collector — for example, while iterating in dev — they can run a local OTLP receiver and temporarily set `OTEL_ENDPOINT` to it. [`otel-desktop-viewer`](https://github.com/CtrlSpice/otel-desktop-viewer) is a single-binary receiver with a browser UI that makes the trace tree easy to inspect (the Docker image listens on `4317` for gRPC, `4318` for HTTP, and serves the UI on `8000`).

This is **not** a substitute for the MC-side `get_agent_metadata` check — only the latter proves the trace reached Monte Carlo. But it is useful for:

- Confirming the wiring (`@trace_with_workflow` produces a root, `@trace_with_task` nests under it, the auto-instrumentor's LLM spans land where expected).
- Distinguishing "traces never emitted" from "traces emitted but dropped in transit" when troubleshooting Step 10 failures.

Note: the published Docker image's JSON-RPC API may lag the repo's `main` branch — its method names tend to be stable across releases, but new methods may not be available yet on `latest-arm64` / `latest-amd64`.

---

## Common mistakes

- **Calling `get_agent_metadata` only once** (after the edits). Without the BEFORE snapshot, you can't tell new from existing. Wrong.
- **Comparing only `agentName`, not MCON.** Misses the dev/prod twin case where the same name already exists. Wrong.
- **Polling `get_agent_metadata` in a loop while waiting.** Wasteful and unnecessary — the customer running the agent is the trigger.
- **Skipping the `test_connection` pre-flight.** Verification fails silently if MCP is misconfigured, and the customer ships uninstrumented or unverified code.
- **Concluding "instrumentation works" without running the agent and re-checking.** Premature — code changes alone prove nothing.
- **Assuming a Lambda customer's missing trace is a credential issue.** Usually it's the `SimpleSpanProcessor` foot-gun. Check serverless first.

# Workflow

End-to-end procedure for instrumenting a customer's Python AI agent with Monte Carlo Agent Observability. Read top-to-bottom — each step gates the next. The output of this workflow is traces that the `monitoring-advisor` skill later consumes.

> **CRITICAL — never modify any file without explicit user approval.** This skill proposes diffs; the user accepts them. That includes dependency files (`requirements.txt`, `pyproject.toml`, `Pipfile`), application source (where `mc.setup()` and decorators land), and anything else on disk. If the user says "go ahead and apply it," that's approval for that specific diff and nothing more. Ask again for the next file.

The workflow has a pre-flight check followed by eleven steps, in order:

0. Pre-flight — confirm MCP connectivity via `test_connection`
1. Detect libraries, runtime, and existing setup
2. Ask about the OTel collector (MC-hosted vs. self-hosted)
3. Ask whether stricter privacy requirements warrant redaction (default is full capture)
4. Snapshot existing agents via `get_agent_metadata` (BEFORE changes)
5. Resolve and confirm the final OTLP endpoint
6. Propose dependency-file edits
7. Propose `mc.setup()` insertion
8. Propose `@trace_with_workflow` / `@trace_with_task` decorator diffs
9. Confirm env vars (presence-only)
10. Verify via `get_agent_metadata` (AFTER user runs the agent)
11. On failure, branch to `troubleshooting.md`

---

## Step 0 — Pre-flight: confirm MCP connectivity

Before beginning the workflow, confirm the Monte Carlo MCP server is configured and authenticated by calling `test_connection`.

- **If `test_connection` succeeds** — proceed to Step 1. Record that MCP is available; Steps 4 and 10 will call `get_agent_metadata` without re-checking.
- **If `test_connection` fails** — **degrade gracefully**, don't exit. Tell the user that the Monte Carlo MCP server isn't available, point them at https://docs.getmontecarlo.com/docs/mcp-server as informational, and continue the workflow. Explain that they'll need to verify the new agent appears in the Monte Carlo UI manually after running the instrumented agent (Step 4 will skip the BEFORE snapshot and Step 10 will give them manual-UI verification instructions). Record `mcp_available = false` so Steps 4 and 10 know which path to take.

Do this check once, up front, so the user discovers a MCP problem immediately — not after three turns of intake questions.

---

## Step 1 — Detect libraries, runtime, and existing setup

Run the detection helper against the customer's agent code:

```bash
python3 scripts/detect_libraries.py <target_path>
```

It prints a JSON object with the following fields:

- `dependencies` — sorted list of normalized pip package names parsed from `requirements.txt` / `pyproject.toml` / `Pipfile`. Raw surface; the script does not single out AI libraries. The LLM matches these against `fetch_sdk_docs.py`'s `supported_instrumentors` list (see below).
- `runtime` — `serverless`, `long_running`, or `unknown`. `serverless` if any serverless signal is found; `long_running` if a dep manifest was found but no serverless signals; `unknown` when no dep manifest exists at all.
- `serverless_signals` — what triggered a serverless classification (e.g. `lambda_handler`, `serverless.yml`, `mangum`)
- `existing_setup` — `{ found: bool, files: list[str] }` for any pre-existing `mc.setup()` call. The `files` array contains repo-relative paths where `montecarlo_opentelemetry` was detected.

Match `dependencies` against the live PyPI supported-instrumentor list to figure out which instrumentors to install. See `library-detection.md` for the matching rules — including the ambiguous-multipurpose-SDK case (`boto3`, `google-cloud-aiplatform`, etc.) where the LLM must ask the customer before installing.

Parse this output and branch:

- **`existing_setup.found` is `true`** — do not propose a fresh `mc.setup()`. Inspect the paths listed in `existing_setup.files` to understand what already exists. Point the reader at the existing-setup decision matrix in `setup-template.md` to decide whether to keep, reconfigure, or replace the call. Then continue with the rest of the workflow (the user may still need decorator and dependency changes).

### Known limitations

`existing_setup` detection parses Python imports and setup calls. It recognizes `import montecarlo_opentelemetry`, aliases such as `import montecarlo_opentelemetry as mco`, and direct imports such as `from montecarlo_opentelemetry import setup as setup_mc`, but only when the imported module/name is actually called. Malformed Python files fall back to a narrower text check, so if the customer reports an existing setup that was missed, inspect those files manually before proposing a new `mc.setup()`.

### Match in real code, not docs or comments

The same principle that governs `existing_setup` detection applies to every match-scanning step in this workflow — library-import detection, decorator-candidate identification, and existing-`mc.setup()` lookup. Before treating a match as actionable, confirm it lives in executable Python code, not in a docstring, an inline comment, an example block in a Markdown file, or test fixture data. A match inside a `"""..."""` doc block or a `README.md` example is not a real usage.

- **`runtime: "unknown"` and `dependencies: []`** — exit cleanly. No dependency manifest was found in the target tree, so there's nothing to scan. Tell the user: "I didn't find a `requirements.txt`, `pyproject.toml`, or `Pipfile` in the target. Confirm the agent code is actually in this path, then re-run." Do not scaffold anything.
- **`dependencies` non-empty but no PyPI-supported AI library matches** — exit cleanly per `library-detection.md` section 7. Don't scaffold an `mc.setup()` against an empty instrumentor list unless the customer is manually reporting every LLM call with `mc.create_llm_span`.
- **Anything else** — continue to step 2 with the detection output in hand.

Always run `python3 scripts/fetch_sdk_docs.py` alongside `detect_libraries.py`. It pulls the live `supported_instrumentors` list from PyPI — that's the canonical source for which AI libraries the SDK currently supports. The script fails closed if PyPI is unreachable; if it errors, point the user at `https://pypi.org/project/montecarlo-opentelemetry/` directly and ask them to share the current supported list manually. Match the customer's `dependencies` against `supported_instrumentors` to decide which instrumentors to install.

**Next:** with detection settled, ask about the collector.

---

## Step 2 — Ask about the OTel collector

Ask the user verbatim:

> "Are you using your own OTel collector or the MC-hosted one?"

Capture the answer — it gates step 5 (endpoint normalization) and step 9 (env-var checks).

- **MC-hosted** — base URL is `https://integrations.getmontecarlo.com/otel`. Step 9 will require either `MCD_DEFAULT_API_ID` / `MCD_DEFAULT_API_TOKEN` or `OTEL_EXPORTER_OTLP_HEADERS`, depending on the setup template.
- **Self-hosted** — ask: "What's the base URL for your collector?" Capture it as the customer's collector base URL. Step 9 will skip MC credential checks because auth happens at the customer's collector.

Don't try to infer the collector from anything in the codebase — just ask.

**Next:** ask whether the customer has stricter privacy requirements that warrant redaction, so step 3 picks the right `mc.setup()` template.

---

## Step 3 — Ask whether redaction is required

The Monte Carlo OpenTelemetry SDK's value proposition is auto-instrumentation that captures prompts and completions by default. Trace content lives in the customer's environment; the MC-hosted collector is a write-back pass-through with no MC-side persistence of trace content. Full capture is therefore the canonical path, and redaction is opt-in for customers with stricter requirements.

Ask the user verbatim:

> "Do you have stricter requirements (compliance, contractual, or company policy) that would require redacting prompts or completions in traces?"

This is a non-optional gating decision that runs **before** any `mc.setup()` is generated.

- **Yes** — route the user to `redaction.md`. Under redaction, `TRACELOOP_TRACE_CONTENT=false` is **mandatory** when an auto-instrumentor is in use (else the instrumentor emits duplicate-content spans alongside any manual redacted spans). The prompts-disabled `mc.setup()` template in step 7 sets this in code. Customers who want partial capture with placeholder substitution can layer manual `mc.create_llm_span` calls on top — that's an optional additional layer, not a replacement.
- **No** — use the default `mc.setup()` template in step 7. The default leaves auto-instrumentor capture on; prompts and completions flow into the customer's environment with no extra wiring.

**Next:** snapshot existing agents before any code changes land.

---

## Step 4 — Snapshot existing agents via `get_agent_metadata` (BEFORE changes)

This must run **before** step 6, 7, or 8 propose any diffs. The snapshot is what step 10 compares against to prove the new instrumentation actually produced traces.

Branch on the MCP availability flag recorded in Step 0:

- **MCP available** — call `get_agent_metadata`. Save the list of `(agent_name, mcon)` pairs and hold onto the snapshot; step 10 diffs against it.
- **MCP unavailable** — skip the BEFORE snapshot. Tell the customer that without MCP this skill can't capture a baseline, so step 10 will hand them off to verify the new agent in the Monte Carlo UI manually. Continue the workflow.

If MCP was reported available in Step 0 but the `get_agent_metadata` call now fails (e.g. the session expired), re-run `test_connection`. If it still fails, flip `mcp_available = false` and proceed under the manual-UI verification path described above.

See `verify-traces.md` for the full before/after flow and what the response looks like.

**Next:** resolve the OTLP endpoint URL and get user confirmation before generating any code.

---

## Step 5 — Resolve and display the final OTLP endpoint

The endpoint is whatever base URL came out of step 2, normalized to end in `/v1/traces`.

- If the user's URL already ends in `/v1/traces`, use it as-is.
- Otherwise, append `/v1/traces`.
- **Never double-append.** A URL that already ends in `/v1/traces` must not become `…/v1/traces/v1/traces`.

Examples:

| Input                                              | Resolved                                                       |
| -------------------------------------------------- | -------------------------------------------------------------- |
| `https://integrations.getmontecarlo.com/otel`      | `https://integrations.getmontecarlo.com/otel/v1/traces`        |
| `https://integrations.getmontecarlo.com/otel/v1/traces` | `https://integrations.getmontecarlo.com/otel/v1/traces`    |
| `https://collector.example.com:4318`               | `https://collector.example.com:4318/v1/traces`                 |

Render the resolved final URL to the user and ask for confirmation before generating any code. See `setup-template.md` for the full normalization rules.

**Next:** propose dependency edits using the install set from step 1.

---

## Step 6 — Propose dependency-file edits

Determine the install set by matching `detect_libraries.py`'s `dependencies` against `fetch_sdk_docs.py`'s `supported_instrumentors` per `library-detection.md`. Always include the MC SDK package itself.

**Pinning is required, not optional.** Each `supported_instrumentors` entry from `fetch_sdk_docs.py` may include a `version_constraint` (e.g. `<=0.53.4`) parsed from the PyPI README's `pip install` line. Apply that constraint directly in the proposed diff — never strip it. If `fetch_sdk_docs.py` failed (PyPI unreachable) it exits with an error rather than substituting stale data; point the user at `https://pypi.org/project/montecarlo-opentelemetry/` to resolve pins manually.

Some instrumentors have transitive constraints PyPI doesn't expose. The most common today is `wrapt<2`, required alongside the OpenLLMetry instrumentors. The skill **does not** preemptively bake that pin into every install diff — it's surfaced as a symptom-driven fix in `troubleshooting.md` (the customer hits a `TypeError: wrap_function_wrapper() got an unexpected keyword argument 'module'` and the troubleshooting reference names the pin). If the customer reports that error after installing, route them to that section.

Propose the additions as a unified diff against the customer's actual dependency file — `requirements.txt`, `pyproject.toml`, or `Pipfile`. Wait for **explicit per-file approval** before any edit lands.

> **CRITICAL — never edit dependency files autonomously.** Even if the change looks trivial. The user reviews and accepts each diff. See `library-detection.md` for the install rules.

If any of the customer's `dependencies` is ambiguous (e.g. `boto3` could mean Bedrock, SageMaker, or generic AWS; `google-cloud-aiplatform` could be Vertex inference or Vertex Search), surface the candidates and ask the user before deciding what to install. Don't guess. See `library-detection.md` section 4.

**Next:** propose the `mc.setup()` insertion.

---

## Step 7 — Propose `mc.setup()` insertion as a diff

Use the runtime classification from step 1 to pick the template:

- **`runtime: "serverless"`** — use the serverless template, which uses `SimpleSpanProcessor` so spans flush before the Lambda freeze. See `setup-template.md` for the canonical template. The serverless `BatchSpanProcessor` foot-gun is covered in `troubleshooting.md`.
- **`runtime: "long_running"`** — use the default template in `setup-template.md`. `BatchSpanProcessor` is appropriate here.
- **`runtime: "unknown"`** — by step 7 you should never be here; step 1 would have exited cleanly. If you somehow are, ask the user to classify before proposing a template.

If the customer opted into redaction in step 3, use the prompts-disabled variant of the chosen template — it sets `TRACELOOP_TRACE_CONTENT=false` in code, which is mandatory under redaction to prevent auto-instrumentors from emitting duplicate-content spans. If the customer did not opt into redaction, use the default template, which leaves auto-instrumentor capture on.

If step 1 reported `existing_setup.found: true`, don't propose a fresh insertion — apply the decision from `setup-template.md`'s existing-setup matrix instead.

Propose the change as a diff. Wait for explicit approval before writing the file.

**Next:** propose decorator placement.

---

## Step 8 — Propose `@trace_with_workflow` and `@trace_with_task` decorator diffs

Identify two kinds of functions in the agent code:

- **Orchestration entry points** — the function the customer calls to run the agent end-to-end. Decorate with `@trace_with_workflow`.
- **LLM-calling task functions** — the discrete units of work the workflow calls (a single LLM call, a tool invocation, a retrieval step). Decorate each with `@trace_with_task`.

> **CRITICAL — `@trace_with_workflow` and `@trace_with_task` are the only two decorators in scope for V1.** `monitoring-advisor` is built around the workflow/task model; other tracing primitives the SDK exposes are not part of the v1 surface.

Propose each decorator addition as a separate diff. Wait for **explicit per-diff approval**. See `decorator-placement.md` for placement guidance and the canonical example.

**Next:** confirm env vars are set (or skip, depending on step 2).

---

## Step 9 — Confirm env vars

Branches on the answer from step 2:

- **MC-hosted collector** — confirm the auth env vars for the chosen setup template are present in the customer's runtime environment. Use **presence-only checks**, e.g.:

  ```python
  bool(os.environ.get("MCD_DEFAULT_API_ID")) and bool(os.environ.get("MCD_DEFAULT_API_TOKEN"))
  # or, for the standard OTel header path:
  bool(os.environ.get("OTEL_EXPORTER_OTLP_HEADERS"))
  ```

  > **CRITICAL — never read or echo the credential value.** Presence (`bool(...)`) only. If a check needs to land in a logging or diagnostic file, mask everything but `True`/`False`. See `setup-template.md` on credential safety.

- **Self-hosted collector** — skip this step entirely. Auth is handled at the customer's collector; the MC SDK doesn't need `MCD_*` env vars in this branch. Don't ask the user to set them, and don't propose a check.

**Next:** hand back to the user to run their agent, then verify.

---

## Step 10 — Verify via `get_agent_metadata` (AFTER user runs the instrumented agent)

Ask the user to run the instrumented agent against their environment so it produces at least one workflow trace. Then branch on the MCP availability flag recorded in Step 0:

- **MCP available** — call `get_agent_metadata` again and diff against the snapshot from step 4. Expected outcomes:
  - A new entry exists with `agent_name` matching whatever the customer passed to `mc.setup(agent_name=...)`, and a new MCON.
  - If the same `agent_name` already existed in the snapshot (e.g. a dev/prod twin), the new MCON should still be different — confirm that.
  - If nothing new appears after a reasonable wait (see `verify-traces.md` for timing), go to step 11.
- **MCP unavailable** — hand off to manual UI verification. Tell the customer: "Sign in to Monte Carlo, go to Agent Observability, and confirm a new agent with the name you passed to `mc.setup(agent_name=...)` appears. First-time visibility for low-traffic dev agents can take 10–15 minutes; if it still isn't visible after that, go to step 11." Don't claim verification on the customer's behalf — they confirm.

See `verify-traces.md` for the full diffing logic, timing expectations, and edge cases.

**Next:** if verification passed, the workflow is done. If not, troubleshoot.

---

## Step 11 — On failure, branch to `troubleshooting.md`

The four common failure modes, in roughly the order to check:

1. **SDK init not running** — `mc.setup()` is in the file but the import path or entry point isn't actually loading it at runtime.
2. **Wrong instrumentor versions** — the installed OTel instrumentors are incompatible with the SDK or with each other.
3. **Missing credentials** — the selected MC-hosted auth env vars are not present in the runtime (`MCD_DEFAULT_API_ID` / `MCD_DEFAULT_API_TOKEN` or `OTEL_EXPORTER_OTLP_HEADERS`).
4. **Upstream pipeline not actually deployed** — the agent code with `mc.setup()` exists in the repo but the deployed runtime is still the old build.

`troubleshooting.md` also covers the **serverless `SimpleSpanProcessor` foot-gun** — Lambda freezing the process before `BatchSpanProcessor` flushes, producing partial or missing traces. If the runtime is serverless and traces look incomplete (rather than absent), that's the first thing to check.

Walk the customer through whichever branch matches their symptoms. Once a fix is in place, re-run step 10.

#!/usr/bin/env python3
"""
Detect runtime classification, dependency surface, and any existing Monte
Carlo OpenTelemetry setup in a Python codebase.

The script is intentionally a thin discovery layer. It walks dependency
manifests (requirements.txt, pyproject.toml, Pipfile), serverless deployment
markers, and existing `mc.setup()` calls — then emits a JSON document the
skill consumes. The script does **not** classify AI libraries or pick
instrumentor packages; that is the LLM's job, working from `dependencies[]`
plus the live PyPI list from `fetch_sdk_docs.py`.

Usage:
    python3 detect_libraries.py [TARGET_PATH]

TARGET_PATH defaults to the current working directory. Output is JSON
on stdout. Exit code is 0 on success and 1 on hard errors (missing or
unreadable target path).
"""

from __future__ import annotations

import argparse
import ast
import json
import os
import re
import sys
from pathlib import Path
from typing import Iterable

# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------

MAX_FILE_BYTES = 5 * 1024 * 1024  # 5 MB per-file cap

SKIP_DIRS = {
    ".venv",
    "venv",
    "node_modules",
    "__pycache__",
    ".git",
    "dist",
    "build",
    "target",
    ".tox",
    ".pytest_cache",
    ".mypy_cache",
}

SERVERLESS_FILES = {
    "serverless.yml",
    "serverless.yaml",
    "template.yaml",
    "template.yml",
    "vercel.json",
    "netlify.toml",
    "wrangler.toml",
    "zappa_settings.json",
    "modal.toml",
}

SERVERLESS_DEPS = {
    "aws-lambda-powertools",
    "mangum",
    "chalice",
    "zappa",
    "aws-cdk-lib",
    "aws-sam-cli",
    "modal",
    "sst",
}

SERVERLESS_CODE_PATTERNS = [
    re.compile(r"def\s+lambda_handler\s*\("),
    re.compile(r"from\s+chalice\s+import\s+Chalice"),
    re.compile(r"from\s+mangum\s+import\s+Mangum"),
    re.compile(r"app\s*=\s*Chalice\s*\("),
]

# Existing-setup detection requires BOTH an import of montecarlo_opentelemetry
# AND an actual setup() call in the same file. Matching on imports alone
# false-positives any file that uses the SDK's decorators (e.g. handler.py
# importing `montecarlo_opentelemetry as mc` to use `@mc.trace_with_workflow`)
# but doesn't actually call setup().
EXISTING_SETUP_IMPORT_PATTERNS = [
    "import montecarlo_opentelemetry",
    "from montecarlo_opentelemetry",
]
EXISTING_SETUP_FALLBACK_CALL_PATTERN = re.compile(
    r"\b(?:mc|montecarlo_opentelemetry)\.setup\s*\("
)


# ---------------------------------------------------------------------------
# TOML loader (stdlib tomllib in 3.11+, fall back to tomli, else None)
# ---------------------------------------------------------------------------


def _load_toml_module():
    try:
        import tomllib  # type: ignore[import-not-found]

        return tomllib
    except ImportError:
        pass
    try:
        import tomli  # type: ignore[import-not-found]

        return tomli
    except ImportError:
        return None


_TOML = _load_toml_module()


# ---------------------------------------------------------------------------
# Filesystem helpers
# ---------------------------------------------------------------------------


def _is_within(path: Path, root: Path) -> bool:
    """True if `path` (resolved) is inside `root` (resolved)."""
    try:
        resolved = path.resolve()
    except OSError:
        return False
    try:
        resolved.relative_to(root)
        return True
    except ValueError:
        return False


def _safe_read_text(path: Path) -> str | None:
    """Read a file as UTF-8 text, skipping if too large or unreadable."""
    try:
        size = path.stat().st_size
    except OSError as exc:
        print(f"warning: cannot stat {path}: {exc}", file=sys.stderr)
        return None
    if size > MAX_FILE_BYTES:
        print(
            f"warning: skipping {path} ({size} bytes exceeds {MAX_FILE_BYTES})",
            file=sys.stderr,
        )
        return None
    try:
        return path.read_text(encoding="utf-8", errors="replace")
    except OSError as exc:
        print(f"warning: cannot read {path}: {exc}", file=sys.stderr)
        return None


def _walk_files(root: Path) -> Iterable[Path]:
    """Yield files under root, skipping noise dirs and out-of-tree symlinks."""
    root_resolved = root.resolve()
    for dirpath, dirnames, filenames in os.walk(root, followlinks=False):
        # Filter directories in-place so os.walk doesn't descend into them.
        dirnames[:] = [d for d in dirnames if d not in SKIP_DIRS]

        # Drop any dir that resolves outside the target tree (symlink escape).
        kept: list[str] = []
        for d in dirnames:
            full = Path(dirpath) / d
            if _is_within(full, root_resolved):
                kept.append(d)
        dirnames[:] = kept

        for name in filenames:
            full = Path(dirpath) / name
            if full.is_symlink() and not _is_within(full, root_resolved):
                continue
            yield full


# ---------------------------------------------------------------------------
# Dependency parsing
# ---------------------------------------------------------------------------

# PEP 508 / requirements line — captures the project name only.
# Allowed name characters per PEP 508: letters, digits, ., -, _
_REQ_NAME_RE = re.compile(r"^([A-Za-z0-9][A-Za-z0-9._-]*)")
_EGG_RE = re.compile(r"[#&]egg=([A-Za-z0-9][A-Za-z0-9._-]*)")


def _normalize_dep(name: str) -> str:
    return name.strip().lower()


def _parse_requirements_line(line: str) -> str | None:
    """Extract a package name from a single requirements.txt line, or None."""
    raw = line.strip()
    if not raw:
        return None
    # Strip inline comments — preserve URLs that contain '#egg=' first.
    if "#egg=" not in raw and "#" in raw:
        raw = raw.split("#", 1)[0].strip()
    if not raw:
        return None

    lowered = raw.lower()

    # Skip include directives and pip flags.
    if (
        lowered.startswith("-r ")
        or lowered.startswith("--requirement ")
        or lowered.startswith("-c ")
        or lowered.startswith("--constraint ")
        or lowered.startswith("--index-url")
        or lowered.startswith("--extra-index-url")
        or lowered.startswith("--find-links")
        or lowered.startswith("--no-")
        or lowered.startswith("--pre")
        or lowered.startswith("--trusted-host")
    ):
        return None

    # Editable / VCS / URL specs — name comes from #egg=<name>.
    # Note: bare "-e ./local_pkg" without #egg= yields no package name and is skipped.
    if (
        lowered.startswith("-e ")
        or lowered.startswith("--editable ")
        or lowered.startswith("git+")
        or lowered.startswith("hg+")
        or lowered.startswith("svn+")
        or lowered.startswith("bzr+")
        or lowered.startswith("http://")
        or lowered.startswith("https://")
        or lowered.startswith("file://")
    ):
        m = _EGG_RE.search(raw)
        return _normalize_dep(m.group(1)) if m else None

    # Drop any "[extras]" segment, then match the leading package name.
    bracket = raw.find("[")
    if bracket > 0:
        candidate = raw[:bracket]
    else:
        candidate = raw
    m = _REQ_NAME_RE.match(candidate)
    return _normalize_dep(m.group(1)) if m else None


def _parse_requirements_file(path: Path) -> list[str]:
    text = _safe_read_text(path)
    if text is None:
        return []
    deps: list[str] = []
    try:
        for line in text.splitlines():
            name = _parse_requirements_line(line)
            if name:
                deps.append(name)
    except Exception as exc:  # noqa: BLE001 — tolerate any parse glitch
        print(f"warning: failed to parse {path}: {exc}", file=sys.stderr)
    return deps


def _pep508_name(spec: str) -> str | None:
    """Pull the project name from a PEP 508 requirement string."""
    candidate = spec.strip()
    if not candidate:
        return None
    bracket = candidate.find("[")
    if bracket > 0:
        candidate = candidate[:bracket]
    m = _REQ_NAME_RE.match(candidate)
    return _normalize_dep(m.group(1)) if m else None


def _parse_pyproject(path: Path) -> list[str]:
    if _TOML is None:
        print(
            f"warning: skipping {path} — no TOML parser available "
            "(install tomli or use Python 3.11+)",
            file=sys.stderr,
        )
        return []
    text = _safe_read_text(path)
    if text is None:
        return []
    try:
        data = _TOML.loads(text)
    except Exception as exc:  # noqa: BLE001
        print(f"warning: failed to parse {path}: {exc}", file=sys.stderr)
        return []

    deps: list[str] = []

    # PEP 621: [project] dependencies + optional-dependencies.
    project = data.get("project") if isinstance(data, dict) else None
    if isinstance(project, dict):
        for spec in project.get("dependencies", []) or []:
            if isinstance(spec, str):
                name = _pep508_name(spec)
                if name:
                    deps.append(name)
        opt = project.get("optional-dependencies") or {}
        if isinstance(opt, dict):
            for group in opt.values():
                if not isinstance(group, list):
                    continue
                for spec in group:
                    if isinstance(spec, str):
                        name = _pep508_name(spec)
                        if name:
                            deps.append(name)

    # Poetry: [tool.poetry.dependencies] + [tool.poetry.group.<g>.dependencies]
    tool = data.get("tool") if isinstance(data, dict) else None
    poetry = tool.get("poetry") if isinstance(tool, dict) else None
    if isinstance(poetry, dict):
        poetry_deps = poetry.get("dependencies") or {}
        if isinstance(poetry_deps, dict):
            for name in poetry_deps.keys():
                if isinstance(name, str) and name.lower() != "python":
                    deps.append(_normalize_dep(name))
        groups = poetry.get("group") or {}
        if isinstance(groups, dict):
            for group in groups.values():
                if not isinstance(group, dict):
                    continue
                gdeps = group.get("dependencies") or {}
                if isinstance(gdeps, dict):
                    for name in gdeps.keys():
                        if isinstance(name, str) and name.lower() != "python":
                            deps.append(_normalize_dep(name))

    return deps


def _parse_pipfile(path: Path) -> list[str]:
    if _TOML is None:
        print(
            f"warning: skipping {path} — no TOML parser available "
            "(install tomli or use Python 3.11+)",
            file=sys.stderr,
        )
        return []
    text = _safe_read_text(path)
    if text is None:
        return []
    try:
        data = _TOML.loads(text)
    except Exception as exc:  # noqa: BLE001
        print(f"warning: failed to parse {path}: {exc}", file=sys.stderr)
        return []

    deps: list[str] = []
    for section in ("packages", "dev-packages"):
        section_data = data.get(section) if isinstance(data, dict) else None
        if isinstance(section_data, dict):
            for name in section_data.keys():
                if isinstance(name, str):
                    deps.append(_normalize_dep(name))
    return deps


def _scan_tree(target: Path) -> dict:
    """Walk *target* once and bucket files by role.

    Returns a dict with:
    - ``dep_files``: paths to dependency manifests (requirements*.txt,
      pyproject.toml, Pipfile).
    - ``serverless_files``: paths whose filename matches SERVERLESS_FILES.
    - ``py_files``: paths to ``*.py`` source files.
    - ``py_contents``: ``{path: text}`` — eagerly read content of each Python
      file (None values are omitted; callers treat a missing key as unreadable).
    """
    dep_files: list[Path] = []
    serverless_files: list[Path] = []
    py_files: list[Path] = []
    py_contents: dict[Path, str] = {}

    serverless_names_lower = {n.lower() for n in SERVERLESS_FILES}

    for path in _walk_files(target):
        name = path.name
        lower = name.lower()
        suffix = path.suffix.lower()

        if lower == "requirements.txt" or (
            lower.startswith("requirements") and lower.endswith(".txt")
        ):
            dep_files.append(path)
        elif lower == "pyproject.toml":
            dep_files.append(path)
        elif lower == "pipfile":
            dep_files.append(path)

        if lower in serverless_names_lower:
            serverless_files.append(path)

        if suffix == ".py":
            py_files.append(path)
            text = _safe_read_text(path)
            if text is not None:
                py_contents[path] = text

    return {
        "dep_files": dep_files,
        "serverless_files": serverless_files,
        "py_files": py_files,
        "py_contents": py_contents,
    }


def _collect_dependencies(scan: dict) -> set[str]:
    """Collect normalized dep names from pre-scanned manifest files."""
    found: set[str] = set()
    for path in scan["dep_files"]:
        lower = path.name.lower()
        if lower == "requirements.txt" or (
            lower.startswith("requirements") and lower.endswith(".txt")
        ):
            found.update(_parse_requirements_file(path))
        elif lower == "pyproject.toml":
            found.update(_parse_pyproject(path))
        elif lower == "pipfile":
            found.update(_parse_pipfile(path))
    return found


# ---------------------------------------------------------------------------
# Runtime detection
# ---------------------------------------------------------------------------


def _detect_serverless(scan: dict, deps: set[str]) -> list[str]:
    """Return the list of serverless signals observed."""
    signals: list[str] = []

    # File-level markers — check anywhere in the walked tree; this catches
    # monorepo subprojects too.
    seen_files: set[str] = set()
    for path in scan["serverless_files"]:
        if path.name not in seen_files:
            signals.append(path.name)
            seen_files.add(path.name)

    # Dependency markers.
    for dep in sorted(deps):
        if dep in SERVERLESS_DEPS:
            signals.append(dep)

    # Code patterns — only meaningful tokens, not which file they came from.
    code_signals: set[str] = set()
    code_signal_labels = {
        SERVERLESS_CODE_PATTERNS[0]: "lambda_handler",
        SERVERLESS_CODE_PATTERNS[1]: "chalice_import",
        SERVERLESS_CODE_PATTERNS[2]: "mangum_import",
        SERVERLESS_CODE_PATTERNS[3]: "chalice_app",
    }
    py_contents = scan["py_contents"]
    for path in scan["py_files"]:
        text = py_contents.get(path)
        if text is None:
            continue
        for pattern, label in code_signal_labels.items():
            if label in code_signals:
                continue
            if pattern.search(text):
                code_signals.add(label)
        if len(code_signals) == len(code_signal_labels):
            break
    signals.extend(sorted(code_signals))

    return signals


# ---------------------------------------------------------------------------
# Existing setup detection
# ---------------------------------------------------------------------------


def _has_existing_setup_call(text: str) -> bool:
    try:
        tree = ast.parse(text)
    except SyntaxError:
        has_import = any(pat in text for pat in EXISTING_SETUP_IMPORT_PATTERNS)
        return has_import and bool(EXISTING_SETUP_FALLBACK_CALL_PATTERN.search(text))

    module_aliases: set[str] = set()
    setup_names: set[str] = set()

    for node in ast.walk(tree):
        if isinstance(node, ast.Import):
            for alias in node.names:
                if alias.name == "montecarlo_opentelemetry":
                    module_aliases.add(alias.asname or alias.name)
        elif isinstance(node, ast.ImportFrom):
            if node.module != "montecarlo_opentelemetry":
                continue
            for alias in node.names:
                if alias.name == "setup":
                    setup_names.add(alias.asname or alias.name)
                elif alias.name == "*":
                    setup_names.add("setup")

    if not module_aliases and not setup_names:
        return False

    for node in ast.walk(tree):
        if not isinstance(node, ast.Call):
            continue
        func = node.func
        if (
            isinstance(func, ast.Attribute)
            and func.attr == "setup"
            and isinstance(func.value, ast.Name)
            and func.value.id in module_aliases
        ):
            return True
        if isinstance(func, ast.Name) and func.id in setup_names:
            return True

    return False


def _detect_existing_setup(scan: dict, target: Path) -> dict:
    files: list[str] = []
    target_resolved = target.resolve()
    py_contents = scan["py_contents"]
    for path in scan["py_files"]:
        text = py_contents.get(path)
        if text is None:
            continue
        if not _has_existing_setup_call(text):
            continue
        try:
            rel = path.resolve().relative_to(target_resolved)
            files.append(str(rel))
        except ValueError:
            files.append(str(path))
    files.sort()
    return {"found": bool(files), "files": files}


# ---------------------------------------------------------------------------
# Entry point
# ---------------------------------------------------------------------------


def detect(target: Path) -> dict:
    scan = _scan_tree(target)
    deps = _collect_dependencies(scan)

    serverless_signals = _detect_serverless(scan, deps)
    if serverless_signals:
        runtime = "serverless"
    elif scan["dep_files"]:
        runtime = "long_running"
    else:
        runtime = "unknown"

    existing_setup = _detect_existing_setup(scan, target)

    return {
        "dependencies": sorted(deps),
        "runtime": runtime,
        "serverless_signals": serverless_signals,
        "existing_setup": existing_setup,
    }


def main() -> None:
    parser = argparse.ArgumentParser(
        description=(
            "Detect runtime style, Python dependencies, and any existing "
            "Monte Carlo OpenTelemetry setup in a codebase. AI-library "
            "matching is the LLM's job; this script just emits the raw "
            "discovery surface."
        )
    )
    parser.add_argument(
        "target",
        nargs="?",
        default=".",
        help="Path to the codebase to scan (defaults to the current directory).",
    )
    args = parser.parse_args()

    target = Path(args.target)
    if not target.exists():
        print(
            json.dumps({"error": f"Target path does not exist: {target}"}, indent=2)
        )
        sys.exit(1)
    if not target.is_dir():
        print(
            json.dumps({"error": f"Target path is not a directory: {target}"}, indent=2)
        )
        sys.exit(1)
    if not os.access(target, os.R_OK):
        print(
            json.dumps({"error": f"Target path is not readable: {target}"}, indent=2)
        )
        sys.exit(1)

    try:
        result = detect(target)
    except OSError as exc:
        print(json.dumps({"error": f"Filesystem error: {exc}"}, indent=2))
        sys.exit(1)

    print(json.dumps(result, indent=2))


if __name__ == "__main__":
    main()

#!/usr/bin/env python3
"""
Fetch Monte Carlo OpenTelemetry SDK docs at runtime so the instrument-agent
skill stays in sync with the SDK without per-release skill updates.

PyPI is the canonical public source: the SDK's GitHub repo is private, so a
runtime fetch against it would always fail. We fetch the PyPI JSON metadata
for `montecarlo-opentelemetry`, parse the README that PyPI mirrors under
`info.description` for the supported instrumentor list, and emit a JSON
document on stdout. On any failure (network, parse, no instrumentors found)
we fail closed with exit code 1 and a JSON error payload pointing at
https://pypi.org/project/montecarlo-opentelemetry/.

Usage:
    python3 fetch_sdk_docs.py
    python3 fetch_sdk_docs.py --quiet
"""

from __future__ import annotations

import argparse
import json
import re
import sys
import urllib.error
import urllib.request
from datetime import datetime, timezone

PYPI_URL = "https://pypi.org/pypi/montecarlo-opentelemetry/json"
PYPI_PROJECT_URL = "https://pypi.org/project/montecarlo-opentelemetry/"
READ_BYTES_CAP = 1_000_000  # 1 MB
TIMEOUT_SECONDS = 10

MAX_INSTRUMENTOR_MATCHES = 50

# "# For Langchain/LangGraph" or "### For OpenAI"
_HEADER_RE = re.compile(
    r"^\s*(?:#{1,6}\s+|<!--\s*)?For\s+([A-Za-z0-9_./+\- ]+?)\s*(?:-->|$)",
    re.MULTILINE,
)
# pip install "opentelemetry-instrumentation-<lib><=0.53.4>"
_PIP_INSTALL_RE = re.compile(
    r"""pip\s+install\s+["']?
        (opentelemetry-instrumentation-[a-z0-9_\-]+)
        \s*
        (
            (?:[<>=!~]=?|===)\s*[A-Za-z0-9_.\-+*]+
            (?:\s*,\s*(?:[<>=!~]=?|===)\s*[A-Za-z0-9_.\-+*]+)*
        )?
        ["']?""",
    re.IGNORECASE | re.VERBOSE,
)
# Markdown bullet listing each supported package as a PyPI link, e.g.
#   * [opentelemetry-instrumentation-anthropic](https://pypi.org/project/opentelemetry-instrumentation-anthropic/)
_BULLET_PACKAGE_RE = re.compile(
    r"""^\s*[-*+]\s+
        \[\s*opentelemetry-instrumentation-([a-z0-9][a-z0-9_\-]*)\s*\]
        \(\s*https?://pypi\.org/project/opentelemetry-instrumentation-[a-z0-9_\-]+/?\s*\)
    """,
    re.MULTILINE | re.IGNORECASE | re.VERBOSE,
)


# ---------------------------------------------------------------------------
# HTTP helpers
# ---------------------------------------------------------------------------


def _fetch_bytes(url: str) -> bytes:
    """Fetch up to READ_BYTES_CAP+1 bytes with an explicit timeout.

    Caller must check `len(result) > READ_BYTES_CAP` to detect overruns.
    """
    req = urllib.request.Request(
        url, headers={"User-Agent": "mc-agent-toolkit/instrument-agent"}
    )
    with urllib.request.urlopen(req, timeout=TIMEOUT_SECONDS) as resp:
        # Read one extra byte so the caller can detect responses that exceed
        # the cap (rather than silently truncating).
        return resp.read(READ_BYTES_CAP + 1)


# ---------------------------------------------------------------------------
# README parsing
# ---------------------------------------------------------------------------


def _canonical_libraries(label: str) -> list[str]:
    """Map a header label to canonical lowercase library identifiers.

    "Langchain/LangGraph" -> ["langchain", "langgraph"]
    "OpenAI" -> ["openai"]
    "Google Gen AI" -> ["google_gen_ai"]
    """
    parts = re.split(r"[/,&+]| and ", label)
    libs: list[str] = []
    for part in parts:
        cleaned = part.strip()
        if not cleaned:
            continue
        slug = re.sub(r"[^a-z0-9]+", "_", cleaned.lower()).strip("_")
        if slug and slug not in libs:
            libs.append(slug)
    return libs


def _parse_supported_instrumentors(
    readme_text: str,
    warnings: list[str],
) -> list[dict]:
    """Extract `(library, package, version_constraint)` tuples from the README.

    The README has two surfaces describing supported instrumentors:

    1. Quick-start `# For <Label>` headers paired with a `pip install` line —
       these include explicit version constraints (e.g. `<=0.53.4`).
    2. A bullet list further down ("See a selection of available instrumentation
       libraries below.") with one PyPI link per supported package — no version
       constraints, but covers the long tail (Anthropic, Bedrock, CrewAI,
       SageMaker, Vertex AI, ...).

    We parse both, deduplicate by `(library, package)`, and return the union.
    Header-derived entries take precedence so any version_constraint they
    surface is preserved.

    We do NOT exec/eval/compile/import any fetched bytes — this is plain
    regex over text. Bounded to MAX_INSTRUMENTOR_MATCHES to avoid pathological
    inputs.
    """
    instrumentors: list[dict] = []
    seen: set[tuple[str, str]] = set()
    parse_warned = False

    # ----- Pass 1: header + pip install pairs (with version_constraint) -----
    headers = list(_HEADER_RE.finditer(readme_text))
    for idx, header in enumerate(headers):
        if len(instrumentors) >= MAX_INSTRUMENTOR_MATCHES:
            break

        body_start = header.end()
        body_end = (
            headers[idx + 1].start() if idx + 1 < len(headers) else len(readme_text)
        )
        # Cap the body window so a missing next-header doesn't make us scan
        # the whole document.
        body_end = min(body_end, body_start + 4_000)
        body = readme_text[body_start:body_end]

        pip_match = _PIP_INSTALL_RE.search(body)
        if not pip_match:
            continue

        package = pip_match.group(1).strip()
        version_constraint = (pip_match.group(2) or "").strip() or None
        libraries = _canonical_libraries(header.group(1))
        if not libraries:
            if not parse_warned:
                warnings.append(
                    f"Could not derive canonical library from header: {header.group(1)!r}"
                )
                parse_warned = True
            continue

        for library in libraries:
            key = (library, package)
            if key in seen:
                continue
            seen.add(key)
            entry: dict = {"library": library, "package": package}
            if version_constraint:
                entry["version_constraint"] = version_constraint
            instrumentors.append(entry)
            if len(instrumentors) >= MAX_INSTRUMENTOR_MATCHES:
                break

    # ----- Pass 2: PyPI-link bullet list (no version_constraint) ------------
    for match in _BULLET_PACKAGE_RE.finditer(readme_text):
        if len(instrumentors) >= MAX_INSTRUMENTOR_MATCHES:
            break
        suffix = match.group(1).lower()
        # The library identifier is the package suffix (post
        # "opentelemetry-instrumentation-").
        library = suffix
        package = f"opentelemetry-instrumentation-{suffix}"
        key = (library, package)
        if key in seen:
            continue
        seen.add(key)
        instrumentors.append({"library": library, "package": package})

    return instrumentors


# ---------------------------------------------------------------------------
# PyPI fetch
# ---------------------------------------------------------------------------


def _fetch_pypi() -> dict:
    """Fetch PyPI metadata. Raises on overrun, HTTP, or network errors."""
    raw = _fetch_bytes(PYPI_URL)
    if len(raw) > READ_BYTES_CAP:
        raise OSError(f"PyPI metadata exceeded {READ_BYTES_CAP} byte cap")
    payload = json.loads(raw.decode("utf-8"))
    info = payload.get("info") or {}
    project_urls = info.get("project_urls") or {}
    pypi_project_url = (
        project_urls.get("Homepage")
        or project_urls.get("Source")
        or PYPI_PROJECT_URL
    )
    requires_dist = info.get("requires_dist") or []
    if not isinstance(requires_dist, list):
        requires_dist = []
    description = info.get("description") or ""
    if not isinstance(description, str):
        description = ""
    return {
        "version": info.get("version") or "",
        "pypi_url": pypi_project_url,
        "requires_dist": [str(r) for r in requires_dist],
        # README content as PyPI knows it; parsed below for the instrumentor
        # list. Captured under a separate key so it is not surfaced in the
        # final SDK metadata block.
        "_description": description,
    }


def _describe_fetch_error(exc: BaseException) -> str:
    if isinstance(exc, urllib.error.HTTPError):
        return f"HTTP {exc.code} {exc.reason}"
    if isinstance(exc, urllib.error.URLError):
        return f"network error: {exc.reason}"
    if isinstance(exc, TimeoutError):
        return f"timed out after {TIMEOUT_SECONDS}s"
    return f"{type(exc).__name__}: {exc}"


# ---------------------------------------------------------------------------
# Output assembly
# ---------------------------------------------------------------------------


def _now_iso() -> str:
    return datetime.now(timezone.utc).isoformat(timespec="seconds")


def _build_success(
    sdk_meta: dict,
    instrumentors: list[dict],
    warnings: list[str],
) -> dict:
    # Strip the internal `_description` field from the published sdk block.
    public_sdk = {k: v for k, v in sdk_meta.items() if not k.startswith("_")}
    return {
        "source": "pypi",
        "fetched_at": _now_iso(),
        "sdk": public_sdk,
        "supported_instrumentors": instrumentors,
        "warnings": warnings,
    }


def _emit_failure(reason: str, warnings: list[str]) -> None:
    payload = {
        "source": "error",
        "fetched_at": _now_iso(),
        "error": reason,
        "guidance": (
            "Live PyPI fetch failed. Run `pip install montecarlo-opentelemetry` "
            f"and consult {PYPI_PROJECT_URL} to identify the current set of "
            "supported instrumentors."
        ),
        "warnings": warnings,
    }
    print(json.dumps(payload, indent=2))


# ---------------------------------------------------------------------------
# Entry point
# ---------------------------------------------------------------------------


def main() -> None:
    parser = argparse.ArgumentParser(
        description=(
            "Fetch Monte Carlo OpenTelemetry SDK metadata from PyPI for the "
            "instrument-agent skill."
        ),
    )
    parser.add_argument(
        "--quiet", action="store_true", help="Suppress stderr warnings"
    )
    args = parser.parse_args()

    warnings: list[str] = []

    # ----- PyPI fetch -------------------------------------------------------
    try:
        sdk_meta = _fetch_pypi()
    except (
        urllib.error.HTTPError,
        urllib.error.URLError,
        TimeoutError,
        OSError,
        json.JSONDecodeError,
    ) as exc:
        reason = f"PyPI fetch failed: {_describe_fetch_error(exc)}"
        warnings.append(reason)
        if not args.quiet:
            for w in warnings:
                print(w, file=sys.stderr)
        _emit_failure(reason, warnings)
        sys.exit(1)

    description = sdk_meta.get("_description") or ""
    if not description.strip():
        reason = "PyPI metadata has no 'info.description' to parse for supported instrumentors."
        warnings.append(reason)
        if not args.quiet:
            for w in warnings:
                print(w, file=sys.stderr)
        _emit_failure(reason, warnings)
        sys.exit(1)

    # ----- Parse PyPI description ------------------------------------------
    instrumentors = _parse_supported_instrumentors(description, warnings)

    if not instrumentors:
        reason = (
            "Parsed PyPI 'info.description' but found no supported instrumentors. "
            "The README format on PyPI may have changed."
        )
        warnings.append(reason)
        if not args.quiet:
            for w in warnings:
                print(w, file=sys.stderr)
        _emit_failure(reason, warnings)
        sys.exit(1)

    if not args.quiet and warnings:
        for w in warnings:
            print(w, file=sys.stderr)

    result = _build_success(sdk_meta, instrumentors, warnings)
    print(json.dumps(result, indent=2))
    sys.exit(0)


if __name__ == "__main__":
    main()

#!/usr/bin/env python3
"""
Smoke test for detect_libraries.py — runs it against each fixture in
fixtures/ and asserts the JSON output is correct.

The script's contract is "raw discovery surface": dependencies (sorted list
of normalized pip package names), runtime classification, serverless
signals, and existing-`mc.setup()` detection. AI-library disambiguation
is the LLM's job and is not exercised here.

Run:
    python3 skills/instrument-agent/tests/test_detect_libraries.py
"""

from __future__ import annotations

import json
import subprocess
import sys
from pathlib import Path

TESTS_DIR = Path(__file__).parent
SKILL_ROOT = TESTS_DIR.parent
DETECT_SCRIPT = SKILL_ROOT / "scripts" / "detect_libraries.py"
FIXTURES_DIR = TESTS_DIR / "fixtures"

PASSED = 0
FAILED = 0


def run_detect(fixture: str) -> dict:
    """Run detect_libraries.py against a fixture and return parsed JSON."""
    result = subprocess.run(
        [sys.executable, str(DETECT_SCRIPT), str(FIXTURES_DIR / fixture)],
        capture_output=True,
        text=True,
        check=True,
        timeout=30,
    )
    return json.loads(result.stdout)


def check(label: str, condition: bool, hint: str = "") -> None:
    """Record a single check.

    Increments the global PASSED/FAILED counters and prints a PASS/FAIL line.
    On failure, raises AssertionError so pytest catches per-test failures
    (each `test_*` function fails on its first failed check). The standalone
    runner in `main()` wraps each test in its own try/except so all tests
    still run regardless of intermediate failures.
    """
    global PASSED, FAILED
    if condition:
        PASSED += 1
        print(f"  PASS  {label}")
        return
    FAILED += 1
    suffix = f" — {hint}" if hint else ""
    msg = f"FAIL  {label}{suffix}"
    print(f"  {msg}")
    raise AssertionError(msg)


def test_requirements_txt() -> None:
    print("\n== requirements.txt ==")
    out = run_detect("requirements")
    deps = out["dependencies"]
    check("dependencies includes langchain", "langchain" in deps)
    check("dependencies includes openai", "openai" in deps)
    check("dependencies includes anthropic", "anthropic" in deps)
    check(
        "runtime is long_running",
        out["runtime"] == "long_running",
        hint=f"got {out['runtime']!r}",
    )
    check(
        "no existing setup",
        out["existing_setup"]["found"] is False,
        hint=f"existing_setup={out['existing_setup']!r}",
    )
    check(
        "no serverless signals",
        out["serverless_signals"] == [],
        hint=f"got {out['serverless_signals']!r}",
    )
    check(
        "dependencies sorted",
        deps == sorted(deps),
        hint=f"dependencies={deps!r}",
    )


def test_poetry_pyproject() -> None:
    print("\n== Poetry pyproject.toml ==")
    out = run_detect("poetry-pyproject")
    deps = out["dependencies"]
    check("dependencies includes langchain", "langchain" in deps)
    check("dependencies includes openai", "openai" in deps)
    check("dependencies includes crewai", "crewai" in deps)
    check(
        "python entry was filtered out",
        "python" not in deps,
        hint=f"dependencies={deps!r}",
    )
    check(
        "runtime is long_running",
        out["runtime"] == "long_running",
        hint=f"got {out['runtime']!r}",
    )
    check("no serverless signals", out["serverless_signals"] == [])


def test_pep621_pyproject() -> None:
    print("\n== PEP 621 pyproject.toml ==")
    out = run_detect("pep621-pyproject")
    deps = out["dependencies"]
    check("dependencies includes langchain", "langchain" in deps)
    check("dependencies includes anthropic", "anthropic" in deps)
    check(
        "dependencies includes google-cloud-aiplatform (Vertex AI surface)",
        "google-cloud-aiplatform" in deps,
        hint=f"dependencies={deps!r}",
    )
    check(
        "runtime is long_running",
        out["runtime"] == "long_running",
        hint=f"got {out['runtime']!r}",
    )


def test_pipfile() -> None:
    print("\n== Pipfile ==")
    out = run_detect("pipfile")
    deps = out["dependencies"]
    check("dependencies includes langchain", "langchain" in deps)
    check("dependencies includes openai", "openai" in deps)
    check(
        "runtime is long_running",
        out["runtime"] == "long_running",
        hint=f"got {out['runtime']!r}",
    )


def test_serverless() -> None:
    print("\n== serverless ==")
    out = run_detect("serverless")
    check(
        "runtime is serverless",
        out["runtime"] == "serverless",
        hint=f"got {out['runtime']!r}",
    )
    signals = out["serverless_signals"]
    check(
        "serverless_signals contains serverless.yml",
        "serverless.yml" in signals,
        hint=f"got {signals!r}",
    )
    check(
        "serverless_signals contains lambda_handler",
        "lambda_handler" in signals,
        hint=f"got {signals!r}",
    )
    check(
        "dependencies still includes langchain",
        "langchain" in out["dependencies"],
    )


def test_existing_setup() -> None:
    print("\n== existing setup ==")
    out = run_detect("existing-setup")
    existing = out["existing_setup"]
    check(
        "existing_setup.found is True",
        existing["found"] is True,
        hint=f"got {existing!r}",
    )
    check(
        "existing_setup.files contains src/tracing.py",
        any(f.replace("\\", "/") == "src/tracing.py" for f in existing["files"]),
        hint=f"got files={existing['files']!r}",
    )
    check(
        "existing_setup.files contains aliased module setup call",
        any(
            f.replace("\\", "/") == "src/tracing_alias.py"
            for f in existing["files"]
        ),
        hint=f"got files={existing['files']!r}",
    )
    check(
        "existing_setup.files contains direct imported setup call",
        any(
            f.replace("\\", "/") == "src/tracing_direct.py"
            for f in existing["files"]
        ),
        hint=f"got files={existing['files']!r}",
    )
    check(
        "dependencies still includes langchain",
        "langchain" in out["dependencies"],
    )


def test_no_deps() -> None:
    print("\n== no-deps ==")
    out = run_detect("no-deps")
    check(
        "dependencies is empty",
        out["dependencies"] == [],
        hint=f"got {out['dependencies']!r}",
    )
    check(
        "runtime is unknown",
        out["runtime"] == "unknown",
        hint=f"got {out['runtime']!r}",
    )
    check(
        "existing_setup.found is False",
        out["existing_setup"]["found"] is False,
    )
    check(
        "no serverless signals",
        out["serverless_signals"] == [],
    )


def test_mixed() -> None:
    print("\n== mixed requirements + pyproject ==")
    out = run_detect("mixed-requirements-pyproject")
    deps = out["dependencies"]
    check(
        "dependencies includes langchain (from pyproject)",
        "langchain" in deps,
        hint=f"dependencies={deps!r}",
    )
    check(
        "dependencies includes openai (from requirements.txt)",
        "openai" in deps,
        hint=f"dependencies={deps!r}",
    )
    check(
        "runtime is long_running",
        out["runtime"] == "long_running",
        hint=f"got {out['runtime']!r}",
    )


def test_boto3_only() -> None:
    """boto3 lands in dependencies like any other package.

    Disambiguation (is this Bedrock, SageMaker, or just S3?) is the LLM's
    job — it sees boto3 in the deps list and asks the user. The script
    itself doesn't single boto3 out; this test pins that contract.
    """
    print("\n== boto3-only ==")
    out = run_detect("boto3-only")
    deps = out["dependencies"]
    check(
        "dependencies includes langchain",
        "langchain" in deps,
        hint=f"dependencies={deps!r}",
    )
    check(
        "dependencies includes boto3 (raw, no special handling)",
        "boto3" in deps,
        hint=f"dependencies={deps!r}",
    )
    check(
        "runtime is long_running",
        out["runtime"] == "long_running",
        hint=f"got {out['runtime']!r}",
    )


def test_sample_agent() -> None:
    """Phase 3 structural smoke — long-running LangGraph fixture.

    Validates that detect_libraries produces JSON the workflow's step #1 can
    consume to drive the rest of the flow toward the long-running mc.setup()
    template path.
    """
    print("\n== sample_agent (Phase 3 smoke, long-running) ==")
    out = run_detect("sample_agent")
    deps = out["dependencies"]
    check("dependencies includes langchain", "langchain" in deps)
    check("dependencies includes langgraph", "langgraph" in deps)
    check("dependencies includes openai", "openai" in deps)
    check(
        "runtime is long_running",
        out["runtime"] == "long_running",
        hint=f"got {out['runtime']!r}",
    )
    check(
        "no serverless signals",
        out["serverless_signals"] == [],
        hint=f"got {out['serverless_signals']!r}",
    )
    check(
        "no false-positive existing setup",
        out["existing_setup"]["found"] is False,
        hint=f"existing_setup={out['existing_setup']!r}",
    )


def test_sample_serverless_agent() -> None:
    """Phase 3 structural smoke — Lambda-shaped LangGraph fixture.

    Validates that detect_libraries flips runtime to "serverless" and
    surfaces the framework signals the workflow needs to route toward the
    SimpleSpanProcessor mc.setup() variant.
    """
    print("\n== sample_serverless_agent (Phase 3 smoke, serverless) ==")
    out = run_detect("sample_serverless_agent")
    deps = out["dependencies"]
    check("dependencies includes langchain", "langchain" in deps)
    check("dependencies includes langgraph", "langgraph" in deps)
    check("dependencies includes openai", "openai" in deps)
    check(
        "runtime is serverless",
        out["runtime"] == "serverless",
        hint=f"got {out['runtime']!r}",
    )
    signals = out["serverless_signals"]
    check(
        "serverless_signals contains serverless.yml",
        "serverless.yml" in signals,
        hint=f"got {signals!r}",
    )
    check(
        "serverless_signals contains lambda_handler",
        "lambda_handler" in signals,
        hint=f"got {signals!r}",
    )
    check(
        "serverless_signals contains aws-lambda-powertools",
        "aws-lambda-powertools" in signals,
        hint=f"got {signals!r}",
    )
    check(
        "no false-positive existing setup",
        out["existing_setup"]["found"] is False,
        hint=f"existing_setup={out['existing_setup']!r}",
    )


def main() -> None:
    tests = [
        test_requirements_txt,
        test_poetry_pyproject,
        test_pep621_pyproject,
        test_pipfile,
        test_serverless,
        test_existing_setup,
        test_no_deps,
        test_mixed,
        test_boto3_only,
        test_sample_agent,
        test_sample_serverless_agent,
    ]
    # Run every test even when an early one fails — `check()` raises on the
    # first failure inside a single test, but we want a complete pass/fail
    # summary across all tests when invoked standalone. pytest invokes each
    # `test_*` directly without going through main(), so per-test fast-fail
    # via AssertionError is the right behavior under pytest.
    for fn in tests:
        try:
            fn()
        except AssertionError:
            # Already logged by check(); continue so the summary covers all tests.
            pass
    print(f"\n{'=' * 40}")
    print(f"Results: {PASSED} passed, {FAILED} failed")
    sys.exit(0 if FAILED == 0 else 1)


if __name__ == "__main__":
    main()

#!/usr/bin/env python3
"""
Tests for fetch_sdk_docs.py — unit-tests internal helpers and an end-to-end
subprocess check for the PyPI-fetch-failure fail-closed path.

Run:
    python3 skills/instrument-agent/tests/test_fetch_sdk_docs.py
    pytest skills/instrument-agent/tests/test_fetch_sdk_docs.py -q
"""

from __future__ import annotations

import json
import subprocess
import sys
from pathlib import Path
from unittest.mock import patch

TESTS_DIR = Path(__file__).parent
SKILL_ROOT = TESTS_DIR.parent
SCRIPT_DIR = SKILL_ROOT / "scripts"
FETCH_SCRIPT = SCRIPT_DIR / "fetch_sdk_docs.py"

sys.path.insert(0, str(SCRIPT_DIR))
from fetch_sdk_docs import (  # noqa: E402
    _build_success,
    _canonical_libraries,
    _parse_supported_instrumentors,
)

PASSED = 0
FAILED = 0


def check(label: str, condition: bool, hint: str = "") -> None:
    """Record a single check.

    Increments the global PASSED/FAILED counters and prints a PASS/FAIL line.
    On failure, raises AssertionError so pytest catches per-test failures
    (each `test_*` function fails on its first failed check). The standalone
    runner in `main()` wraps each test in its own try/except so all tests
    still run regardless of intermediate failures.
    """
    global PASSED, FAILED
    if condition:
        PASSED += 1
        print(f"  PASS  {label}")
        return
    FAILED += 1
    suffix = f" — {hint}" if hint else ""
    msg = f"FAIL  {label}{suffix}"
    print(f"  {msg}")
    raise AssertionError(msg)


# ---------------------------------------------------------------------------
# test_canonical_libraries
# ---------------------------------------------------------------------------


def test_canonical_libraries() -> None:
    print("\n== _canonical_libraries ==")

    result = _canonical_libraries("Langchain/LangGraph")
    check(
        "Langchain/LangGraph -> [langchain, langgraph]",
        result == ["langchain", "langgraph"],
        hint=f"got {result!r}",
    )

    result = _canonical_libraries("OpenAI")
    check(
        "OpenAI -> [openai]",
        result == ["openai"],
        hint=f"got {result!r}",
    )

    result = _canonical_libraries("Google Gen AI")
    check(
        "Google Gen AI -> [google_gen_ai]",
        result == ["google_gen_ai"],
        hint=f"got {result!r}",
    )

    result = _canonical_libraries("")
    check(
        "empty string -> []",
        result == [],
        hint=f"got {result!r}",
    )

    result = _canonical_libraries("   /   ")
    check(
        "whitespace-only slashes -> []",
        result == [],
        hint=f"got {result!r}",
    )

    result = _canonical_libraries("///")
    check(
        "only separators -> []",
        result == [],
        hint=f"got {result!r}",
    )


# ---------------------------------------------------------------------------
# test_parse_supported_instrumentors
# ---------------------------------------------------------------------------

_FIXTURE_README = """\
# Monte Carlo OpenTelemetry SDK

## For OpenAI

Install the following instrumentor:

```bash
pip install "opentelemetry-instrumentation-openai<=0.53.4"
```

## For Anthropic

```bash
pip install opentelemetry-instrumentation-anthropic
```

## Available instrumentation libraries

See a selection of available instrumentation libraries below.

* [opentelemetry-instrumentation-anthropic](https://pypi.org/project/opentelemetry-instrumentation-anthropic/)
* [opentelemetry-instrumentation-crewai](https://pypi.org/project/opentelemetry-instrumentation-crewai/)
* [opentelemetry-instrumentation-bedrock](https://pypi.org/project/opentelemetry-instrumentation-bedrock/)
"""


def test_parse_supported_instrumentors() -> None:
    print("\n== _parse_supported_instrumentors ==")
    warnings: list[str] = []
    instrumentors = _parse_supported_instrumentors(_FIXTURE_README, warnings)

    libraries = {entry["library"] for entry in instrumentors}

    check(
        "openai found via header+pip-install surface",
        "openai" in libraries,
        hint=f"libraries={libraries!r}",
    )
    check(
        "anthropic found (header or bullet list surface)",
        "anthropic" in libraries,
        hint=f"libraries={libraries!r}",
    )
    check(
        "crewai found via bullet list surface",
        "crewai" in libraries,
        hint=f"libraries={libraries!r}",
    )
    check(
        "bedrock found via bullet list surface",
        "bedrock" in libraries,
        hint=f"libraries={libraries!r}",
    )

    # Entries from header+pip should carry version_constraint
    openai_entry = next(
        (e for e in instrumentors if e.get("library") == "openai"), None
    )
    check(
        "openai entry has version_constraint from pip install line",
        openai_entry is not None
        and "version_constraint" in openai_entry
        and "0.53.4" in (openai_entry.get("version_constraint") or ""),
        hint=f"openai_entry={openai_entry!r}",
    )

    # Bullet-only entries should NOT carry version_constraint
    crewai_entry = next(
        (e for e in instrumentors if e.get("library") == "crewai"), None
    )
    check(
        "crewai (bullet-only) has no version_constraint",
        crewai_entry is not None and "version_constraint" not in crewai_entry,
        hint=f"crewai_entry={crewai_entry!r}",
    )

    # Deduplication: anthropic appears in both surfaces — only one entry
    anthropic_entries = [e for e in instrumentors if e.get("library") == "anthropic"]
    check(
        "anthropic deduplicated to a single entry",
        len(anthropic_entries) == 1,
        hint=f"count={len(anthropic_entries)}",
    )


# ---------------------------------------------------------------------------
# test_parse_failure_paths
# ---------------------------------------------------------------------------


def test_parse_failure_paths() -> None:
    """Description-parse must yield an empty list (caller fails closed)."""
    print("\n== _parse_supported_instrumentors failure paths ==")

    warnings: list[str] = []
    instrumentors = _parse_supported_instrumentors("", warnings)
    check(
        "empty description -> no instrumentors",
        instrumentors == [],
        hint=f"got {instrumentors!r}",
    )

    warnings = []
    instrumentors = _parse_supported_instrumentors(
        "# Some unrelated readme\n\nNothing to see here.\n", warnings
    )
    check(
        "readme without instrumentor markers -> no instrumentors",
        instrumentors == [],
        hint=f"got {instrumentors!r}",
    )


# ---------------------------------------------------------------------------
# test_build_success_shape
# ---------------------------------------------------------------------------


def test_build_success_shape() -> None:
    """Verify the success-path JSON shape — keys and types only."""
    print("\n== _build_success output shape ==")

    sdk_meta = {
        "version": "1.2.3",
        "pypi_url": "https://pypi.org/project/montecarlo-opentelemetry/",
        "requires_dist": ["opentelemetry-api>=1.0", "wrapt<2"],
        "_description": "# README body here",
    }
    instrumentors = [
        {
            "library": "openai",
            "package": "opentelemetry-instrumentation-openai",
            "version_constraint": "<=0.53.4",
        }
    ]
    warnings: list[str] = []

    result = _build_success(sdk_meta, instrumentors, warnings)

    check(
        "source == 'pypi'",
        result.get("source") == "pypi",
        hint=f"source={result.get('source')!r}",
    )
    check(
        "fetched_at present",
        isinstance(result.get("fetched_at"), str) and bool(result["fetched_at"]),
        hint=f"fetched_at={result.get('fetched_at')!r}",
    )
    check(
        "sdk block present and a dict",
        isinstance(result.get("sdk"), dict),
        hint=f"sdk={result.get('sdk')!r}",
    )
    check(
        "sdk block does not leak internal _description",
        "_description" not in (result.get("sdk") or {}),
        hint=f"sdk keys={list((result.get('sdk') or {}).keys())}",
    )
    check(
        "sdk.version preserved",
        (result.get("sdk") or {}).get("version") == "1.2.3",
        hint=f"sdk={result.get('sdk')!r}",
    )
    check(
        "sdk.requires_dist preserved",
        (result.get("sdk") or {}).get("requires_dist") == [
            "opentelemetry-api>=1.0",
            "wrapt<2",
        ],
        hint=f"sdk={result.get('sdk')!r}",
    )
    check(
        "supported_instrumentors matches input",
        result.get("supported_instrumentors") == instrumentors,
        hint=f"supported_instrumentors={result.get('supported_instrumentors')!r}",
    )
    check(
        "warnings list present",
        isinstance(result.get("warnings"), list),
        hint=f"warnings={result.get('warnings')!r}",
    )


# ---------------------------------------------------------------------------
# test_pypi_fetch_success (mocked)
# ---------------------------------------------------------------------------


def _make_pypi_payload(description: str, version: str = "1.2.3") -> bytes:
    return json.dumps(
        {
            "info": {
                "version": version,
                "description": description,
                "project_urls": {
                    "Homepage": "https://pypi.org/project/montecarlo-opentelemetry/",
                },
                "requires_dist": ["opentelemetry-api>=1.0"],
            }
        }
    ).encode("utf-8")


def test_pypi_fetch_success() -> None:
    """Mocked PyPI success path: _fetch_pypi parses the JSON and returns metadata."""
    print("\n== _fetch_pypi (mocked success) ==")

    from fetch_sdk_docs import _fetch_pypi

    payload = _make_pypi_payload(_FIXTURE_README)
    with patch("fetch_sdk_docs._fetch_bytes", return_value=payload):
        result = _fetch_pypi()

    check(
        "version extracted from PyPI payload",
        result.get("version") == "1.2.3",
        hint=f"got {result!r}",
    )
    check(
        "pypi_url extracted from project_urls.Homepage",
        result.get("pypi_url") == "https://pypi.org/project/montecarlo-opentelemetry/",
        hint=f"got {result!r}",
    )
    check(
        "requires_dist list preserved",
        result.get("requires_dist") == ["opentelemetry-api>=1.0"],
        hint=f"got {result!r}",
    )
    check(
        "_description carries README body",
        result.get("_description") == _FIXTURE_README,
        hint=f"got {result.get('_description')!r}",
    )


def test_pypi_fetch_failure() -> None:
    """Mocked PyPI failure path: _fetch_pypi propagates the error."""
    print("\n== _fetch_pypi (mocked failure) ==")

    import urllib.error

    from fetch_sdk_docs import _fetch_pypi

    with patch(
        "fetch_sdk_docs._fetch_bytes",
        side_effect=urllib.error.URLError("simulated network failure"),
    ):
        raised = False
        try:
            _fetch_pypi()
        except urllib.error.URLError:
            raised = True
        check(
            "URLError propagates out of _fetch_pypi",
            raised,
            hint="_fetch_pypi swallowed URLError instead of propagating",
        )


# ---------------------------------------------------------------------------
# test_e2e_fail_closed_on_pypi_failure
# ---------------------------------------------------------------------------


def test_e2e_fail_closed_on_pypi_failure() -> None:
    """End-to-end: when PyPI is unreachable, the script exits non-zero with
    source="error" and a JSON payload that includes guidance.
    """
    print("\n== end-to-end fail-closed on PyPI failure ==")

    # Monkey-patch the PYPI_URL via a wrapper script that imports and mutates
    # the module, then runs main(). We use a one-off Python invocation so the
    # subprocess shape mirrors normal usage but routes the fetch at an
    # unreachable host.
    runner = (
        "import sys; sys.path.insert(0, %r);"
        "import fetch_sdk_docs as m;"
        "m.PYPI_URL = 'https://example.invalid/notfound';"
        "m.main()"
    ) % str(SCRIPT_DIR)
    result = subprocess.run(
        [sys.executable, "-c", runner, "--quiet"],
        capture_output=True,
        text=True,
        timeout=30,
    )

    try:
        payload = json.loads(result.stdout)
    except json.JSONDecodeError as exc:
        check(
            "output is valid JSON",
            False,
            hint=f"JSONDecodeError: {exc}; stdout={result.stdout[:200]!r}",
        )
        return

    check("output is valid JSON", True)
    check(
        "source == 'error' on unreachable PyPI",
        payload.get("source") == "error",
        hint=f"source={payload.get('source')!r}",
    )
    check(
        "exit code is non-zero on error",
        result.returncode != 0,
        hint=f"returncode={result.returncode}",
    )
    check(
        "error payload has 'error' key",
        "error" in payload,
        hint=f"payload keys={list(payload.keys())}",
    )
    check(
        "error payload has 'guidance' key pointing at PyPI",
        isinstance(payload.get("guidance"), str)
        and "pypi.org/project/montecarlo-opentelemetry" in payload["guidance"],
        hint=f"guidance={payload.get('guidance')!r}",
    )


# ---------------------------------------------------------------------------
# main
# ---------------------------------------------------------------------------


def main() -> None:
    tests = [
        test_canonical_libraries,
        test_parse_supported_instrumentors,
        test_parse_failure_paths,
        test_build_success_shape,
        test_pypi_fetch_success,
        test_pypi_fetch_failure,
        test_e2e_fail_closed_on_pypi_failure,
    ]
    # Run every test even when an early one fails — `check()` raises on the
    # first failure inside a single test, but we want a complete pass/fail
    # summary across all tests when invoked standalone. pytest invokes each
    # `test_*` directly without going through main(), so per-test fast-fail
    # via AssertionError is the right behavior under pytest.
    for fn in tests:
        try:
            fn()
        except AssertionError:
            # Already logged by check(); continue so the summary covers all tests.
            pass
    print(f"\n{'=' * 40}")
    print(f"Results: {PASSED} passed, {FAILED} failed")
    sys.exit(0 if FAILED == 0 else 1)


if __name__ == "__main__":
    main()