# Evaluation Report

Evaluation of the `doca-bench-extension` skill before publication through Skill Evaluator.

This benchmark summarizes 3-Tier Evaluation from Skill Evaluator results for the skill. The goal is to document whether the skill is safe, discoverable, effective, and useful for agents before it is published for broader workflow use.

## Evaluation Summary

- Skill: `doca-bench-extension`
- Evaluation date: 2026-07-26
- Environment: `k8s-sandbox`
- Dataset: 4 evaluation tasks
- Attempts per task: 1
- Pass threshold: 50%
- Overall verdict: PASS

## Agents Used

- Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`)
- Codex (`openai/openai/gpt-5.5`)

## Metrics Used

Reported benchmark dimensions:

- Security: checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access.
- Correctness: checks whether the agent follows the expected workflow and produces the correct final output.
- Discoverability: checks whether the agent loads the skill when relevant and avoids using it when irrelevant.
- Effectiveness: checks whether the agent performs measurably better with the skill than without it.
- Efficiency: checks whether the agent uses fewer tokens and avoids redundant work.

Underlying evaluation signals used in this run:

- `security` (Security): checks for unsafe operations, secret leakage, and unauthorized access.
- `skill_execution` (Skill Execution): verifies that the agent loaded the expected skill and workflow.
- `skill_efficiency` (Efficiency): checks routing quality, decoy avoidance, and redundant tool usage.
- `accuracy` (Accuracy): grades final-answer correctness against the reference answer.
- `goal_accuracy` (Goal Accuracy): checks whether the overall user task completed successfully.
- `behavior_check` (Behavior Check): verifies expected behavior steps, including safety expectations.

## Test Tasks

The benchmark dataset contained 4 evaluation tasks:

- Positive tasks: 3 tasks where the skill was expected to activate.
- Negative tasks: 1 tasks where no skill was expected.
- Unlabeled tasks: 0 tasks where positive/negative intent could not be inferred.

Task composition is derived from the evaluation dataset when possible. Entries with `expected_skill` set are treated as positive skill-activation cases, while entries with `expected_skill: null` are treated as negative activation cases.

## Results

| Dimension | Num | Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`) | Codex (`openai/openai/gpt-5.5`) |
|---|---:|---:|---:|
| Security | 4 | 100% (+0%) | 100% (+0%) |
| Correctness | 4 | 100% (+65%) | 100% (+25%) |
| Discoverability | 4 | 98% (+23%) | 95% (+38%) |
| Effectiveness | 4 | 92% (+61%) | 99% (+62%) |
| Efficiency | 4 | 93% (+28%) | 100% (+64%) |

Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available.

## Tier 1: Static Validation Summary

Tier 1 validation passed with observations. Skill Evaluator ran 1 checks and found 7 total findings.

Top findings:

- MEDIUM SCHEMA/folder_hierarchy: Unexpected nesting depth for general skill (`skills/tools/doca-bench-extension`)
- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/tools/doca-bench-extension/SKILL.md`)
- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (`skills/tools/doca-bench-extension/SKILL.md`)
- MEDIUM SCHEMA/author_missing: Author not specified in metadata (`skills/tools/doca-bench-extension/SKILL.md`)
- LOW SCHEMA/unexpected_file: Unexpected 'SKILLCARD.yaml' in skill root (`skills/tools/doca-bench-extension/SKILLCARD.yaml`)

## Tier 2: Deduplication Summary

This tier was not run or did not produce findings in this report.

## Publication Recommendation

The skill is suitable to proceed toward Skill Evaluator publication based on this benchmark. Skill owners should keep this file with the skill and refresh it when the evaluation dataset, skill behavior, or target agents materially change.

# DOCA Bench Extension — Capabilities

**Where to start:** The DOCA Bench Extension framework is
the plug-in surface that augments
[`doca-bench`](../doca-bench/SKILL.md). The pattern overview
below names the recurring extension-class questions. Pick
the pattern first, then drill into the H2 that owns the
substance. For the *how* of executing each pattern, jump to
[TASKS.md](TASKS.md). For the parent tool's built-in modes
and inventory, see
[`doca-bench CAPABILITIES.md`](../doca-bench/CAPABILITIES.md).

This file is loaded by [`SKILL.md`](SKILL.md). It documents
*what an extension can do that built-in `doca-bench` modes
cannot*, *what the API surface looks like in broad strokes
(the shipped `doca_bench_cuda` reference exemplar is the
schema by example)*, *how the build / registration /
discovery flow works*, *what versions it ships in
(including the `DOCA_EXPERIMENTAL`-stability overlay)*,
*what its layered error and observability surfaces look
like*, and *the safety posture* the tool's role as a
loadable plug-in into a benchmark that touches the
dataplane forces.

## Pattern overview

Five recurring patterns drive every
`doca-bench-extension`-class question, and the H2s below
own one each. Pick the pattern, then jump to the owning H2.

| Pattern | Recognise it when … | Owning H2 |
| --- | --- | --- |
| **Extension-vs-built-in decision** | "Does `doca-bench` measure my workload natively, or do I need an extension?" | [`## Capabilities and modes`](#capabilities-and-modes) (Extension-as-exit-ramp) |
| **API surface design** | "What does my extension's entry-point need to look like?" "How do I copy the `doca_bench_cuda` reference?" | [`## Capabilities and modes`](#capabilities-and-modes) (API surface shape) |
| **Build / discovery flow** | "How does `doca-bench` find and load my extension at runtime?" "What does the `meson.build` look like?" | [`## Capabilities and modes`](#capabilities-and-modes) (Build / registration / discovery) |
| **`DOCA_EXPERIMENTAL` stability** | "The API is marked experimental — what does that mean for cross-release stability?" | [`## Version compatibility`](#version-compatibility) |
| **Failure mode** | "I built the extension, but `doca-bench` cannot find / load / call it." | [`## Error taxonomy`](#error-taxonomy) + [`## Observability`](#observability) |

Two non-patterns the agent must NOT collapse into the
above:

- *"Which built-in `doca-bench` mode do I pick?"* — that is
  not an extension question. Route to
  [`doca-bench`](../doca-bench/SKILL.md). Extensions exist
  only when the built-in modes don't cover the workload.
- *"How do I program a custom DOCA Flow / Comch / RMAX
  application?"* — that is a library-level question for
  the underlying primitive's skill. Extensions plug
  workloads INTO `doca-bench`; they are not a substitute
  for using the primitive directly.

## Capabilities and modes

The DOCA Bench Extension framework has three orthogonal
decisions the operator commits to in
[`## configure`](TASKS.md#configure), and one
non-decision (the parent-tool relationship) the agent
re-asserts every time.

### Extension-as-exit-ramp from built-in modes

The framework exists for ONE reason: a workload class that
no built-in `doca-bench` mode measures. The agent's first
question, before any extension-shaped answer, is *which
built-in mode might apply?* Per
[`doca-bench CAPABILITIES.md`](../doca-bench/CAPABILITIES.md),
the parent tool ships its own inventory of built-in workload
modes; the operator must walk that inventory before
committing to an extension.

Genuine extension cases the agent should validate against:

- **GPU-side workloads** that need to drive DOCA GPUNetIO
  RX / TX queues from CUDA kernels (the case the shipped
  `doca_bench_cuda` reference exemplar covers — RX /
  TX / bidir kernels in CUDA, accounting via
  `doca_bench_cuda_kernel_stats`, lifetime gated by a
  `stop_flag`).
- **Custom hardware engine workloads** that none of the
  built-in modes drives (the operator must show the
  agent why the built-in mode inventory does not apply).
- **Aggregated / multi-engine workloads** that combine
  primitives in a way no built-in mode expresses.

False extension cases — the agent should push back when
the operator proposes an extension for:

- *"I want different units / output format"* — that is a
  reporting question, not a workload question;
- *"I want to compare two built-in modes head to head"* —
  that is the parent tool's job;
- *"I want to add CLI flags"* — extensions don't add flags
  to `doca-bench`; they expose a workload class the parent
  can invoke.

### API surface shape

The shipped `doca_bench_cuda/doca_bench_cuda.h` reference
exemplar on the user's DOCA install is the schema by
example. The agent treats it as the *shape* of any custom
extension's surface, not as a memorized inventory of
required functions. Recurring axes the reference surface
demonstrates:

- A small set of **`DOCA_EXPERIMENTAL`-marked C entry
  points** (the reference exposes `*_init`,
  `*_device_query`, `*_device_synchronize`, and per-workload
  `*_start_*_kernel` family functions for nop / eth-recv /
  eth-send / eth-bidir kernels).
- Per-workload **settings structs** that the parent passes
  through (the reference uses
  `doca_bench_cuda_kernel_settings` for common fields —
  block count, threads-per-block, a `stop_flag`, a `stats`
  pointer — plus per-workload extensions that carry RX /
  TX queue handles, buffer address / mkey / size, and
  per-call timing parameters).
- Per-workload **accounting structs** (the reference uses
  `doca_bench_cuda_kernel_stats` — `jobs_processed`,
  `bytes_processed` — so the parent has a uniform way to
  read measurement output).
- A **lifetime contract** — the kernel runs until the
  `stop_flag` is set by the parent; the entry point's
  return code follows the `doca_error_t` convention.

The agent must surface that these are *shapes from the
shipped reference*, not a documented stable API the agent
can replicate from memory. A custom extension's surface
should be adapted from the reference; the reference is the
ground truth on the user's install, NOT this skill.

### Build / registration / discovery flow

Per the shipped `/opt/mellanox/doca/tools/bench_extension/meson.build`
and `/opt/mellanox/doca/tools/bench_extension/doca_bench_cuda/`
subtree, an extension is:

- a **versioned shared library** (the reference builds
  `doca_bench_cuda_impl` as a `shared_library` with
  `version : doca_version` and `soversion : doca_so_version`
  — this is the version-overlay rule in
  [`## Version compatibility`](#version-compatibility)
  applied at build time);
- compiled with the Meson `tool_cpp_args` + GPU compile
  flags the reference uses (when the extension is GPU-side);
- installed (`install : true`) into the DOCA library path
  on the user's install.

The exact runtime *discovery* mechanism the parent
`doca-bench` uses to locate and load the extension's
shared library — search path, naming convention,
registration call sequence — lives in the public DOCA
Bench documentation on `docs.nvidia.com` and the
installed `doca-bench` binary's `--help` /
extension-related flags. The agent does NOT invent the
mechanism from memory. The shipped `meson.build` provides
the build-time half of the contract; the parent tool
provides the runtime half; the agent points the operator
at both.

### Parent-tool relationship (non-decision)

Every extension lifetime is bounded by a `doca-bench`
invocation. The extension:

- has no standalone CLI of its own;
- is loaded, called, and unloaded by the parent;
- measures via the parent's reporting machinery (the
  `doca_bench_cuda_kernel_stats`-style accounting the
  reference uses is consumed by the parent);
- inherits all of the parent's safety preconditions per
  [`doca-bench CAPABILITIES.md ## Safety policy`](../doca-bench/CAPABILITIES.md#safety-policy).

The agent re-asserts this on every extension answer:
extensions plug INTO `doca-bench`; they do not replace it.

## Version compatibility

`doca-bench-extension` is **shipped with the DOCA release**
and is versioned *with the surrounding DOCA install*, not
on a separate cadence. Specifically:

- the shipped `doca_bench_cuda_impl` reference shared
  library carries `version : doca_version` and
  `soversion : doca_so_version` per the in-tree
  `meson.build` — the extension's ABI is the DOCA
  release's ABI;
- the extension's API surface is marked
  **`DOCA_EXPERIMENTAL`**. That carries a real cross-release
  stability cost the agent must surface:
  - the surface MAY change across DOCA releases without a
    deprecation window;
  - a custom extension built against one DOCA release's
    headers MUST be rebuilt against the new release's
    headers when DOCA is upgraded;
  - "rebuild on DOCA upgrade" is the **rule**, not the
    exception, for any custom extension.

This skill **does NOT** maintain its own version-handling
rules in parallel with
[`doca-version`](../../doca-version/SKILL.md). The agent
treats `doca-version` as the source of truth for:

- the four-way DOCA install match (host package, kernel
  module, firmware, parent `doca-bench`'s version);
- the BlueField-mode / device-mode questions;
- the cross-release behaviour-change questions.

The extension-specific overlay on top of that:

- **The `DOCA_EXPERIMENTAL` rebuild rule.** Every DOCA
  upgrade re-opens the question: does my custom
  extension still build, link, and load? The agent treats
  this as a smoke per [`TASKS.md ## test`](TASKS.md#test)
  on every DOCA upgrade, not as a "do it once" task.
- **`soversion` match.** The custom extension's
  `soversion` must match the running DOCA `so_version`
  for the dynamic linker / parent loader to accept it.
  Mismatches surface as load failures in
  [`## Error taxonomy`](#error-taxonomy) layer 3.
- **Toolchain version.** When the extension is GPU-side
  (CUDA), the CUDA toolkit version that built the
  extension matters; route the CUDA toolchain version
  questions to NVIDIA's public CUDA documentation on
  `docs.nvidia.com`, not to this skill.

Concretely the agent applies this rule in
[`TASKS.md ## configure`](TASKS.md#configure) (the
DOCA-version preconditions step) and in
[`TASKS.md ## debug`](TASKS.md#debug) (layer 6 — version).

## Error taxonomy

`doca-bench-extension` failures fall into seven layers. The
[TASKS.md `## debug`](TASKS.md#debug) verb walks them in
order; this section names them so the agent can route fast.

1. **Built-in-mode-would-have-sufficed.** The first layer
   is upstream of any extension build — the operator
   never needed an extension. The agent's first check
   when an extension is misbehaving is whether the
   workload class could have been expressed by a built-in
   `doca-bench` mode. Route back to
   [`doca-bench TASKS.md ## configure`](../doca-bench/TASKS.md#configure).
2. **Build-failed.** The extension's `meson.build` /
   compiler / linker reports an error. Common causes:
   wrong DOCA headers, wrong CUDA toolkit version,
   missing GPU build flags. Route the toolchain side to
   the public CUDA documentation; route the DOCA-side to
   [`doca-version TASKS.md ## debug`](../../doca-version/TASKS.md#debug)
   for the header-version question.
3. **Load-failed.** The library built, but the parent
   `doca-bench` cannot load it at runtime. Common causes:
   `soversion` mismatch with the running DOCA, library
   not on the dynamic linker's search path,
   architecture / ABI mismatch, missing CUDA runtime when
   the extension is GPU-side. Route the `ldd` /
   `LD_LIBRARY_PATH` / dynamic-linker side to
   [`doca-debug ## debug`](../../doca-debug/SKILL.md).
4. **Registration-mismatch.** The library loaded, but the
   parent cannot find the extension's expected entry
   points / version handshake. The agent's response is to
   re-check the extension's entry-point signatures against
   the shipped reference (the `DOCA_EXPERIMENTAL`-marked
   symbols in `doca_bench_cuda.h` are the shape) and
   confirm the parent's documented expectation per the
   public DOCA Bench documentation.
5. **Runtime-call-failed.** The extension was loaded and
   registered, but a per-workload call returned
   `doca_error_t` other than `DOCA_SUCCESS` or the kernel
   crashed / hung. The agent walks the extension's own
   logging plus the per-workload settings the parent
   passed in; common GPU-side cause is the `stop_flag`
   never being set so the kernel runs forever.
6. **Version.** Walk
   [`doca-version TASKS.md ## debug`](../../doca-version/TASKS.md#debug)
   end-to-end. Common extension-specific symptoms: the
   extension was built against a different DOCA release's
   headers than the parent is from (the
   `DOCA_EXPERIMENTAL` rebuild rule from
   [`## Version compatibility`](#version-compatibility))
   was missed; the CUDA toolkit version is wrong; the
   firmware doesn't expose what the extension assumes.
7. **Cross-cutting.** Hand off to
   [`doca-debug ## debug`](../../doca-debug/SKILL.md) and
   [`doca-setup ## debug`](../../doca-setup/TASKS.md#debug)
   for the env-side layers (driver, firmware, CUDA driver,
   GPU PCIe state, dynamic linker).

The full procedural ladder for each layer lives in
[`TASKS.md ## debug`](TASKS.md#debug); this section names
them so the agent can route on the FIRST symptom.

## Observability

Three kinds of evidence the framework emits, and how the
agent is expected to read each:

- **the parent `doca-bench`'s own output**, including its
  extension-load logging and its consumption of the
  extension's accounting structs (the reference's
  `doca_bench_cuda_kernel_stats` — `jobs_processed`,
  `bytes_processed`). The parent is the canonical sink
  for the extension's measurement; the extension's
  internal logging is a debugging aid, NOT the
  authoritative output.
- **the extension's own per-workload logging** during the
  smoke (the agent treats stdout / stderr from the
  extension as raw signal; quoting verbatim is the rule).
  When the extension is GPU-side, also capture relevant
  CUDA driver logging from `dmesg`.
- **the host / device cross-cutting evidence** the
  preconditions require: DOCA version, BlueField /
  ConnectX generation, firmware version, CUDA toolkit
  version (when GPU-side), the extension's
  `version` / `soversion`, the `ldd` output for the
  built library. Partial captures are not actionable.

For the cross-cutting host / device observability surfaces
(dynamic linker, CUDA driver, PCIe, firmware), route to
[`doca-debug ## Observability`](../../doca-debug/SKILL.md)
and [`doca-setup TASKS.md ## test`](../../doca-setup/TASKS.md#test).

The skill explicitly does NOT add a streaming-telemetry
export of extension state — the extension is invoked by
the parent and reports through the parent's machinery.

## Safety policy

> **Hardware-safety meta-policy applies.** Every operator
> action below inherits the safety contract defined in
> [`doca-hardware-safety ## Safety policy`](../../doca-hardware-safety/CAPABILITIES.md#safety-policy):
> the pre-flight inventory (DOCA version, firmware,
> kernel module, BlueField mode, OOB access, dataplane
> co-tenancy), the OOB / blast-radius rules, the
> change-class classification, and the smoke-before-bulk
> discipline. This section names ONLY the
> `doca-bench-extension`-specific overlay on top of that
> meta-policy.

`doca-bench-extension` is an unusual surface: it asks the
operator to LOAD CUSTOM CODE into a benchmark that touches
the device. That has four operational consequences the
agent must surface:

- **The extension runs in the parent `doca-bench`'s
  address space.** A bug in the extension can crash, hang,
  or corrupt the parent's measurement. The
  `DOCA_EXPERIMENTAL` marking on the API means the parent
  does not promise the surface won't change, but it also
  means the operator has accepted the responsibility for
  the loaded code's correctness.
- **The shipped `doca_bench_cuda` reference is GPU-side,
  and GPU-side workloads can hang the device.** A CUDA
  kernel that never sets its `stop_flag` (the reference's
  `*_kernel_settings` field) runs forever; recovering
  requires resetting the GPU. The OOB / reset
  precondition from
  [`doca-hardware-safety ## Safety policy`](../../doca-hardware-safety/CAPABILITIES.md#safety-policy)
  is non-negotiable when the extension drives a GPU.
- **All of `doca-bench`'s safety constraints apply.** Per
  [`doca-bench CAPABILITIES.md ## Safety policy`](../doca-bench/CAPABILITIES.md#safety-policy),
  the parent tool's preconditions (firmware, BlueField
  mode, dataplane co-tenancy) inherit into the extension
  invocation. Extensions add MORE preconditions; they
  never remove any.
- **The smoke-before-bulk rule from
  [`doca-hardware-safety ## Safety policy`](../../doca-hardware-safety/CAPABILITIES.md#safety-policy)
  applies twice here.** First on the build: confirm the
  extension builds cleanly and the parent can load it
  with a no-op kernel (the reference's
  `doca_bench_cuda_start_nop_kernel` is the canonical
  smoke for this) before any I/O-side workload. Second
  on the workload: confirm one minimal invocation
  completes cleanly before scaling up. Skipping either
  smoke and going straight to a full GPUNetIO RX / TX
  workload is the canonical failure mode.

In addition:

- **OOB access.** When the extension drives a GPU, OOB
  reset access is required. When the extension touches
  BlueField Flow / Comch state, the BlueField OOB
  preconditions from the parent's safety policy apply.
- **No mutation of the parent `doca-bench` binary.**
  Extensions add code at load time; they do not patch
  the parent.
- **Capture before retry.** Per
  [`doca-hardware-safety ## Error taxonomy`](../../doca-hardware-safety/CAPABILITIES.md#error-taxonomy):
  if an extension load or invocation fails, capture the
  parent's logs, the extension's logs, `ldd` output,
  and the DOCA + toolchain version stack BEFORE
  retrying. Retrying without capture is the canonical
  lost-signal failure.
- **Stop-flag discipline.** Per the reference's
  `stop_flag` field, every long-running extension kernel
  must respect a parent-set stop signal. The agent
  refuses to recommend a kernel design that has no
  bounded termination.

The full procedural application of the safety overlay
(when to abort, when to escalate, what to capture) lives
in [`TASKS.md ## debug`](TASKS.md#debug) and
[`TASKS.md ## test`](TASKS.md#test) plus the
[`doca-debug`](../../doca-debug/SKILL.md) cross-cutting
debug ladder. This section names the rules that constrain
those verbs for the bench-extension framework
specifically.

## Description: <br>
Use this skill when the operator is authoring, building, loading, or debugging a custom doca-bench plug-in — a versioned shared library with DOCA_EXPERIMENTAL-marked C entry points that doca-bench loads to measure a workload class its built-in modes do not cover, with doca_bench_cuda as the shipped reference exemplar. <br>

This skill is ready for commercial/non-commercial use. <br>

## Owner
NVIDIA <br>

### License/Terms of Use: <br>
Apache 2.0 AND CC-BY-4.0 <br>
## Use Case: <br>
Developers and performance engineers who use doca-bench for built-in workload modes and need to author, build, load, or debug a custom extension plug-in for workload classes that built-in modes do not cover. <br>

### Deployment Geography for Use: <br>
Global <br>

## Requirements / Dependencies: <br>
**Requires API Key or External Credential:** [No] <br>
**Credential Type(s):** [None] <br>

Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate. <br>

## Known Risks and Mitigations: <br>
Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills. <br>
Mitigation: Review and scan skill before deployment. <br>

## Reference(s): <br>
- [DOCA SDK Documentation](https://docs.nvidia.com/doca/sdk/index.html) <br>
- [DOCA Samples](https://github.com/NVIDIA-DOCA/doca-samples) <br>
- [DOCA Platform Framework](https://github.com/NVIDIA/doca-platform) <br>


## Skill Output: <br>
**Output Type(s):** [Shell commands, Configuration instructions, Code] <br>
**Output Format:** [Markdown with inline bash and C code blocks] <br>
**Output Parameters:** [1D] <br>
**Other Properties Related to Output:** [None] <br>

## Evaluation Agents Used: <br>
- Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`) <br>
- Codex (`openai/openai/gpt-5.5`) <br>



## Evaluation Tasks: <br>
Evaluated against 4 internal evaluation tasks (3 positive skill-activation, 1 negative). <br>

## Evaluation Metrics Used: <br>
Reported benchmark dimensions: <br>
- Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access. <br>
- Correctness: Checks whether the agent follows the expected workflow and produces the correct final output. <br>
- Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant. <br>
- Effectiveness: Checks whether the agent performs measurably better with the skill than without it. <br>
- Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work. <br>

Underlying evaluation signals used in this run: <br>
- `security`: Checks for unsafe operations, secret leakage, and unauthorized access. <br>
- `skill_execution`: Verifies that the agent loaded the expected skill and workflow. <br>
- `skill_efficiency`: Checks routing quality, decoy avoidance, and redundant tool usage. <br>
- `accuracy`: Grades final-answer correctness against the reference answer. <br>
- `goal_accuracy`: Checks whether the overall user task completed successfully. <br>
- `behavior_check`: Verifies expected behavior steps, including safety expectations. <br>



## Evaluation Results: <br>
| Dimension | Num | Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`) | Codex (`openai/openai/gpt-5.5`) |
|---|---:|---:|---:|
| Security | 4 | 100% (+0%) | 100% (+0%) |
| Correctness | 4 | 100% (+65%) | 100% (+25%) |
| Discoverability | 4 | 98% (+23%) | 95% (+38%) |
| Effectiveness | 4 | 92% (+61%) | 99% (+62%) |
| Efficiency | 4 | 93% (+28%) | 100% (+64%) |

## Skill Version(s): <br>
56cf891 (source: git SHA, committed 2026-07-26) <br>

## Ethical Considerations: <br>
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>

(For Release on NVIDIA Platforms Only) <br>
Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail). <br>

---
license: Apache-2.0
name: doca-bench-extension
description: >
  Use this skill when the operator is authoring, building, loading,
  or debugging a custom doca-bench plug-in — a versioned shared
  library with DOCA_EXPERIMENTAL-marked C entry points that
  doca-bench loads to measure a workload class its built-in modes
  do not cover, with doca_bench_cuda as the shipped reference
  exemplar. Trigger even when the user does not say
  "doca-bench-extension" or "doca_bench_cuda" — typical implicit
  phrasings include "no built-in doca-bench mode fits my workload",
  "how do I benchmark a CUDA GPUNetIO RX/TX kernel", "doca-bench
  cannot find or load my custom .so", "extension exported symbols
  do not match what the parent expects", "soversion mismatch after
  a DOCA upgrade", or "my GPU kernel hangs because stop_flag was
  never set". Refuse and route elsewhere for questions about which
  built-in doca-bench mode to pick, DOCA GPUNetIO programming
  semantics, CUDA toolkit installation, or contributor work on
  in-tree extensions — those belong to other skills.
metadata:
  kind: tool
compatibility: >
  Requires DOCA SDK installed at /opt/mellanox/doca on Linux
  (Ubuntu 22.04/24.04 or RHEL/SLES) with a BlueField DPU or
  ConnectX NIC. Source tree: `/opt/mellanox/doca/tools/bench_extension/`
  (underscored, NOT kebab-case); the built shared library
  `libdoca_bench_cuda_impl.so` lands in the platform libdir on a
  binary install. Also needs `pkg-config doca-common` and, for
  the GPU-side reference exemplar (DOCA GPUNetIO RX/TX kernels),
  an NVIDIA GPU + matching CUDA toolkit.
---

# DOCA Bench Extension

**Where to start:** This is a tool skill for the **extension /
plug-in framework** that augments
[`doca-bench`](../doca-bench/SKILL.md) — NOT a workload-shape
skill on its own. Open [`TASKS.md`](TASKS.md) and start at
[`## configure`](TASKS.md#configure) to commit to the three-axis
decision (workload class is genuinely outside doca-bench's
built-in modes × extension API surface fits × parent-tool
co-load is acceptable), then [`## build`](TASKS.md#build) for
how a custom extension is compiled and laid out, then
[`## run`](TASKS.md#run) for how `doca-bench` discovers and
invokes the extension, then [`## test`](TASKS.md#test) for the
smoke-before-bulk loop the agent applies to every new
extension. Open [`CAPABILITIES.md`](CAPABILITIES.md) when the
question is *what an extension can do that built-in
`doca-bench` modes cannot*, *what the extension API surface
looks like in broad strokes (the `DOCA_EXPERIMENTAL` C entry
points the shipped reference exposes)*, *how the
build / registration / discovery flow works*, or *how the
extension's lifetime is bounded by the parent `doca-bench`
invocation*. If `doca-bench` itself is the question, route to
[`doca-bench`](../doca-bench/SKILL.md). If the question is
"which built-in `doca-bench` mode do I pick?", that is also
[`doca-bench`](../doca-bench/SKILL.md) — extensions are the
*exit ramp* for workloads built-in modes do not cover.

## Example questions this skill answers well

- *"My workload class is `<X>` — does `doca-bench` measure it
  natively, or do I need an extension?"* — the
  extension-vs-built-in decision question. The agent walks
  the user back to [`doca-bench`](../doca-bench/SKILL.md)'s
  built-in mode inventory FIRST and only routes to the
  extension framework when no built-in mode applies.
- *"I want to benchmark a CUDA / GPU-side workload that
  drives DOCA GPUNetIO RX and TX queues. Where do I start?
  Is there a reference extension I can copy?"* — the agent
  surfaces the shipped `doca_bench_cuda` extension under
  `/opt/mellanox/doca/tools/bench_extension/doca_bench_cuda/` as the
  reference exemplar and walks the operator through its
  API surface and build shape.
- *"How does `doca-bench` actually discover and load my
  custom extension at runtime? Is it a versioned shared
  library? What does my entry-point need to look like?"* —
  the build / registration / discovery flow question. The
  agent walks the Meson-built shared library shape, the
  versioning, and the parent-tool's runtime discovery path
  (which the agent does NOT invent from memory — the
  shipped extension's `meson.build` and the public DOCA Bench
  documentation on `docs.nvidia.com` are the source of
  truth).
- *"The API headers I have are marked `DOCA_EXPERIMENTAL`.
  What does that mean for my extension's stability across
  DOCA releases? Am I going to have to rebuild it every
  release?"* — the experimental-surface and version
  compatibility question.
- *"Once I build my extension, what is the cheapest possible
  smoke I can run before pointing my real workload at it?
  How do I know `doca-bench` actually loaded it, called
  into it, and that the call returned the data the parent
  tool expected?"* — the smoke-before-bulk question.
- *"My custom extension builds, but `doca-bench` says it
  cannot find / load / call it. Where do I look first?"* —
  the layered-debug question that distinguishes
  build-failures, load-failures, registration-mismatches,
  and runtime-call-failures.

## Audience

Experienced AI agents and platform / performance engineers
who already use [`doca-bench`](../doca-bench/SKILL.md) for
the built-in workload modes and now have a workload class
that the built-in modes do not cover. Readers are expected
to be comfortable with native build systems (Meson, in this
codebase), shared-library packaging on Linux, and the
`DOCA_EXPERIMENTAL` API stability contract. If the user
asks about GPU-side benchmarking via the shipped
`doca_bench_cuda` reference extension, the reader is also
expected to be familiar with DOCA GPUNetIO and CUDA toolchain
basics — those domains live in their own skills, not here.

This skill is NOT for:

- operators who can express their workload with one of
  `doca-bench`'s built-in modes — that is
  [`doca-bench`](../doca-bench/SKILL.md);
- operators who want to benchmark a different DOCA primitive
  (Flow, Comch, RMAX) via that primitive's own
  measurement tool — route to that tool;
- contributors authoring or modifying the in-tree extensions
  themselves (this skill is for external operators consuming
  the framework, not for internal DOCA contributors).

## Language scope

A `doca-bench` extension surfaces as:

1. A **versioned shared library** on Linux (`.so` with
   `soversion` matching the DOCA release), built via the
   `doca-bench-extension` Meson rules in the shipped
   `/opt/mellanox/doca/tools/bench_extension/meson.build` and the
   per-extension subdirectory (the reference exemplar is
   `doca_bench_cuda/`).
2. A small set of **`DOCA_EXPERIMENTAL`-marked C entry
   points** that the parent `doca-bench` invokes — i.e. the
   API surface declared in the extension's header file.
   The shipped `doca_bench_cuda/doca_bench_cuda.h` is the
   reference for what that surface shape looks like in
   practice (`*_init`, `*_device_query`,
   `*_device_synchronize`, and per-workload kernel-start
   entry points such as `*_start_nop_kernel`,
   `*_start_eth_recv_kernel`, `*_start_eth_send_kernel`,
   `*_start_eth_bidir_kernel`).
3. A set of **per-workload settings structs** that the
   parent passes through (e.g. the reference exemplar's
   `doca_bench_cuda_kernel_settings`,
   `doca_bench_cuda_eth_rx_kernel_settings`,
   `doca_bench_cuda_eth_tx_kernel_settings`,
   `doca_bench_cuda_eth_bidir_kernel_settings` carry block
   counts, threads-per-block, RX / TX queues, buffer
   address / mkey / size, a stop flag, and a stats
   pointer).

The skill itself is Markdown. The user's extension source
is whatever language the workload requires (C / C++ / CUDA
in the reference case). The agent does NOT prescribe a
language beyond what the shipped reference demonstrates.

## When to load this skill

Load `doca-bench-extension` when ANY of the following is
true:

- the user explicitly mentions `doca-bench-extension`, the
  `doca_bench_cuda` reference extension, the
  `doca_bench_cuda_impl` shared library, or any of the
  `DOCA_EXPERIMENTAL` extension entry points;
- the user has confirmed (via
  [`doca-bench TASKS.md ## configure`](../doca-bench/TASKS.md#configure))
  that none of `doca-bench`'s built-in workload modes
  measures the class they want, and an extension is the
  exit ramp;
- the user wants to copy / extend the shipped
  `doca_bench_cuda` reference into a custom GPU-side
  workload extension;
- the user is debugging why `doca-bench` cannot find / load
  / call a custom extension they built.

Co-load this skill with:

- [`doca-bench`](../doca-bench/SKILL.md) (the parent tool —
  ALWAYS co-loaded; extensions only have value as
  plug-ins into `doca-bench`);
- [`doca-version`](../../doca-version/SKILL.md) (the
  `DOCA_EXPERIMENTAL` surface is versioned with DOCA; the
  extension's `soversion` is the DOCA `soversion`; the
  four-way version match applies);
- [`doca-gpunetio`](../../libs/doca-gpunetio/SKILL.md) when
  the extension is GPU-side and uses GPUNetIO RX / TX
  queues like the reference exemplar (route the GPUNetIO
  semantics there, not here);
- [`doca-debug`](../../doca-debug/SKILL.md) and
  [`doca-setup`](../../doca-setup/SKILL.md) for the
  env-side debug ladder (driver, firmware, CUDA toolkit,
  dynamic linker).

Do NOT load this skill when the user's workload fits a
`doca-bench` built-in mode — extensions add cost (build
toolchain, version churn, the experimental-surface
contract); the built-in modes are always the first answer to
try.

## What this skill provides

Three companion files in this directory, each owning a
different question shape:

- [`SKILL.md`](SKILL.md) — this file. Audience, scope,
  loading order, related skills. Routes everything else.
- [`CAPABILITIES.md`](CAPABILITIES.md) — *what an extension
  can do that the built-in modes cannot*, *what the API
  surface looks like in broad strokes*, *how the
  build / registration / discovery flow works*, *what
  versions it ships in (including the
  `DOCA_EXPERIMENTAL`-stability overlay on top of
  `doca-version`)*, *the layered error taxonomy*,
  observability, and the safety policy overlay.
- [`TASKS.md`](TASKS.md) — the procedural verbs
  (`configure`, `build`, `run`, `test`, `debug`, etc.) plus
  a `doca-bench-extension`-specific command appendix and
  the agent-side `use` workflow that consumes the captured
  extension run.

The combined skill teaches an AI agent to drive the
*extension-author-and-wire-in class* of `doca-bench`
questions: confirm an extension is needed at all; locate
the shipped reference exemplar
(`/opt/mellanox/doca/tools/bench_extension/doca_bench_cuda/`); copy
its build + API surface shape; build a versioned shared
library that matches the DOCA release; smoke that the
parent `doca-bench` actually loads it; diagnose layered
failures when it does not.

## What this skill deliberately does not ship

- **Inventory of `doca-bench`'s built-in workload modes.**
  That belongs to [`doca-bench`](../doca-bench/SKILL.md).
  This skill is the exit ramp for what the built-in modes
  do not cover; it does not duplicate the parent's mode
  inventory.
- **Invented `DOCA_EXPERIMENTAL` entry-point names beyond
  what the shipped reference declares.** The shipped
  `doca_bench_cuda/doca_bench_cuda.h` on the user's
  install is the reference for what the surface shape
  looks like; the agent does not assert other extensions
  exist with specific signatures.
- **A canonical "right" extension layout.** The shipped
  `doca_bench_cuda` reference IS the canonical layout;
  rewriting it here would drift from the source of truth.
  The agent points the operator at the shipped tree and
  walks the operator through *adapting* it.
- **A documented runtime discovery mechanism the agent
  invents.** The exact mechanism `doca-bench` uses to
  locate and load extensions (search path, naming
  convention, registration call) lives in the public DOCA
  Bench documentation on `docs.nvidia.com` and the
  installed `doca-bench` binary. The agent points the
  operator there rather than asserting a mechanism from
  memory.
- **DOCA GPUNetIO programming details.** When the
  extension is GPU-side (as the reference exemplar is),
  the GPUNetIO RX / TX queue semantics live in
  [`doca-gpunetio`](../../libs/doca-gpunetio/SKILL.md);
  this skill cross-links rather than duplicates.
- **CUDA toolchain installation guidance.** Route to the
  public NVIDIA CUDA Toolkit documentation on
  `docs.nvidia.com`; this skill does not duplicate it.
- **Library-internal `doca-bench` invocation details
  unrelated to extensions.** The parent's CLI flags,
  pipeline shapes, and built-in workload classes belong
  to [`doca-bench`](../doca-bench/SKILL.md).

## Loading order

When a `doca-bench-extension` question arrives:

1. Confirm DOCA is installed AND `doca-bench` is reachable
   on the user's install — if not, route to
   [`doca-setup`](../../doca-setup/SKILL.md);
2. **Confirm none of `doca-bench`'s built-in modes covers
   the workload class** — if any of them does, route back
   to [`doca-bench TASKS.md ## configure`](../doca-bench/TASKS.md#configure)
   and stop. Extensions are the exit ramp, not the first
   answer;
3. Read [`CAPABILITIES.md`](CAPABILITIES.md) to commit to
   the three-axis decision and walk the reference
   exemplar's API surface shape;
4. Read [`TASKS.md`](TASKS.md) and walk
   `## configure → ## build → ## run → ## test → ## debug`
   in that order; do NOT start with `## run` without the
   build precondition step.

## Related skills

Cross-link conventions follow the bundle's relative path
contract from `tools/<X>/`:

- [`doca-bench`](../doca-bench/SKILL.md) — the parent tool.
  **ALWAYS co-loaded.** Extensions are plug-ins into
  `doca-bench`; they do not replace it, they do not have a
  standalone CLI, they do not measure anything without the
  parent invoking them. Every question on this skill
  presupposes the parent.
- [`doca-version`](../../doca-version/SKILL.md) — the
  `DOCA_EXPERIMENTAL` surface is versioned with DOCA; the
  extension's `soversion` matches the DOCA release per
  the shipped `meson.build`. The four-way version match
  applies; rebuilding the extension across DOCA upgrades
  is the rule, not the exception.
- [`doca-gpunetio`](../../libs/doca-gpunetio/SKILL.md) —
  when the extension is GPU-side and uses GPUNetIO RX /
  TX queues like the reference `doca_bench_cuda`. Route
  the GPUNetIO semantics there.
- [`doca-setup`](../../doca-setup/SKILL.md) — DOCA install
  posture (does `doca-bench` exist? does the
  `doca_bench_cuda_impl` reference library exist? is the
  CUDA toolchain installed when needed?).
- [`doca-debug`](../../doca-debug/SKILL.md) — the
  cross-cutting debug ladder for env-side issues (dynamic
  linker, library search path, CUDA driver / toolkit,
  firmware).
- [`doca-public-knowledge-map`](../../doca-public-knowledge-map/SKILL.md)
  — routing to the public DOCA Bench / DOCA GPUNetIO pages
  on `docs.nvidia.com` and the release notes for the
  documented extension lifecycle / discovery mechanism.
- [`doca-structured-tools-contract`](../../doca-structured-tools-contract/SKILL.md)
  — the agent's detect → prefer → fall back → report
  contract for the structured helpers
  (`doca-env --json`, `doca-capability-snapshot`,
  `version-matrix.json`) the build / load preconditions
  rely on.
- [`doca-hardware-safety`](../../doca-hardware-safety/SKILL.md)
  — the canonical hardware-safety meta-policy that
  [`CAPABILITIES.md ## Safety policy`](CAPABILITIES.md#safety-policy)
  overlays. Extensions are external code loaded into
  `doca-bench`; the safety implications of loading
  experimental code into a benchmark that touches the
  dataplane / device are real.

This skill assumes the user has built shared libraries on
Linux before and knows what a Meson build is. Background
material on those topics belongs in the toolchain docs, not
in this skill.

# SKILLCARD.yaml — provisional, signed at NVCARPS scan time
# Schema: https://github.com/NVIDIA/skills/blob/main/docs/skill-cards.mdx
# This file is generated by `runner/build_skillcards.py`; do not hand-edit.
# Re-run after any change to SKILL.md frontmatter or the deep-E2E baseline.

schema_version: "1"

identity:
  name: "doca-bench-extension"
  artifact_id: "doca-bench-extension"
  kind: "tool"
  version: "TBD-at-sign-time"          # NVCARPS fills from the source ref
  signature: "TBD-at-sign-time"        # NVCARPS-generated SHA + ed25519 sig
  signed_by: "TBD-at-sign-time"        # NVCARPS identity (NVIDIA-PSIRT signing key)

description: "Use this skill when the operator is authoring, building, loading,\nor debugging a custom doca-bench plug-in — a versioned shared\nlibrary with DOCA_EXPERIMENTAL-marked C entry points that\ndoca-bench loads to measure a workload class its built-in modes\ndo not cover, with doca_bench_cuda as the shipped reference\nexemplar. Trigger even when the user does not say\n\"doca-bench-extension\" or \"doca_bench_cuda\" — typical implicit\nphrasings include \"no built-in doca-bench mode fits my workload\",\n\"how do I benchmark a CUDA GPUNetIO RX/TX kernel\", \"doca-bench\ncannot find or load my custom .so\", \"extension exported symbols\ndo not match what the parent expects\", \"soversion mismatch after\na DOCA upgrade\", or \"my GPU kernel hangs because stop_flag was\nnever set\". Refuse and route elsewhere for questions about which\nbuilt-in doca-bench mode to pick, DOCA GPUNetIO programming\nsemantics, CUDA toolkit installation, or contributor work on\nin-tree extensions — those belong to other skills."

source:
  repo: "NVIDIA-DOCA/doca-skills"
  path: "skills/tools/doca-bench-extension/SKILL.md"
  branch: "main"
  license: "Apache-2.0 AND CC-BY-4.0"

provenance:
  scan_run_id: "TBD-at-sign-time"      # NVCARPS scan run UUID
  signed_at:   "TBD-at-sign-time"      # ISO-8601 timestamp from NVCARPS
  upstream_catalog: "https://github.com/NVIDIA/skills"
  upstream_component: "doca"           # see components.d/doca.yml

quality:
  baseline:
    verdict: "PASS"
    blocker_findings: 0
    suite:    "runner/reports/2026-05-26_r15_full_e2e"
    aggregate: "runner/reports/2026-05-26_r15_full_e2e/aggregate.json"
    summary:   "runner/reports/2026-05-26_r15_full_e2e/summary.md"
  regression_contract:
    file:    "runner/baseline_grades.json"
    rule:    "no PASS->FAIL on variant C since last green run"
    gate:    "ci/check-no-regression.py --variant C"

behavior_boundaries:
  documentation_only: true             # bundle ships no runnable code
  hardware_touching_changes_routed_to: "skills/doca-hardware-safety/"
  version_policy_routed_to: "skills/doca-version/"
  external_productized_software_routed_to: "skills/doca-public-knowledge-map/"
  see_also:
    - "AGENTS.md"                       # universal rules
    - "AUTHORING.md"                    # per-skill class-shape contract
    - "BENCHMARK.md"                    # how this skill is graded
    - "SECURITY.md"                     # safety-bug intake

contact:
  bug_intake: "https://github.com/NVIDIA-DOCA/doca-skills/issues"
  safety_bug_intake: "https://github.com/NVIDIA-DOCA/doca-skills/issues/new?labels=safety-bug"
  security:   "https://www.nvidia.com/en-us/security/"

# DOCA Bench Extension — Tasks

**Where to start:** The verbs that carry real workflow
content for `doca-bench-extension` are `## configure`,
`## build`, `## run`, `## test`, and `## debug`. `## modify`
is substantive for this tool because the operator IS
modifying the shipped reference extension or authoring one
from scratch. `## install` and `## use` carry routing stubs
and a tightly-scoped agent-side workflow.

This file is loaded by [`SKILL.md`](SKILL.md) after
[`CAPABILITIES.md`](CAPABILITIES.md). It walks the agent
through the task verbs every artifact in this bundle exposes.

## install

The bench-extension framework itself is **shipped pre-built**
with the DOCA install — the reference
`doca_bench_cuda_impl` shared library and the in-tree
`/opt/mellanox/doca/tools/bench_extension/` source tree (which an
operator copies forward into a custom extension) ship with
DOCA when the bench / GPU components are present in the
install profile.

Routing for nearby "install" questions:

- *"`doca-bench` is not on my system."* → that is a parent
  tool install question. Route to
  [`doca-setup ## install`](../../doca-setup/TASKS.md#configure)
  and [`## no-install`](../../doca-setup/TASKS.md#no-install).
- *"The reference `doca_bench_cuda_impl` library is
  missing."* → same routing — the install profile that
  excluded the bench GPU component will have excluded the
  reference extension too.
- *"I need to install the CUDA toolkit for my GPU-side
  extension."* → not a DOCA install question. Route to
  the public CUDA Toolkit documentation on
  `docs.nvidia.com`.
- *"I want to install a newer version of the extension
  headers without upgrading DOCA."* → the bench extension
  surface is versioned WITH DOCA per
  [`CAPABILITIES.md ## Version compatibility`](CAPABILITIES.md#version-compatibility);
  the `DOCA_EXPERIMENTAL` rebuild rule says there is no
  independent upgrade path. Route the cross-release
  decision through
  [`doca-version TASKS.md ## configure`](../../doca-version/TASKS.md#configure).

## configure

`configure` for the bench-extension framework is *"commit
to the three-axis decision (extension is genuinely needed
× the reference's API surface fits × the DOCA toolchain
preconditions hold) AND validate the parent-tool
relationship BEFORE any extension build work"*. Skipping
any step is the canonical failure mode.

Steps the agent should walk the user through, in order:

1. **Confirm DOCA is installed and `doca-bench` is healthy.**
   Run [`doca-setup ## test`](../../doca-setup/TASKS.md#test);
   then confirm the parent `doca-bench` is reachable per
   [`doca-bench TASKS.md ## test`](../doca-bench/TASKS.md#test).
   The extension has no value without a working parent.
2. **Re-validate the extension-is-needed decision.** Per
   [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes)
   (Extension-as-exit-ramp): walk
   [`doca-bench TASKS.md ## configure`](../doca-bench/TASKS.md#configure)
   FIRST. Confirm no built-in workload mode covers the
   class. The agent must NOT skip this — extensions add
   real cost (toolchain, version churn, experimental
   surface).
3. **Locate the shipped reference extension.** Confirm
   the operator's install contains
   `/opt/mellanox/doca/tools/bench_extension/doca_bench_cuda/`
   (the reference exemplar source tree). The agent must
   ask the operator to confirm the path on their install
   rather than asserting one.
4. **Read the reference exemplar's API surface.** Per
   [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes)
   (API surface shape): the shipped `doca_bench_cuda.h`
   on the operator's install is the schema by example.
   Walk the operator through the `DOCA_EXPERIMENTAL`
   entry-point family, the per-workload settings structs,
   the accounting struct, and the lifetime contract
   (the `stop_flag`).
5. **Decide if a NEW extension is needed or if the
   shipped reference suffices.** If the workload is
   already covered by the reference's nop / eth-recv /
   eth-send / eth-bidir kernels (i.e. a GPUNetIO RX / TX
   class workload), the operator may just use the
   reference; no custom extension is needed. If the
   workload is genuinely outside the reference, the
   operator authors a NEW extension by COPYING the
   reference subtree and adapting it.
6. **Commit to the toolchain preconditions.** When the
   extension is GPU-side: the CUDA toolkit version
   compatible with the running DOCA / driver, the GPU
   architecture flags, the GPUNetIO preconditions per
   [`doca-gpunetio`](../../libs/doca-gpunetio/SKILL.md).
   When the extension is non-GPU: the relevant
   primitive's preconditions.
7. **Capture the version stack.** Per
   [`CAPABILITIES.md ## Version compatibility`](CAPABILITIES.md#version-compatibility):
   DOCA version, BlueField / ConnectX generation,
   firmware version, BlueField mode, CUDA toolkit version
   (when applicable), the parent `doca-bench`'s version,
   the shipped reference extension's version (matches the
   DOCA `version`). Partial captures break the
   reproducibility leg of the safety policy.
8. **Sanity check before any build.** Confirm with the
   user: is the workload genuinely outside `doca-bench`'s
   built-in modes? Is the reference exemplar reachable?
   Are the toolchain preconditions met? Is OOB / reset
   access available (when GPU-side)? If any answer is
   unclear, stop and ask.

Do not invent build flags, entry-point function names, or
runtime discovery paths beyond what the shipped reference
demonstrates and what the public DOCA Bench documentation
on `docs.nvidia.com` describes.

## build

`build` for the bench-extension framework is *"build a
versioned shared library that matches the running DOCA's
`soversion`, adapted from the shipped reference exemplar's
`meson.build` and source layout"*. The skill walks the
shape, not a verbatim recipe.

Steps the agent should walk the user through, in order:

1. **Copy the reference exemplar subtree to the
   operator's working tree.** The reference is
   `/opt/mellanox/doca/tools/bench_extension/doca_bench_cuda/` on
   the DOCA install. The operator copies the subtree,
   not edits it in place — the shipped reference must
   stay pristine for cross-check during
   [`## debug`](#debug).
2. **Adapt the `meson.build`.** Per the shipped
   `/opt/mellanox/doca/tools/bench_extension/meson.build` shape:
   - the shared library MUST carry
     `version : doca_version` and
     `soversion : doca_so_version` so the parent loader
     accepts it — this is the
     [`CAPABILITIES.md ## Version compatibility`](CAPABILITIES.md#version-compatibility)
     rule at build time;
   - the compile args (the reference uses
     `tool_cpp_args = [base_cpp_args, gpu_compile_flags]`)
     reflect what the parent's build system expects;
   - GPU extensions inherit `gpu_dependencies`;
   - `install : true` places the library where the parent
     loader can find it at runtime.
   The agent must NOT invent build flags; the reference
   `meson.build` is the source of truth.
3. **Adapt the source files.** Per
   [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes)
   (API surface shape): keep the
   `DOCA_EXPERIMENTAL`-marked entry-point shape, adapt
   the per-workload settings structs / accounting struct
   / lifetime contract for the new workload. The
   reference's `*_init`, `*_device_query`,
   `*_device_synchronize` shape is the lifecycle
   blueprint; new workloads layer their own
   `*_start_<workload>_kernel` entry points.
4. **Build with the matching DOCA / CUDA toolchain.**
   Confirm the CUDA toolkit version (when GPU-side) is
   compatible with the running DOCA driver — mismatches
   surface in
   [`## debug`](#debug) layers 2 and 6.
5. **Confirm the shared library version stamp.** Run
   `readelf -d` or `objdump -p` on the built library to
   confirm `SONAME` carries the expected `soversion`.
6. **Proceed only to smoke.** The built extension may now be
   exercised through [`## run`](#run) steps 2–4: the no-op
   invocation, its load/call confirmation, and one minimal
   workload invocation. Those steps are the evidence consumed
   by [`## test`](#test); blocking them would make testing
   impossible. Block only [`## run`](#run) step 5, scale-up to
   the full workload, until the no-op and minimal-workload
   checks in `## test` have passed.

When recording the build for downstream consumers, write
down: the DOCA version, the CUDA toolkit version, the
extension's `meson.build` adaptations relative to the
reference, the build command line, the resulting
`SONAME` / `soversion`, and the install path.

## modify

`modify` for the bench-extension framework is substantive
— extensions are by definition source artifacts the
operator authors or adapts.

**Do not modify the shipped reference exemplar
(`/opt/mellanox/doca/tools/bench_extension/doca_bench_cuda/`) in
place.** It is the pristine baseline the agent
cross-references in [`## debug`](#debug) and on every DOCA
upgrade per the
[`CAPABILITIES.md ## Version compatibility`](CAPABILITIES.md#version-compatibility)
`DOCA_EXPERIMENTAL` rebuild rule.

What the agent *does* modify is the operator's COPY of
the reference, per [`## build`](#build) steps 1-3.
Recurring rules for the modification:

- **Keep the API surface shape from the shipped
  reference.** The `DOCA_EXPERIMENTAL` entry-point
  family, per-workload settings struct, accounting
  struct, and lifetime contract (`stop_flag`) are the
  template; the operator adapts the *content* of each,
  not the *shape*.
- **Maintain the `version` / `soversion` stamps.** Per
  [`CAPABILITIES.md ## Version compatibility`](CAPABILITIES.md#version-compatibility):
  every operator-built extension MUST carry the running
  DOCA's `version` and `soversion` so the parent loader
  accepts it.
- **Re-smoke after every modification.** Per
  [`## test`](#test): every change to entry-point
  signature, settings struct, kernel body, or build flags
  re-opens the smoke. The agent must NOT recommend
  scaling up after a modification without re-running the
  smoke.
- **Cross-link to the right neighbouring skill for the
  workload domain.** GPUNetIO workloads route to
  [`doca-gpunetio`](../../libs/doca-gpunetio/SKILL.md);
  Flow workloads route to
  [`doca-flow`](../../libs/doca-flow/SKILL.md); Comch
  workloads route to
  [`doca-comch`](../../libs/doca-comch/SKILL.md). The
  extension's BODY uses the underlying primitive's API;
  this skill does not duplicate that semantics.

Routing for nearby "modify" questions:

- *"Patch `doca-bench` itself to add a new built-in mode."*
  → out of scope; that is contributor work, not
  external-consumer work. Use the extension framework
  exactly because it lets the operator add workloads
  WITHOUT modifying the parent.
- *"Modify the shipped reference in place."* → no. The
  shipped reference is the cross-check baseline.

## run

The start → smoke → measure flow. The full invocation
surface lives in the public DOCA Bench documentation on
`docs.nvidia.com`; this section names the *shape* of the
flow.

1. **Confirm preconditions.** Per
   [`## configure`](#configure) steps 1-7 and
   [`## build`](#build) steps 1-5. The build product
   exists, has the expected `SONAME`, and is on a path
   the parent loader can find.
   Build completion authorizes only steps 2–4 below; it
   does not authorize scale-up.
2. **Run a no-op invocation first.** Per
   [`CAPABILITIES.md ## Safety policy`](CAPABILITIES.md#safety-policy)
   (smoke-before-bulk, twice): point the parent
   `doca-bench` at the extension with the cheapest
   possible workload — the reference exemplar's
   `doca_bench_cuda_start_nop_kernel` is the canonical
   example, and any custom extension SHOULD expose an
   equivalent no-op entry. This proves the parent loaded
   the library, found the registration handshake, and can
   call into it.
3. **Confirm the no-op invocation completed cleanly.**
   Parent logs report the extension loaded; the no-op
   entry returned `DOCA_SUCCESS`; the parent's
   accounting consumed the extension's stats struct.
4. **Run ONE minimal workload invocation.** The smallest
   non-no-op kernel the extension exposes — for the GPU
   reference exemplar, a short-duration eth-recv or
   eth-send kernel with a small `max_rx_pkts` /
   `timeout_ns`. The agent must confirm the `stop_flag`
   discipline before scaling.
5. **Scale up to the operator's full workload.** ONLY
   after `## test` has accepted the evidence from steps
   2–4. This is the only run step blocked by the
   smoke-before-bulk gate. The agent captures the
   full output per [`## test`](#test) (capture step).

When recording the run for downstream consumers, write
down: the DOCA version, the host, the BlueField /
ConnectX generation, the firmware version, the BlueField
mode, the CUDA toolkit version (when applicable), the
parent `doca-bench`'s version, the extension's
`SONAME` / `soversion`, the parent's full invocation
command, and the extension's per-workload settings struct
contents. Partial captures break the downstream debug
ladder.

## test

The bench-extension `## test` is **the canonical
smoke-before-bulk loop for the framework**. *"Test"* in
this skill means *"prove ONE no-op invocation, then ONE
minimal workload invocation, completes end-to-end before
scaling up to the operator's full workload"*, not
*"unit-test the framework"*.

**`## test` is an iterative loop, not a one-shot pass.**
Every mutation — an extension source edit, a build flag
change, a `meson.build` edit, a DOCA upgrade, a CUDA
toolkit upgrade — re-opens BOTH smokes (the no-op smoke
AND the minimal-workload smoke).

The smoke-before-bulk shape:

1. **Run the no-op smoke.** Per [`## run`](#run) step 2.
2. **Confirm the parent loaded the extension and called
   the no-op entry cleanly.** Parent logs show the load
   event; the no-op entry returned `DOCA_SUCCESS`. If
   either fails, walk [`## debug`](#debug) layers 2-4.
3. **Run the minimal-workload smoke.** Per
   [`## run`](#run) step 4.
4. **Confirm the minimal workload completed cleanly and
   the `stop_flag` discipline holds.** The kernel
   terminated when the parent set the stop flag; the
   accounting struct's `jobs_processed` /
   `bytes_processed` are non-zero and consistent.
5. **Scale up to the operator's full workload** per
   [`## run`](#run) step 5.
6. **Repeat the scaled run multiple times to establish
   variance** when the operator is measuring (not just
   validating loading). A single-iteration result is
   not defensible.
7. **Capture the full version stack alongside every
   reported number** per [`## configure`](#configure)
   step 7.

Eval-loop overlay (rows apply to every extension
invocation, not just one):

| Step | Why this is a loop, not a step | Where the substance lives |
| --- | --- | --- |
| 1 → ## debug | The parent cannot find / load the library; walk the load-failed layer, then re-build with corrected `soversion` / install path and re-run step 1 | [`## debug`](#debug) layers 2-3 |
| 2 → ## debug | The parent loaded the library but cannot register the entry points; walk the registration-mismatch layer | [`## debug`](#debug) layer 4 |
| 3 → ## debug | The minimal workload kernel fails or hangs; walk the runtime-call layer and confirm the `stop_flag` discipline | [`## debug`](#debug) layer 5 |
| 5 → step 6 | Per-iteration variance is high on the scaled measurement; the printed average is misleading | [`doca-bench TASKS.md ## debug`](../doca-bench/TASKS.md#debug) (parent's variance handling) |
| any → DOCA upgrade → 1 | Per the `DOCA_EXPERIMENTAL` rebuild rule, every DOCA upgrade re-opens the smoke; the prior smoke is stale | [`## configure`](#configure) step 7 + [`CAPABILITIES.md ## Version compatibility`](CAPABILITIES.md#version-compatibility) |
| any → extension source edit → 1 | After ANY edit, re-run BOTH smokes; the prior smoke is stale | [`## modify`](#modify) |
| any → CUDA toolkit upgrade → 1 | After a CUDA toolkit upgrade, re-run BOTH smokes; the prior smoke is stale | [`## debug`](#debug) layer 6 |

The agent's rule: every state-changing action on the
extension source, the build flags, the DOCA version, or
the toolchain re-opens BOTH smokes. Saving a stale smoke
is exactly the failure mode this loop is here to prevent.

This skill does **not** ship a "test fixture" pass / fail
oracle for a custom extension. The expected output is
workload-specific; pinning one would mislead operators on
a different workload.

## debug

When the user reports a build failure, a load failure, a
registration mismatch, a runtime call failure, or an
unexpected measurement, walk the
[`CAPABILITIES.md ## Error taxonomy`](CAPABILITIES.md#error-taxonomy)
layers in order:

1. **Built-in-mode-would-have-sufficed.** First, confirm
   the operator actually needs an extension. If a built-in
   `doca-bench` mode covers the workload, route back to
   [`doca-bench TASKS.md ## configure`](../doca-bench/TASKS.md#configure)
   and drop the extension entirely.
2. **Build-failed.** Quote the compiler / linker error
   verbatim. Common causes: wrong DOCA headers (route to
   [`doca-version TASKS.md ## debug`](../../doca-version/TASKS.md#debug)),
   wrong CUDA toolkit version (route to the public CUDA
   docs), missing GPU build flags (cross-reference the
   shipped reference's `meson.build`).
3. **Load-failed.** Run `ldd` on the built library, check
   `LD_LIBRARY_PATH`, check `SONAME` matches the running
   DOCA `so_version`. Route the dynamic-linker side to
   [`doca-debug ## debug`](../../doca-debug/SKILL.md).
4. **Registration-mismatch.** Cross-check the
   extension's exported symbols (`nm -D` on the library)
   against the shipped reference's
   `DOCA_EXPERIMENTAL`-marked surface. Confirm against
   the public DOCA Bench documentation on
   `docs.nvidia.com` (via
   [`doca-public-knowledge-map ## DOCA tools`](../../doca-public-knowledge-map/SKILL.md#doca-tools)).
5. **Runtime-call-failed.** Quote the parent's error
   verbatim. Check the per-workload settings struct
   contents; the most common GPU-side failure is the
   `stop_flag` never being set, leading to a hung kernel
   that requires OOB reset.
6. **Version.** Walk
   [`doca-version TASKS.md ## debug`](../../doca-version/TASKS.md#debug)
   end-to-end. Common extension-specific symptoms: the
   extension was built against one DOCA release's headers
   and is loaded by another release's `doca-bench`; the
   CUDA toolkit version doesn't match the driver; the
   firmware doesn't expose what the extension assumes.
7. **Cross-cutting.** Hand off to
   [`doca-debug ## debug`](../../doca-debug/SKILL.md) and
   [`doca-setup ## debug`](../../doca-setup/TASKS.md#debug)
   for env-side layers (driver, firmware, CUDA driver,
   PCIe, dynamic linker).

In every case: **capture the parent's logs, the
extension's logs, `ldd` output, `nm -D` exported symbols,
and the version stack verbatim BEFORE retrying.**
Paraphrasing the error is the canonical lost-fidelity
failure for this skill.

## use

`## use` is the agent-side workflow for *consuming* a
captured `doca-bench-extension` run as evidence.

1. **Read the parent `doca-bench`'s output first.** The
   parent is the canonical sink for the extension's
   measurement; the extension's own logging is a
   debugging aid, not the result.
2. **Read the version stack alongside the result.** A
   result without the version stack (DOCA, firmware,
   CUDA, parent `doca-bench`, extension `SONAME`) is not
   actionable; route to [`## debug`](#debug) layer 6 if
   any leg is missing.
3. **Cross-check `jobs_processed` / `bytes_processed`
   (or the equivalent accounting fields for the custom
   extension).** Non-zero counts are necessary but not
   sufficient; the operator must confirm against the
   workload's domain expectations
   (GPUNetIO / Flow / Comch).
4. **Compare ONLY against extension runs with the
   matching version stack.** Comparing across DOCA
   releases (per the `DOCA_EXPERIMENTAL` rebuild rule)
   is NOT apples-to-apples; the agent says so explicitly.
5. **Route to [`doca-bench TASKS.md ## use`](../doca-bench/TASKS.md)**
   for the parent's standard `## use` workflow; the
   extension result feeds into the parent's reporting
   pipeline, not a separate one.

## Deferred task verbs

The verbs below are not `doca-bench-extension` work and
should be routed out before the agent does any of them
under this skill's name.

- **install DOCA** ⇒
  [`doca-setup ## install`](../../doca-setup/TASKS.md#configure)
  and [`## no-install`](../../doca-setup/TASKS.md#no-install).
- **pick a built-in `doca-bench` workload mode** ⇒
  [`doca-bench`](../doca-bench/SKILL.md). Always the
  first answer to try before any extension work.
- **CUDA toolkit installation** ⇒ NVIDIA's public CUDA
  Toolkit documentation on `docs.nvidia.com`. The
  bench-extension framework does not duplicate it.
- **DOCA GPUNetIO programming semantics** ⇒
  [`doca-gpunetio`](../../libs/doca-gpunetio/SKILL.md).
  The extension's kernel BODY uses GPUNetIO; this skill
  describes the wrapper, not the underlying API.
- **DOCA Flow programming semantics** ⇒
  [`doca-flow`](../../libs/doca-flow/SKILL.md) when the
  extension drives a Flow workload.
- **firmware / driver upgrade** ⇒
  [`doca-version`](../../doca-version/SKILL.md) +
  [`doca-setup`](../../doca-setup/SKILL.md). The
  bench-extension framework has no opinion beyond the
  rebuild-on-DOCA-upgrade rule.
- **contributor work on the in-tree extensions** ⇒ out
  of scope; this skill is for external operators
  authoring custom extensions, not contributors patching
  the shipped ones.

## Command appendix

`doca-bench-extension`-specific invocation classes the
verbs above reach for. Every row is a CLASS — the agent
must not invent build flags, runtime discovery paths, or
extension API entry-point names beyond what the shipped
reference exemplar demonstrates and what the public DOCA
Bench documentation on `docs.nvidia.com` describes.

**Infra-aware preamble (every row below).** Per the
bundle's detect → prefer → fall back → report contract
documented in
[`doca-structured-tools-contract ## The agent behavior contract`](../../doca-structured-tools-contract/SKILL.md#the-agent-behavior-contract),
the agent should:

1. Probe for the matching structured helper FIRST
   (`doca-env --json`; `doca-capability-snapshot`;
   `version-matrix.json`).
2. If the probe succeeds, the structured tool's output is
   the authoritative answer.
3. If the probe fails, fall back to the manual command in
   the row.
4. The schemas the structured tools emit are defined in
   [`doca-structured-tools-contract ## Schemas`](../../doca-structured-tools-contract/SKILL.md#schemas);
   the version-handling semantics are owned by
   [`doca-version`](../../doca-version/SKILL.md).

| Purpose (class) | Invocation (shape) | Owning step | Reads as healthy when … |
| --- | --- | --- | --- |
| Discover the documented bench-extension surface | Inspect the shipped `/opt/mellanox/doca/tools/bench_extension/` source tree on the user's install + the public DOCA Bench documentation on `docs.nvidia.com` (via [`doca-public-knowledge-map ## DOCA tools`](../../doca-public-knowledge-map/SKILL.md#doca-tools)) | [`## configure`](#configure) steps 3-4 + [`## debug`](#debug) layer 4 | The reference exemplar is reachable on the user's install; the public guide documents the runtime discovery mechanism the parent uses. |
| Confirm DOCA `version` / `so_version` for the build | `pkg-config --modversion doca-common` or equivalent DOCA version probe on the build host | [`## configure`](#configure) step 7 + [`## debug`](#debug) layer 6 | The reported version matches the parent `doca-bench`'s linked DOCA version per [`doca-version TASKS.md ## debug`](../../doca-version/TASKS.md#debug) layer 2. |
| Build the extension shared library | A `meson` configure + compile invocation adapted from the shipped `/opt/mellanox/doca/tools/bench_extension/meson.build`, with `version : doca_version` and `soversion : doca_so_version` preserved | [`## build`](#build) steps 2-4 | Exits 0; produces a `.so` with the expected `SONAME`; install path is on the parent loader's search path. |
| Confirm the built library's `SONAME` | `readelf -d <library>.so` or `objdump -p <library>.so` | [`## build`](#build) step 5 + [`## debug`](#debug) layer 3 | `SONAME` carries the matching `so_version`; `ldd` resolves all dependencies. |
| Confirm the extension exports the expected entry points | `nm -D <library>.so` cross-referenced against the `DOCA_EXPERIMENTAL`-marked surface in the shipped `doca_bench_cuda.h` | [`## debug`](#debug) layer 4 | The exported symbols match the surface shape (and the *spelling* matches what the parent expects per the public DOCA Bench documentation). |
| Run the no-op smoke through the parent | `doca-bench` invocation that points at the extension and invokes its no-op kernel (the reference's `doca_bench_cuda_start_nop_kernel` family is the example; the exact CLI surface the parent exposes lives in the parent's documentation) | [`## run`](#run) step 2 + [`## test`](#test) step 1 | Exit 0; parent logs show the extension loaded; the no-op entry returned `DOCA_SUCCESS`. |
| Run the minimal-workload smoke through the parent | `doca-bench` invocation that points at the extension and invokes the smallest non-no-op workload kernel | [`## run`](#run) step 4 + [`## test`](#test) step 3 | Exit 0; parent's accounting consumed the extension's stats struct; `jobs_processed > 0`; `stop_flag` discipline held (kernel terminated when the parent set the flag). |
| Save a session snapshot for debug | Capture (a) the parent's invocation + logs, (b) the extension's logs, (c) `ldd` and `nm -D` output for the library, (d) the version stack (DOCA, firmware, CUDA, parent `doca-bench`, extension `SONAME`) | [`## test`](#test) capture step + [`doca-debug TASKS.md ## debug`](../../doca-debug/TASKS.md#debug) | The saved bundle is consumed by the cross-cutting debug ladder. |

Three cross-cutting rules for this appendix:

- **Never invent a build flag, runtime discovery path, or
  `DOCA_EXPERIMENTAL` entry-point name beyond what the
  shipped reference exemplar demonstrates.** The shipped
  `/opt/mellanox/doca/tools/bench_extension/` source tree on the
  user's install plus the public DOCA Bench documentation
  are the joint contract; prose-derived names are the
  most common hallucination failure for this skill.
- **No-op smoke before any non-no-op invocation.** A
  non-no-op invocation that has not been preceded by a
  clean no-op smoke is not defensible; the agent re-runs
  the smoke per [`## test`](#test) before issuing
  anything else.
- **Cross-link instead of duplicate.** Cross-cutting
  commands (`pkg-config`, `dmesg`, `ldd`, `nm`, `readelf`)
  live in
  [`doca-debug TASKS.md ## Command appendix`](../../doca-debug/TASKS.md);
  parent-tool commands (the `doca-bench` CLI itself) live
  in
  [`doca-bench TASKS.md ## Command appendix`](../doca-bench/TASKS.md);
  GPUNetIO-specific commands live in
  [`doca-gpunetio`](../../libs/doca-gpunetio/SKILL.md);
  this appendix names only bench-extension-specific
  invocations on top.

## Cross-cutting

A few rules that apply across every verb in this file:

- The **public DOCA Bench documentation on
  `docs.nvidia.com`** + the shipped
  `/opt/mellanox/doca/tools/bench_extension/` source tree + the
  parent `doca-bench`'s `--help` are the joint source of
  truth.
- The **shipped reference exemplar is the schema by
  example.** Quote the reference's
  `DOCA_EXPERIMENTAL`-marked surface; do not paraphrase
  entry-point names from prose memory.
- **No-op smoke before bulk.** Per
  [`CAPABILITIES.md ## Safety policy`](CAPABILITIES.md#safety-policy)
  the rule is non-negotiable; loading custom code into a
  benchmark that touches the device requires graduated
  validation.
- **`DOCA_EXPERIMENTAL` rebuild on DOCA upgrade.** Every
  DOCA upgrade re-opens the smoke; a custom extension
  built against one DOCA release is NOT guaranteed to
  load against another.
- **Stop-flag discipline.** Every long-running extension
  kernel respects a parent-set stop signal; the agent
  refuses to recommend a kernel design without bounded
  termination.
- This skill **assumes a healthy DOCA install** (or the
  public NGC DOCA container), a healthy parent
  [`doca-bench`](../doca-bench/SKILL.md), and the
  relevant toolchain for the extension's domain (CUDA
  toolkit when GPU-side). If any of those is in doubt,
  route to the appropriate skill before doing any
  extension work here.