Claude Skill

bounded-evaluation

Design small evaluation gates for agent outputs, skill changes, prompts, and tool contracts. Use when a change needs rubric checks, pairwise comparison, position-bias mitigation, regression fixtures, or LLM-as-judge discipline without building a heavy eval platform.

LLM Mart · 0 points · 3 views 0 listing impressions 0 install-command copies
Download markoblogo-abvx-agent-skills-skills_bounded-evaluation-37a7146.zip · 2 KB

Install

skills CLI npx skills add https://github.com/markoblogo/abvx-agent-skills/tree/main/skills/bounded-evaluation
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install markoblogo-abvx-agent-skills@llmmart
Git git clone https://github.com/markoblogo/abvx-agent-skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole markoblogo/abvx-agent-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Bounded Evaluation

Use this skill when a change needs a small, explicit evaluation gate.

Use When

  • reviewing skill changes, AGENTS.md rules, prompt playbooks, or SET bundle changes;
  • comparing two agent outputs or proposals;
  • using an LLM judge where bias or weak rubrics could mislead;
  • creating activation tests, regression cases, or ship gates for agent behavior;
  • deciding whether a repeated workflow improved after a bounded edit.

Evaluation Contract

A bounded eval must state:

  • target artifact or behavior;
  • candidate outputs or versions;
  • rubric dimensions and weights, if any;
  • pass/fail threshold;
  • fixtures or scenarios;
  • judge type: deterministic check, human review, LLM judge, or hybrid;
  • known bias risks and mitigation.

Pairwise And Judge Rules

  • Compare A/B outputs against the same rubric.
  • Swap candidate order when using an LLM judge to detect position bias.
  • Require reasons before scores, but treat reasons as evidence to inspect, not truth.
  • Mark the eval invalid when swapped order changes the winner without a defensible reason.
  • Prefer deterministic checks for syntax, schema, file existence, and command success.
  • Keep human approval for high-stakes or ambiguous decisions.

Workflow

  1. Define the smallest behavior being evaluated.
  2. Pick 1-5 representative fixtures or scenarios.
  3. Choose deterministic checks before LLM judging.
  4. If using pairwise judging, run both candidate orders.
  5. Record results, invalidations, and decision.
  6. Feed accepted/rejected outcomes back into the relevant ledger.

Output Shape

Use:

  • eval target;
  • fixtures;
  • rubric;
  • judge type;
  • results;
  • bias checks;
  • decision: accept, reject, revise, or invalid;
  • follow-up artifact or ledger update.

Guardrails

  • Do not publish benchmark claims without committed fixtures, arms, model, host, run count, and captured artifacts.
  • Do not let an LLM judge approve its own rubric changes.
  • Do not use pairwise preference as proof of correctness.
  • Do not hide invalid or inconsistent judge results.

Provenance

Adapted from advanced-evaluation patterns in muratcankoylan/Agent-Skills-for-Context-Engineering, aligned with ABVX SkillOpt validation gates.

Files (abvx-agent-skills)
  • agents
    • openai.yaml 214 B
      Loading…
  • SKILL.md 2.6 KB
    ---
    name: bounded-evaluation
    description: Design small evaluation gates for agent outputs, skill changes, prompts, and tool contracts. Use when a change needs rubric checks, pairwise comparison, position-bias mitigation, regression fixtures, or LLM-as-judge discipline without building a heavy eval platform.
    license: MIT
    metadata:
      abvx_status: experimental
      abvx_origin: adapted
    ---
    
    # Bounded Evaluation
    
    Use this skill when a change needs a small, explicit evaluation gate.
    
    ## Use When
    
    - reviewing skill changes, AGENTS.md rules, prompt playbooks, or SET bundle changes;
    - comparing two agent outputs or proposals;
    - using an LLM judge where bias or weak rubrics could mislead;
    - creating activation tests, regression cases, or ship gates for agent behavior;
    - deciding whether a repeated workflow improved after a bounded edit.
    
    ## Evaluation Contract
    
    A bounded eval must state:
    
    - target artifact or behavior;
    - candidate outputs or versions;
    - rubric dimensions and weights, if any;
    - pass/fail threshold;
    - fixtures or scenarios;
    - judge type: deterministic check, human review, LLM judge, or hybrid;
    - known bias risks and mitigation.
    
    ## Pairwise And Judge Rules
    
    - Compare A/B outputs against the same rubric.
    - Swap candidate order when using an LLM judge to detect position bias.
    - Require reasons before scores, but treat reasons as evidence to inspect, not truth.
    - Mark the eval invalid when swapped order changes the winner without a defensible reason.
    - Prefer deterministic checks for syntax, schema, file existence, and command success.
    - Keep human approval for high-stakes or ambiguous decisions.
    
    ## Workflow
    
    1. Define the smallest behavior being evaluated.
    2. Pick 1-5 representative fixtures or scenarios.
    3. Choose deterministic checks before LLM judging.
    4. If using pairwise judging, run both candidate orders.
    5. Record results, invalidations, and decision.
    6. Feed accepted/rejected outcomes back into the relevant ledger.
    
    ## Output Shape
    
    Use:
    
    - eval target;
    - fixtures;
    - rubric;
    - judge type;
    - results;
    - bias checks;
    - decision: `accept`, `reject`, `revise`, or `invalid`;
    - follow-up artifact or ledger update.
    
    ## Guardrails
    
    - Do not publish benchmark claims without committed fixtures, arms, model, host, run count, and captured artifacts.
    - Do not let an LLM judge approve its own rubric changes.
    - Do not use pairwise preference as proof of correctness.
    - Do not hide invalid or inconsistent judge results.
    
    ## Provenance
    
    Adapted from advanced-evaluation patterns in `muratcankoylan/Agent-Skills-for-Context-Engineering`, aligned with ABVX SkillOpt validation gates.
    
  • SKILL_CARD.md 1.3 KB
    Loading…

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.