agent-loop-testing
Use this skill when you need evidence-bounded loop state, plan/action/observation cycles, stop conditions, budgets, repetition, and trace evidence; triggers include Agent 循环 and Agent loop.
Install
npx skills add https://github.com/naodeng/awesome-qa-skills/tree/main/skills/en/testing-types/agent-loop-testing
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install naodeng-awesome-qa-skills@llmmart
git clone https://github.com/naodeng/awesome-qa-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole naodeng/awesome-qa-skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Agent Loop Testing
When to Use
- Use this skill when you need evidence-bounded analysis, design, or validation preparation for loop state, plan/action/observation cycles, stop conditions, budgets, repetition, and trace evidence.
- Use it to review an Agent, RAG, or LLM plan, result, or evidence set and produce actionable improvements.
- Use it when context is incomplete but a bounded first pass with assumptions, gaps, and human-decision boundaries is still useful.
Output Format Options
- Default to Markdown organized by domain risk, evidence state, priority, and boundary.
- When the user requests tables, CSV, JSON, or ticket fields, preserve the same finding fields, evidence, and decision boundaries.
- Before machine consumption, confirm the schema, enums, required fields, and evidence sources.
How to Use
- Read and follow
prompts/agent-loop-testing.md, including its input audit, domain coverage, and output order. - Extract scope, environment, version, time window, constraints, success criteria, and available evidence, with attention to loop state, plan/action/observation, stop condition, budget, repetition.
- Separate confirmed facts, evidence-backed inferences, candidate recommendations, and Human decisions before ranking by risk and evidence strength.
- Turn high-risk items into preconditions, steps, expected behavior or decision criteria, required evidence, and a validation method.
- When input is incomplete, deliver a bounded first pass, state unsupported conclusions, and never present static material as execution evidence.
Reference Files
When calibrating domain outcomes, valid controls, or evidence gaps, read references/domain-decision-example.md for sanitized input, per-item oracles, and reproducible calculations.
Always read
prompts/agent-loop-testing.md; it is the complete execution specification for this skill.For evaluation, read
evals/eval.yamland the matching cases underevals/cases/.Load
references/,examples/,scripts/, oroutput-formats.mdonly when those directories exist and the task needs them.
Domain Execution Essentials
- Record task/run/step IDs, plan, tool and normalized arguments, observation identity/digest, business state delta, and cumulative time/cost; redact sensitive values while preserving deduplication evidence.
- Progress requires new goal-relevant evidence or state advancement. Repeated action/observation with no advancement counts as a no-progress cycle; successful calls or repeated prose do not establish progress.
- Apply supplied no-progress limits, retry policy, and time/cost budgets. Safety/cancellation stops immediately; any exhausted hard budget precedes more planning. Ask for missing limits rather than invent numbers.
- Repetition takes the project-authorized change-strategy, stop, or escalation branch. Report trigger step, termination class, remaining work, and absence of extra effects.
Core Constraints
- Keep the analysis focused on loop state, plan/action/observation cycles, stop conditions, budgets, repetition, and trace evidence; do not replace business owners or Human risk acceptance, exception approval, or safety decisions.
- Never invent system behavior, fields, model outputs, data, thresholds, root causes, execution records, or pass claims.
- Static design, plans, file presence, or a dry run retain their evidence state and cannot become proof of real execution.
- When evidence is insufficient, use pending confirmation, blocked, unassessed, or NOT_SCORED and give the smallest validation method.
- For user data, production, or safety work, use least privilege, masked data, mocks, dry runs, or isolation.
Delivery Checklist
- Covered loop state, plan/action/observation, stop condition, budget, repetition, with source, evidence state, and validation method for each.
- Separated facts, inferences, candidate recommendations, gaps, and Human decisions.
- Gave high-risk items P0/P1/P2/P3 or an equivalent priority, owner role, and close condition.
- Did not turn plans, static checks, or dry runs into test execution, all-passed, or safety-approved claims.
- Stated residual risk, stop/escalation conditions, and next actions.
Common Pitfalls
- Listing checks without triggers, expected concerns, owner roles, close conditions, and evidence.
- Treating adjacent tests or model tools as a complete Agent loop judgment.
- Using unexplained numbers for false precision or writing correlation as causation.
- Refusing incomplete input, or pretending that incomplete evidence is conclusive.
Best Practices
- Start with paths most likely to cause user harm, business loss, or decision blockage.
- Use the smallest verifiable experiment to reduce uncertainty and record conditions, versions, sources, and evidence.
- Make the Skill independently installable, executable, and reviewable by another engineer.
Files (awesome-qa-skills)
-
agents
-
openai.yaml 290 B
Loading…
-
-
evals
-
cases
-
basic-success.yaml 1.4 KB
Loading… -
domain-method-regression.yaml 1.4 KB
Loading… -
edge-incomplete-input.yaml 917 B
Loading… -
edge-scope-boundary.yaml 953 B
Loading…
-
-
fixtures
-
check-domain-decisions.sh 95 B
Loading… -
check_domain_decisions.py 3.7 KB
Loading… -
domain-oracle.json 129 B
Loading…
-
-
eval.yaml 489 B
Loading… -
local-rules.json 137 B
Loading… -
trigger-prompts.csv 582 B · in bundle
-
-
prompts
-
agent-loop-testing.md 5.6 KB
Loading…
-
-
references
-
domain-decision-example.md 1.6 KB
Loading…
-
-
SKILL.md 5.4 KB
--- name: agent-loop-testing description: Use this skill when you need evidence-bounded loop state, plan/action/observation cycles, stop conditions, budgets, repetition, and trace evidence; triggers include Agent 循环 and Agent loop. license: PolyForm-Noncommercial-1.0.0 metadata: version: "1.0.0" compatibility: "" author: "Awesome QA Skills contributors" language: "en" category: "testing-types" source: "https://github.com/naodeng/awesome-qa-skills" license-url: "https://polyformproject.org/licenses/noncommercial/1.0.0" --- # Agent Loop Testing ## When to Use - Use this skill when you need evidence-bounded analysis, design, or validation preparation for loop state, plan/action/observation cycles, stop conditions, budgets, repetition, and trace evidence. - Use it to review an Agent, RAG, or LLM plan, result, or evidence set and produce actionable improvements. - Use it when context is incomplete but a bounded first pass with assumptions, gaps, and human-decision boundaries is still useful. ## Output Format Options - Default to Markdown organized by domain risk, evidence state, priority, and boundary. - When the user requests tables, CSV, JSON, or ticket fields, preserve the same finding fields, evidence, and decision boundaries. - Before machine consumption, confirm the schema, enums, required fields, and evidence sources. ## How to Use 1. Read and follow `prompts/agent-loop-testing.md`, including its input audit, domain coverage, and output order. 2. Extract scope, environment, version, time window, constraints, success criteria, and available evidence, with attention to loop state, plan/action/observation, stop condition, budget, repetition. 3. Separate confirmed facts, evidence-backed inferences, candidate recommendations, and Human decisions before ranking by risk and evidence strength. 4. Turn high-risk items into preconditions, steps, expected behavior or decision criteria, required evidence, and a validation method. 5. When input is incomplete, deliver a bounded first pass, state unsupported conclusions, and never present static material as execution evidence. ## Reference Files - When calibrating domain outcomes, valid controls, or evidence gaps, read [references/domain-decision-example.md](references/domain-decision-example.md) for sanitized input, per-item oracles, and reproducible calculations. - Always read `prompts/agent-loop-testing.md`; it is the complete execution specification for this skill. - For evaluation, read `evals/eval.yaml` and the matching cases under `evals/cases/`. - Load `references/`, `examples/`, `scripts/`, or `output-formats.md` only when those directories exist and the task needs them. ## Domain Execution Essentials 1. Record task/run/step IDs, plan, tool and normalized arguments, observation identity/digest, business state delta, and cumulative time/cost; redact sensitive values while preserving deduplication evidence. 2. Progress requires new goal-relevant evidence or state advancement. Repeated action/observation with no advancement counts as a no-progress cycle; successful calls or repeated prose do not establish progress. 3. Apply supplied no-progress limits, retry policy, and time/cost budgets. Safety/cancellation stops immediately; any exhausted hard budget precedes more planning. Ask for missing limits rather than invent numbers. 4. Repetition takes the project-authorized change-strategy, stop, or escalation branch. Report trigger step, termination class, remaining work, and absence of extra effects. ## Core Constraints - Keep the analysis focused on loop state, plan/action/observation cycles, stop conditions, budgets, repetition, and trace evidence; do not replace business owners or Human risk acceptance, exception approval, or safety decisions. - Never invent system behavior, fields, model outputs, data, thresholds, root causes, execution records, or pass claims. - Static design, plans, file presence, or a dry run retain their evidence state and cannot become proof of real execution. - When evidence is insufficient, use pending confirmation, blocked, unassessed, or NOT_SCORED and give the smallest validation method. - For user data, production, or safety work, use least privilege, masked data, mocks, dry runs, or isolation. ## Delivery Checklist - [ ] Covered loop state, plan/action/observation, stop condition, budget, repetition, with source, evidence state, and validation method for each. - [ ] Separated facts, inferences, candidate recommendations, gaps, and Human decisions. - [ ] Gave high-risk items P0/P1/P2/P3 or an equivalent priority, owner role, and close condition. - [ ] Did not turn plans, static checks, or dry runs into test execution, all-passed, or safety-approved claims. - [ ] Stated residual risk, stop/escalation conditions, and next actions. ## Common Pitfalls - Listing checks without triggers, expected concerns, owner roles, close conditions, and evidence. - Treating adjacent tests or model tools as a complete Agent loop judgment. - Using unexplained numbers for false precision or writing correlation as causation. - Refusing incomplete input, or pretending that incomplete evidence is conclusive. ## Best Practices - Start with paths most likely to cause user harm, business loss, or decision blockage. - Use the smallest verifiable experiment to reduce uncertainty and record conditions, versions, sources, and evidence. - Make the Skill independently installable, executable, and reviewable by another engineer.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.