Claude Skill

using-cap-evolve

Front door for cap-evolve: routes an optimization request to the right pipeline phase. Use when someone wants an agent, skill, system prompt, tool surface, or MCP toolset to score higher on an eval, benchmark, or task suite — "optimize my skill", "raise the pass rate on these tas

LLM Mart · 0 points · 13 views 0 listing impressions 0 install-command copies
Virus-scanned Reviewed automatically before listing.

Full trust report

Download skillberry-ai-cap-evolve-skills_orchestrate_using-cap-evolve-49fcedb.zip · 7 KB
Part of skillberry-ai/cap-evolve — 22 skills

Install

skills CLI npx skills add https://github.com/skillberry-ai/cap-evolve/tree/main/skills/orchestrate/using-cap-evolve
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install skillberry-ai-cap-evolve@llmmart
Git git clone https://github.com/skillberry-ai/cap-evolve.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole skillberry-ai/cap-evolve collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

using-cap-evolve — the router

The front door: it works out where the user is and hands off, running no phase and editing nothing. Boundary: this router picks the door, orchestrate drives the run.

Routing decision

Run from the user's project dir; S is the absolute path of the directory you loaded this SKILL.md from — the one location always known here (no env var is set for a plugin install):

S=<this skill's own directory>; python "$S/scripts/run.py" --base .capevolve

Follow next; pass reason on to the user. Two things the JSON cannot say for itself:

  • On a fresh request go through intake, and if an input it needs is missing, ask the user for it rather than inventing one (intake owns that rule).
  • An existing run is never restarted from zero: interrupted → cap-evolve run --resume; sealed and the user wants another attempt → cap-evolve run --reuse-baseline <run dir>.

Three ways to run — orchestrate has the detail

  1. Phase chain — /cap-evolve:<phase> turn by turn, so each step is inspected.
  2. Deterministic — cap-evolve run --spec .capevolve/project/capevolve.yaml sequences the check gate → baseline → algorithm → finalize → report. It presumes intake already happened; it does not run intake.
  3. Agent handoff — with orchestration_mode: agent, cap-evolve run stops after baseline and hands the loop back to you; no sealed-test number until you finalize.

No plugin, or a non-Claude host: follow RUN.md step by step. Same engine, same rules.

Files (cap-evolve)
  • scripts
    • abstract.py 231 B
      """using-cap-evolve is a pure router; it declares no adapter abstract methods.
      
      It only inspects on-disk project state and recommends the next command.
      check.py verifies it can resolve that state and that run.py exposes main()."""
      
    • check.py 4.5 KB
      """Behavioral check for using-cap-evolve.
      
      Asserts the router actually routes, and that the three misroutes issue #339
      reproduced stay fixed:
      
        * fresh (empty base) -> `fresh` / intake; a scaffolded base is not `fresh`.
        * a run dir with no `state.json` is NOT reported as `scaffolded` (it would send
          the agent to implement an adapter that already exists).
        * a newer live run is never masked by an older sealed one (which would claim a
          sealed headline number while an optimization is mid-flight).
        * `running` routes to `--resume` and `finalized` to `--reuse-baseline` — report
          continues nothing.
      """
      
      from __future__ import annotations
      
      import json
      import sys
      import tempfile
      from pathlib import Path
      
      import _bootstrap  # noqa: F401
      
      
      def main() -> int:
          rep = {"ok": False, "problems": [], "notes": []}
          try:
              import run
              if not hasattr(run, "main") or not hasattr(run, "resolve_state"):
                  rep["problems"].append("run.py missing main()/resolve_state()")
              else:
                  with tempfile.TemporaryDirectory() as td:
                      base = Path(td) / ".capevolve"
                      base.mkdir(parents=True)
                      fresh = run.resolve_state(base)
                      if fresh.get("state") != "fresh":
                          rep["problems"].append(f"empty base should be 'fresh', got {fresh.get('state')!r}")
                      if "intake" not in (fresh.get("next") or ""):
                          rep["problems"].append("fresh state must route to intake")
      
                      # scaffolded: project dir + capevolve.yaml, no run, no adapter
                      proj = base / "project"
                      proj.mkdir(parents=True)
                      (proj / "capevolve.yaml").write_text("capability_path: seed\n", encoding="utf-8")
                      scaffolded = run.resolve_state(base)
                      if scaffolded.get("state") == "fresh":
                          rep["problems"].append("base with capevolve.yaml should not be 'fresh'")
                      rep["notes"].append(f"fresh -> {fresh.get('next')}; "
                                          f"scaffolded -> {scaffolded.get('next')}")
      
                      # case A: a run dir mid-create (no state.json) must not read as scaffolded.
                      (base / "run_20260101_000000").mkdir()
                      started = run.resolve_state(base)
                      if started.get("state") == "scaffolded":
                          rep["problems"].append("a run dir with no state.json must not resolve to "
                                                 "'scaffolded' (sends the agent to re-implement)")
                      if "--resume" not in (started.get("next") or ""):
                          rep["problems"].append("a half-created run must route to `run --resume`")
      
                      # case C: newer live run must not be masked by an older sealed one.
                      sealed = base / "run_20260101_000000"
                      (sealed / "state.json").write_text("{}", encoding="utf-8")
                      (sealed / "splits.json").write_text('{"test_used": true}', encoding="utf-8")
                      fin = run.resolve_state(base)
                      if fin.get("state") != "finalized":
                          rep["problems"].append(f"sealed run should be 'finalized', got {fin.get('state')!r}")
                      if "--reuse-baseline" not in (fin.get("next") or ""):
                          rep["problems"].append("'finalized' must offer a new run with --reuse-baseline, "
                                                 "not just a report")
      
                      live = base / "run_20260202_000000"
                      live.mkdir()
                      (live / "state.json").write_text("{}", encoding="utf-8")
                      (live / "splits.json").write_text('{"test_used": false}', encoding="utf-8")
                      mid = run.resolve_state(base)
                      if mid.get("state") != "running":
                          rep["problems"].append(f"a newer unsealed run must win over an older sealed "
                                                 f"one; got {mid.get('state')!r}")
                      if "--resume" not in (mid.get("next") or ""):
                          rep["problems"].append("'running' must route to `cap-evolve run --resume`")
                      rep["notes"].append(f"run_started -> {started.get('next')}; "
                                          f"finalized -> {fin.get('state')}; running -> {mid.get('next')}")
          except Exception as e:  # noqa: BLE001
              rep["problems"].append(f"import/exec failed: {e}")
          rep["ok"] = not rep["problems"]
          print(json.dumps(rep, indent=2))
          return 0 if rep["ok"] else 1
      
      
      if __name__ == "__main__":
          sys.exit(main())
      
    • run.py 6.7 KB
      """using-cap-evolve — resolve where the user is and recommend the next command.
      
      This router does NOT optimize. It inspects the on-disk state of a cap-evolve
      project and prints a routing decision the agent (or the orchestrator) acts on:
      
          {state, next, sequence, reason, intent}
      
      States (a simple, deterministic state machine over the project dir):
        fresh       — no .capevolve/project/, or no capevolve.yaml
                                                    -> next: /cap-evolve:intake
        scaffolded  — capevolve.yaml exists, check not yet green
                                                    -> next: /cap-evolve:implement-and-check
        ready       — `cap-evolve check` is green, no run yet
                                                    -> next: baseline / `cap-evolve run`
        run_started — a run_* dir with no state.json (a torn/partial create — see
                      RunDir.create's exist_ok contract in core/cap_evolve/rundir.py)
                                                    -> next: `cap-evolve run --resume --run-ts`
        running     — a run_* dir, test split not used
                                                    -> next: `cap-evolve run --resume`
        finalized   — splits.json test_used         -> next: a NEW run with --reuse-baseline
                                                       (report only inspects the sealed one)
      
      Runs are globbed unconditionally. Requiring state.json would hide a run that is
      mid-create behind an older sealed one and report it as `finalized` while an
      optimization is in flight — the worst answer this router can give.
      
      The check is best-effort: if core isn't importable we fall back to "is there a
      capevolve.yaml" rather than failing. Pure stdlib + cap_evolve (optional).
      """
      
      from __future__ import annotations
      
      import argparse
      import json
      import sys
      from pathlib import Path
      
      import _bootstrap  # noqa: F401
      
      
      PHASE_CHAIN = ["intake", "implement-and-check", "baseline",
                     "<algorithm>", "finalize", "report"]
      
      
      def _latest_run(base: Path) -> Path | None:
          runs = sorted((r for r in base.glob("run_*") if r.is_dir()), key=lambda r: r.name)
          return runs[-1] if runs else None
      
      
      def _check_green(project: Path) -> bool | None:
          """True/False if we can run the real check, None if core unavailable."""
          try:
              from cap_evolve.check import run_check
          except Exception:
              return None
          try:
              return run_check(project).ok
          except Exception:
              return False
      
      
      def _run_state(run: Path) -> dict:
          """Classify the newest run dir. Never claims a seal it cannot read."""
          ts = run.name[len("run_"):]
          sp = run / "splits.json"
          if not (run / "state.json").exists() and not sp.exists():
              return {"state": "run_started", "next": f"cap-evolve run --resume --run-ts {ts}",
                      "run": str(run),
                      "reason": "a run dir exists but has no state.json — a torn/partial create. "
                                "Resume it by ts (plain --resume skips a run with no state.json); "
                                "the adapter is already implemented, do not re-run intake."}
          if sp.exists():
              try:
                  spd = json.loads(sp.read_text(encoding="utf-8"))
              except Exception as e:  # noqa: BLE001
                  return {"state": "running", "next": f"cap-evolve run --resume --run-ts {ts}",
                          "run": str(run),
                          "reason": f"splits.json is unreadable ({e}) — the split is an "
                                    "honesty-critical artifact, so the seal cannot be confirmed "
                                    "either way. Inspect it before spending more budget."}
              if spd.get("test_used"):
                  return {"state": "finalized",
                          "next": ("cap-evolve run --spec .capevolve/project/capevolve.yaml "
                                   f"--reuse-baseline {run}"),
                          "run": str(run),
                          "reason": "test is sealed/used — this run's headline number is recorded "
                                    "and cannot be re-scored. For another attempt start a new run "
                                    "reusing this baseline; `/cap-evolve:report` only inspects it."}
          return {"state": "running", "next": f"cap-evolve run --resume --run-ts {ts}",
                  "run": str(run),
                  "reason": "a run is in progress and the test split is still sealed — resume it "
                            "at iteration N+1; `/cap-evolve:report` shows status without continuing."}
      
      
      def resolve_state(base: Path) -> dict:
          base = Path(base)
          project = base / "project"
          yaml = project / "capevolve.yaml"
      
          if not project.is_dir():
              return {"state": "fresh", "next": "/cap-evolve:intake",
                      "reason": "no .capevolve/project/ — start with intake (Phase 1)."}
      
          run = _latest_run(base)
          if run is not None:
              return _run_state(run)
      
          if not yaml.exists():
              return {"state": "fresh", "next": "/cap-evolve:intake",
                      "reason": "project dir exists but no capevolve.yaml — run intake. Warn the "
                                "user first if the dir is non-empty: intake scaffolds into it."}
      
          green = _check_green(project)
          if green is True:
              return {"state": "ready", "next": "/cap-evolve:baseline",
                      "reason": "cap-evolve check is green — baseline then the algorithm, "
                                "or `cap-evolve run --spec` for the automatic path."}
          if green is False:
              return {"state": "scaffolded", "next": "/cap-evolve:implement-and-check",
                      "reason": "capevolve.yaml present but `cap-evolve check` is not green — "
                                "implement the adapter and pass the hard gate first."}
          # green is None: core not importable here — recommend the gate step conservatively.
          return {"state": "scaffolded", "next": "/cap-evolve:implement-and-check",
                  "reason": "capevolve.yaml present; could not run check here — verify the "
                            "hard gate via implement-and-check before baseline."}
      
      
      def main(argv=None) -> int:
          p = argparse.ArgumentParser(prog="using-cap-evolve")
          p.add_argument("intent", nargs="*", help="free-text 'what to optimize' (echoed back)")
          p.add_argument("--base", default=".capevolve")
          args = p.parse_args(argv)
      
          decision = resolve_state(Path(args.base))
          decision["sequence"] = PHASE_CHAIN
          decision["intent"] = " ".join(args.intent) or None
          decision["run_modes"] = {
              "standalone": "drive /cap-evolve:<phase> turn by turn",
              "automatic": "cap-evolve run --spec .capevolve/project/capevolve.yaml",
              "agent_handoff": "orchestration_mode: agent — run stops after baseline, you drive",
              "host_agnostic": "follow RUN.md (no plugin / non-Claude host)",
          }
          print(json.dumps(decision, indent=2))
          return 0
      
      
      if __name__ == "__main__":
          sys.exit(main())
      
    • _bootstrap.py 3.6 KB
      """Thin shim: locate cap_evolve, then defer to cap_evolve._bootstrap.
      
      Skill scripts ``import _bootstrap`` first. The real path-resolution logic lives
      ONCE in ``cap_evolve._bootstrap`` (so it can't drift across skills); this shim
      only has to find that package, which means a minimal upward walk for ``core/`` —
      the single bit of bootstrapping that genuinely must run before cap_evolve is
      importable. Everything else delegates.
      """
      
      from __future__ import annotations
      
      import os
      import sys
      from pathlib import Path
      
      
      def _seed_path() -> None:
          """Minimal: put a dir containing the cap_evolve package on sys.path.
      
          ``CAPEVOLVE_CORE`` is honoured BEFORE any ambient import. An editable install of a
          *different* cap-evolve checkout registers a ``sys.meta_path`` finder, which outranks
          both ``sys.path`` and ``PYTHONPATH`` — so "cap_evolve imports fine" is not evidence that
          it imports the checkout you are standing in. Deferring to the ambient package here made
          an explicit override unreachable, and the symptom was a stale core silently answering
          for this one (``ModuleNotFoundError: cap_evolve.constraints`` from a checkout that
          predates that module). An explicit env var wins.
          """
          env = os.environ.get("CAPEVOLVE_CORE")
          want = Path(env).resolve() if env else None
          if want and (want / "cap_evolve" / "__init__.py").exists():
              loaded = sys.modules.get("cap_evolve")
              already = getattr(loaded, "__file__", None)
              if already and Path(already).resolve().parent.parent == want:
                  return                      # right checkout already imported: touch nothing
              p = str(want)
              if p in sys.path:
                  sys.path.remove(p)
              sys.path.insert(0, p)
              if loaded is not None:
                  # Evicting a module makes a re-import yield a DIFFERENT object, so anything
                  # already holding a reference fails an `is` check. Only ever do it when the
                  # loaded package really is the wrong checkout — otherwise this "fix" becomes
                  # the bug (it broke two identity assertions in core/tests exactly once).
                  for name in [m for m in sys.modules
                               if m == "cap_evolve" or m.startswith("cap_evolve.")]:
                      sys.modules.pop(name, None)
              for finder in list(sys.meta_path):
                  if "cap_evolve" in getattr(finder, "MAPPING", {}):
                      sys.meta_path.remove(finder)
              return
          # A checkout's own core outranks an ambient install. Without this, a skill script run
          # from checkout X silently executed against checkout Y's cap_evolve (an editable install
          # registers a sys.meta_path finder, which outranks sys.path), and the only symptom was
          # missing modules — or, worse, a green result measured against the wrong tree.
          here = Path(__file__).resolve()
          own = next((p / "core" for p in here.parents
                      if (p / "core" / "cap_evolve" / "__init__.py").exists()), None)
          if own is not None:
              os.environ.setdefault("CAPEVOLVE_CORE", str(own))
              return _seed_path()
          try:
              import cap_evolve  # noqa: F401
              return
          except Exception:
              pass
          cands = []
          for parent in here.parents:
              cands.append(parent / "core")
              cands.append(parent)
          for c in cands:
              if (c / "cap_evolve" / "__init__.py").exists():
                  p = str(c)
                  if p not in sys.path:
                      sys.path.insert(0, p)
                  return
      
      
      _seed_path()
      from cap_evolve._bootstrap import ensure_core  # noqa: E402
      
      # Anchor the upward walk at THIS skill script's location (not the core module's).
      ensure_core(Path(__file__).resolve())
      
  • meta.yaml 456 B
    component: orchestrate
    name: using-cap-evolve
    summary: Session-start router — turns an "optimize this" request into the cap-evolve pipeline; resolves the on-disk state and names the next command (intake, the check gate, a resumed run, or a new run reusing a sealed baseline).
    entry: scripts/run.py
    abstract: scripts/abstract.py
    check: scripts/check.py
    needs: []
    provides: []
    compatible_with:
      capabilities: ["*"]
      optimizers: ["*"]
      algorithms: ["*"]
    
  • SKILL.md 2.5 KB
    ---
    name: using-cap-evolve
    description: 'Front door for cap-evolve: routes an optimization request to the right pipeline phase. Use when someone wants an agent, skill, system prompt, tool surface, or MCP toolset to score higher on an eval, benchmark, or task suite — "optimize my skill", "raise the pass rate on these tasks", "my agent keeps failing these cases", "get this prompt''s accuracy up on my evals" — even when they never say "optimize", and whenever a .capevolve/ project or an unfinished run is in the tree. Routes to intake, the check gate, a resumed run, or the report; optimizes nothing itself. Not for making code or a query faster, and not for rewording one prompt with no eval to score it against. When the user names a phase or algorithm outright (baseline, gate, hill-climb, gepa), use that skill directly.'
    component: orchestrate
    argument-hint: "[what to optimize] [--base .capevolve]"
    allowed-tools: Read, Bash
    provides: []
    needs: []
    sources: [evo, superpowers]
    ---
    
    # using-cap-evolve — the router
    
    The front door: it works out *where the user is* and hands off, running no phase and
    editing nothing. Boundary: this router picks the door, `orchestrate` drives the run.
    
    ## Routing decision
    Run from the user's project dir; `S` is the absolute path of the directory you loaded this
    SKILL.md from — the one location always known here (no env var is set for a plugin install):
    ```bash
    S=<this skill's own directory>; python "$S/scripts/run.py" --base .capevolve
    ```
    Follow `next`; pass `reason` on to the user. Two things the JSON cannot say for itself:
    - On a fresh request go through `intake`, and if an input it needs is missing, ask the
      user for it rather than inventing one (`intake` owns that rule).
    - An existing run is never restarted from zero: interrupted → `cap-evolve run --resume`;
      sealed and the user wants another attempt → `cap-evolve run --reuse-baseline <run dir>`.
    
    ## Three ways to run — `orchestrate` has the detail
    1. **Phase chain** — `/cap-evolve:<phase>` turn by turn, so each step is inspected.
    2. **Deterministic** — `cap-evolve run --spec .capevolve/project/capevolve.yaml`
       sequences the check gate → baseline → algorithm → finalize → report. It presumes
       intake already happened; it does not run intake.
    3. **Agent handoff** — with `orchestration_mode: agent`, `cap-evolve run` stops after
       baseline and hands the loop back to you; no sealed-test number until you finalize.
    
    No plugin, or a non-Claude host: follow `RUN.md` step by step. Same engine, same rules.
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related