# Concepts — optimizing a system prompt

Depth behind `SKILL.md`'s lever menu: what the prompt actually controls, the
authoring practices and failure modes in full, how to adapt to the runtime reader,
and the pitfalls that look like improvements.

## What the system prompt controls

- **Role & task framing** — who the agent is and what "done" means.
- **Output contract** — the exact shape the downstream consumer or scorer expects.
  The common silent failure is a capable agent that formats its answer wrong and
  scores zero, so diagnose shape before content.
- **Decision rules** — when to call which tool, when to ask versus act, refusal and
  escalation rules. Many agents are scored on adherence to such rules.
- **Reasoning scaffolds & exemplars** — added inline to shape how the model works
  through the task before answering.

## Adapting to the runtime reader

The right prompt edit depends on WHO reads this prompt at runtime (see the
`THE READER` block in your instructions, if present). Most of the advice here — soften
imperatives, explain the reason, keep exemplars minimal — assumes a strong reader.
Flip it for a weaker one:

- **strong reader:** lean, reasoning-first prose; give the reason; keep exemplars
  minimal; soften brittle imperatives, because over-constraining hurts this reader.
- **mid / weak reader:** be explicit. Prefer imperative step-by-step rules; include at
  least one worked exemplar per non-trivial behavior; keep decision chains short; make
  the output contract rigid and literal; and push behavioral rules into tool code
  rather than prose this reader will skip.

When no reader is declared, default to the strong-reader advice — and say so in
`PROCESS.md` so a later run can set the tier deliberately.

## The six authoring practices

1. **Be clear, direct, and specific — write for a capable new hire with no context.**
   If a colleague with minimal context would be confused by the instruction, so will
   the model. Spell out the desired output and the constraints; number steps when the
   order matters.
2. **Give the reason, not just the command.** A rule with its rationale extends to
   cases the author never enumerated. The canonical rewrite: "never use ellipses" →
   "the output is read by a TTS engine that cannot pronounce ellipses."
3. **Say what TO do, not only what NOT to do.** "Compose your reply in flowing prose"
   beats "do not use markdown" — a prohibition fences off one path, a positive
   instruction names the target.
4. **Structure deliberately.** Wrap instructions, context, examples, and input in
   their own sections or tags so the model does not conflate them; put long reference
   data before the instruction that acts on it, and keep the output contract adjacent
   to the point where the model produces the output rather than buried mid-preamble.
5. **Define the output contract explicitly, and use exemplars where prose cannot
   describe the shape.** For structured output, a schema or enum in the tool surface
   constrains it more reliably than prose asking for it.
6. **Keep the prompt lean and self-consistent, and tune trigger strength to the
   reader.** Redundant preambles and conflicting clauses compete for attention; when a
   cluster shows over-eagerness, soften `CRITICAL/MUST/ALWAYS` rather than adding more.

## The five failure modes

1. **Missing or loose output contract** — right content, wrong shape, zero reward.
   The most common silent prompt failure; diagnose shape before content. When the
   scorer reads the final message, the contract must require the agent to state every
   value the scorer checks. *Illustration:* on a customer-service benchmark scored on
   communicated figures (a total, a refund, a saving, a count, a balance), agents
   performed the write correctly and never reported the number — "After computing a
   refund, state the exact figure in your final message (e.g. 'Your refund is
   $42.00')" recovered the class. Read it as one instance of the general shape, not as
   a rule about money.
2. **Conflicting or over-broad instructions** — a later clause contradicts an earlier
   one and the resolution is not predictable. Detectable by reading the artifact
   alone: list the rules governing the same action, check for two different verdicts on
   one input, rewrite toward the stricter.
3. **Redundant preamble** — repeated or stale guidance competes with the rules that
   matter. Length is not safety, and `validate()` reports the counts so growth is
   visible across iterations.
4. **Negative-only phrasing** — "don't do X" with no positive alternative leaves the
   model to guess what Y is.
5. **Stale over-strong language** — anti-laziness `MUST/ALWAYS` phrasing that helped
   an older reader produces over-engineering, excess tool calls, and behaviors
   triggered where they did not apply. The fix for an over-doing cluster is a cut.

## Edit model

Artifact = one or more text files (`prompt.txt`, `policy.md`, `SYSTEM.md`). Edit ops:
`set`, `append`, `ensure_contains`. `validate` requires at least one non-empty prompt
file, reports per-file line/token/constraint-line counts, and — given a `baseline` —
warns when the candidate carries fewer constraint-bearing lines than its parent. That
warning implements `SKILL.md`'s never-drop rule mechanically; it flags a net loss for
a human or the optimizer to justify, and cannot tell a legitimate consolidation from a
lost rule.

## Pitfalls

- **Verbosity creep** — each iteration tends to add. Prune deliberately; the counts
  from `validate()` make the trend visible.
- **Overfitting to the scorer's quirks** rather than the task — watch the val→test
  gap on an accepted candidate.
- **Prompt-injection surface** — a rule that task input can override is not a rule.
  Prefer phrasing that survives adversarial input, and enforce anything security-
  relevant in code.
- **Editing prose where the failure was never a knowledge gap** — the most expensive
  wasted iteration in this capability, because a longer prompt looks like progress.

## Sources

- OpenAI / Anthropic prompt-engineering guides (instruction following, output
  contracts, exemplars) — https://platform.openai.com/docs/guides/prompt-engineering ,
  https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview
- "Large Language Models Are Human-Level Prompt Engineers" (APE), arXiv:2211.01910.
- "Large Language Models as Optimizers" (OPRO), arXiv:2309.03409.
- tau-bench (adherence to decision rules as a scored behavior), arXiv:2406.12045.

"""system-prompt capability — concrete handlers for a prompt/policy artifact.

A "system prompt" capability is one or more text files (default ``prompt.txt``)
that constitute the agent's instructions/policy. These handlers are concrete (a
prompt is just text), so there is nothing to stub — they form a small reusable
library a project adapter's ``apply`` can call.

Edit schema (what an optimizer may emit, mirrored by the mock ops):
    {"file": "prompt.txt", "op": "set"|"append"|"ensure_contains", "text": "..."}

``apply`` and ``validate`` also account for CONSTRAINT-BEARING lines, so the
never-drop-a-needed-rule invariant is reported rather than merely documented: an
optimizer that deletes a rule to make one iteration's metric go up leaves the class
permanently broken, and ``op: "set"`` can do it in a single edit. The accounting is a
crude line count -- it flags a net loss for a human or the optimizer to justify and
cannot tell a legitimate consolidation from a lost rule -- so it is a warning, never a
failure.
"""

from __future__ import annotations

from pathlib import Path

DEFAULT_FILES = ["prompt.txt", "policy.md", "SYSTEM.md"]

# Markdown scaffolding that carries no constraint on its own.
_STRUCTURE_CHARS = set("-=*_ \t")


def rule_lines(text: str) -> int:
    """Count the constraint-bearing lines of a prompt.

    Headings, code fences and horizontal rules structure a prompt without stating a
    rule; every other non-blank line might state one. Deliberately crude: the number
    only has to make a NET LOSS visible, not judge meaning.
    """
    n = 0
    for line in text.splitlines():
        s = line.strip()
        if not s or s.startswith("#") or s.startswith("```") or set(s) <= _STRUCTURE_CHARS:
            continue
        n += 1
    return n


def _stats(parts: dict) -> dict:
    """Per-file size signals so "the preamble is too long" becomes a number."""
    return {
        name: {"lines": len(text.splitlines()),
               "tokens": len(text) // 4,  # ~chars/4, the same estimate skill-package uses
               "rule_lines": rule_lines(text)}
        for name, text in parts.items()
    }


def _rule_loss(before: str, after: str) -> int:
    """How many constraint-bearing lines an edit removed (0 when it added or held)."""
    return max(0, rule_lines(before) - rule_lines(after))


def materialize(capability_dir: Path) -> dict:
    """Read the prompt artifact into a named-component dict (gepa's view)."""
    capability_dir = Path(capability_dir)
    parts = {}
    for name in DEFAULT_FILES:
        f = capability_dir / name
        if f.exists():
            parts[name] = f.read_text(encoding="utf-8")
    if not parts:
        # fall back to any single .txt/.md file present
        for f in sorted(capability_dir.glob("*.txt")) + sorted(capability_dir.glob("*.md")):
            parts[f.name] = f.read_text(encoding="utf-8")
    return parts


def apply(capability_dir: Path, edits: list[dict] | None = None) -> dict:
    """Apply edits to the prompt files. Returns a report of what changed."""
    capability_dir = Path(capability_dir)
    report = {"changed": [], "warnings": []}
    for e in edits or []:
        target = capability_dir / e["file"]
        op = e.get("op", "set")
        text = e.get("text", "")
        cur = target.read_text(encoding="utf-8") if target.exists() else ""
        if op == "set":
            new = text
        elif op == "append":
            new = cur + text
        elif op == "ensure_contains":
            new = cur if text.strip() and text.strip() in cur else cur + text
        else:
            raise ValueError(f"unknown op {op!r}")
        if new != cur:
            lost = _rule_loss(cur, new)
            target.write_text(new, encoding="utf-8")
            report["changed"].append(e["file"])
            if lost:
                report["warnings"].append(
                    f"{e['file']}: op {op!r} removed {lost} constraint-bearing line(s) "
                    f"({rule_lines(cur)} -> {rule_lines(new)}). Change/consolidate/add rather "
                    f"than delete: confirm every dropped constraint survives somewhere "
                    f"(rewritten, merged, or enforced deterministically) and say where in "
                    f"PROCESS.md.")
    return report


def is_empty(capability_dir: Path) -> bool:
    """Return True when the capability directory has no meaningful content yet.

    "Meaningful" is judged after ``strip()`` — the same notion of non-empty that
    ``validate()`` uses — so a missing prompt file and an empty/whitespace-only one
    are both treated as an empty seed (nothing for the optimizer to build on yet)."""
    return not any(v.strip() for v in materialize(Path(capability_dir)).values())


def validate(capability_dir: Path, baseline: Path | dict | None = None) -> dict:
    """A prompt artifact is valid if it has at least one non-empty text file.

    A capability with no non-empty (non-whitespace) prompt content is accepted as a
    valid empty-seed starting state so the optimizer can create the initial content
    from failing trajectories.

    ``baseline`` is the text this candidate was derived from -- a directory (e.g. the
    parent candidate) or an already-materialized ``{file: text}`` dict. When given,
    a file carrying fewer constraint-bearing lines than its baseline is reported in
    ``warnings``. ``warnings`` is always present on both branches so a caller can read
    it without guarding.
    """
    capability_dir = Path(capability_dir)
    if is_empty(capability_dir):
        return {"ok": True, "empty": True, "files": [], "stats": {},
                "problems": [], "warnings": []}
    parts = materialize(capability_dir)
    nonempty = {k: v for k, v in parts.items() if v.strip()}
    warnings = []
    if baseline is not None:
        base = baseline if isinstance(baseline, dict) else materialize(Path(baseline))
        for name, text in sorted(base.items()):
            lost = _rule_loss(text, parts.get(name, ""))
            if lost:
                warnings.append(
                    f"{name}: {lost} constraint-bearing line(s) fewer than the baseline "
                    f"({rule_lines(text)} -> {rule_lines(parts.get(name, ''))}). A needed "
                    f"rule must be changed, consolidated, or relocated -- not dropped; "
                    f"record where each one went.")
    return {"ok": bool(nonempty), "files": list(nonempty), "stats": _stats(nonempty),
            "problems": [] if nonempty else ["no non-empty prompt file found"],
            "warnings": warnings}

"""Round-trip materialize → apply → validate on a temp prompt artifact."""

from __future__ import annotations

import json
import sys
import tempfile
from pathlib import Path

import _bootstrap  # noqa: F401

import abstract


def main() -> int:
    report = {"skill": "system-prompt", "ok": False, "problems": [], "notes": []}
    with tempfile.TemporaryDirectory() as d:
        cap = Path(d)
        (cap / "prompt.txt").write_text("You are helpful.", encoding="utf-8")
        parts = abstract.materialize(cap)
        if "prompt.txt" not in parts:
            report["problems"].append("materialize did not read prompt.txt")
        rep = abstract.apply(cap, [{"file": "prompt.txt", "op": "ensure_contains", "text": " Be concise."}])
        if "prompt.txt" not in rep["changed"]:
            report["problems"].append("apply did not record a change")
        v = abstract.validate(cap)
        if not v["ok"]:
            report["problems"].append(f"validate failed: {v['problems']}")
        if "warnings" not in v:
            report["problems"].append("validate omitted 'warnings' on the non-empty branch")
        report["notes"].append("materialize/apply/validate round-trip ok")

        # The two ops that can destroy content, plus the unknown-op guard.
        abstract.apply(cap, [{"file": "prompt.txt", "op": "append", "text": "\nCite sources.\n"}])
        if "Cite sources." not in (cap / "prompt.txt").read_text(encoding="utf-8"):
            report["problems"].append("apply op=append did not concatenate")
        rep = abstract.apply(cap, [{"file": "prompt.txt", "op": "set", "text": "Be helpful.\n"}])
        if (cap / "prompt.txt").read_text(encoding="utf-8") != "Be helpful.\n":
            report["problems"].append("apply op=set did not replace the file")
        if not rep["warnings"]:
            report["problems"].append("apply op=set dropped rule-bearing lines without warning")
        report["notes"].append("set/append ops behave; a rule-dropping set is flagged")
        try:
            abstract.apply(cap, [{"file": "prompt.txt", "op": "nope", "text": "x"}])
        except ValueError:
            pass
        else:
            report["problems"].append("apply accepted an unknown op")

        # validate against a baseline sees the loss the edit above introduced.
        vb = abstract.validate(cap, baseline={"prompt.txt": "Rule one.\nRule two.\nRule three.\n"})
        if not vb["warnings"]:
            report["problems"].append("validate(baseline=) missed a rule-bearing-line drop")

    # The empty-seed branch is a valid starting state, not a failure.
    with tempfile.TemporaryDirectory() as d:
        ve = abstract.validate(Path(d))
        if not (ve["ok"] and ve.get("empty") and ve["warnings"] == []):
            report["problems"].append(f"validate on an empty dir should be ok/empty: {ve}")
        report["notes"].append("empty seed validates as a valid starting state")
    report["ok"] = not report["problems"]
    print(json.dumps(report, indent=2))
    return 0 if report["ok"] else 1


if __name__ == "__main__":
    sys.exit(main())

"""Expose a system-prompt artifact as a Candidate for the algorithm to optimize."""

from __future__ import annotations

import argparse
import json
import sys
from pathlib import Path

import _bootstrap  # noqa: F401

from cap_evolve import Candidate

import abstract


def main(argv=None) -> int:
    p = argparse.ArgumentParser(prog="system-prompt")
    p.add_argument("--path", required=True, help="capability dir holding the prompt file(s)")
    p.add_argument("--baseline", help="the dir this candidate was derived from (e.g. the parent "
                                     "candidate); warns when a prompt file lost rule-bearing lines")
    args = p.parse_args(argv)
    parts = abstract.materialize(Path(args.path))
    v = abstract.validate(Path(args.path), baseline=Path(args.baseline) if args.baseline else None)
    cand = Candidate(id="seed", component="system-prompt", text_parts=parts, dir=str(args.path))
    print(json.dumps({"candidate": cand.to_dict(), "valid": v}, indent=2))
    return 0 if v["ok"] else 1


if __name__ == "__main__":
    sys.exit(main())

"""Thin shim: locate cap_evolve, then defer to cap_evolve._bootstrap.

Skill scripts ``import _bootstrap`` first. The real path-resolution logic lives
ONCE in ``cap_evolve._bootstrap`` (so it can't drift across skills); this shim
only has to find that package, which means a minimal upward walk for ``core/`` —
the single bit of bootstrapping that genuinely must run before cap_evolve is
importable. Everything else delegates.
"""

from __future__ import annotations

import os
import sys
from pathlib import Path


def _seed_path() -> None:
    """Minimal: put a dir containing the cap_evolve package on sys.path.

    ``CAPEVOLVE_CORE`` is honoured BEFORE any ambient import. An editable install of a
    *different* cap-evolve checkout registers a ``sys.meta_path`` finder, which outranks
    both ``sys.path`` and ``PYTHONPATH`` — so "cap_evolve imports fine" is not evidence that
    it imports the checkout you are standing in. Deferring to the ambient package here made
    an explicit override unreachable, and the symptom was a stale core silently answering
    for this one (``ModuleNotFoundError: cap_evolve.constraints`` from a checkout that
    predates that module). An explicit env var wins.
    """
    env = os.environ.get("CAPEVOLVE_CORE")
    want = Path(env).resolve() if env else None
    if want and (want / "cap_evolve" / "__init__.py").exists():
        loaded = sys.modules.get("cap_evolve")
        already = getattr(loaded, "__file__", None)
        if already and Path(already).resolve().parent.parent == want:
            return                      # right checkout already imported: touch nothing
        p = str(want)
        if p in sys.path:
            sys.path.remove(p)
        sys.path.insert(0, p)
        if loaded is not None:
            # Evicting a module makes a re-import yield a DIFFERENT object, so anything
            # already holding a reference fails an `is` check. Only ever do it when the
            # loaded package really is the wrong checkout — otherwise this "fix" becomes
            # the bug (it broke two identity assertions in core/tests exactly once).
            for name in [m for m in sys.modules
                         if m == "cap_evolve" or m.startswith("cap_evolve.")]:
                sys.modules.pop(name, None)
        for finder in list(sys.meta_path):
            if "cap_evolve" in getattr(finder, "MAPPING", {}):
                sys.meta_path.remove(finder)
        return
    # A checkout's own core outranks an ambient install. Without this, a skill script run
    # from checkout X silently executed against checkout Y's cap_evolve (an editable install
    # registers a sys.meta_path finder, which outranks sys.path), and the only symptom was
    # missing modules — or, worse, a green result measured against the wrong tree.
    here = Path(__file__).resolve()
    own = next((p / "core" for p in here.parents
                if (p / "core" / "cap_evolve" / "__init__.py").exists()), None)
    if own is not None:
        os.environ.setdefault("CAPEVOLVE_CORE", str(own))
        return _seed_path()
    try:
        import cap_evolve  # noqa: F401
        return
    except Exception:
        pass
    cands = []
    for parent in here.parents:
        cands.append(parent / "core")
        cands.append(parent)
    for c in cands:
        if (c / "cap_evolve" / "__init__.py").exists():
            p = str(c)
            if p not in sys.path:
                sys.path.insert(0, p)
            return


_seed_path()
from cap_evolve._bootstrap import ensure_core  # noqa: E402

# Anchor the upward walk at THIS skill script's location (not the core module's).
ensure_core(Path(__file__).resolve())