Claude Skill

agent-safety

Use when bounding an LLM agent that already runs — scoping its task domain, gating tools to least privilege, defending against prompt injection in untrusted web/email/RAG text, requiring human approval on irreversible actions, capping runtime and cost, or triaging what it already

LLM Mart · 0 points · 1 views 0 listing impressions 0 install-command copies
Download ericrisco-rsc-harness-skills_agent-safety-28a51a9.zip · 8 KB

Install

skills CLI npx skills add https://github.com/ericrisco/rsc-harness/tree/main/skills/agent-safety
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install ericrisco-rsc-harness@llmmart
Git git clone https://github.com/ericrisco/rsc-harness.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole ericrisco/rsc-harness collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Agent safety

You are the security review for an agent's agency, not for its code. The loop works, tools are wired, memory persists — your job is to make that autonomy bounded. If you want to review ordinary endpoints, auth, or secrets handling, that is ../security-audit/SKILL.md — this skill is the Agentic Top 10, the risks that exist only because a model has tools and autonomy. If the loop or tools do not exist yet, that is ../building-agents/SKILL.md. You arrive after both.

references/threat-model.md carries the OWASP Agentic Top 10 2026 risks mapped to the controls below, the pre-ship guardrail checklist, and the incident-triage flow for "the agent did X" — open it when you are reviewing before ship or reconstructing an incident.

The ownership split

Agent security splits into four layers — Model · Harness · Tools · Environment. The model provider owns only the Model layer (alignment, refusals). Everything else is yours: the Harness (loop, memory, context assembly), the Tools (what the agent can do), and the Environment (creds, network, blast radius). Do not outsource a layer you own to "the model is aligned."

Three excesses cause almost every agentic incident. Cut all three:

  • Excessive functionality — tools the task never needs.
  • Excessive permissions — broader scopes/creds than the tool needs.
  • Excessive autonomy — acting without checking back when it should.

The operating principle is least agency: autonomy is earned per task, not defaulted.

Scope limits

  • Declare the allowed task domain as a hard boundary in the system prompt. Why: an undeclared scope is an infinite scope; "you are a refund assistant; you do not touch payroll" is a constraint a reviewer can check.
  • Deny by default — the agent starts with zero tools. Each tool earns its place by a task justification. Why: an opt-out tool list grows; an opt-in list stays minimal.
  • Segregate the instruction channel from the data channel. System/developer prompt = trusted instructions. Everything the agent reads at runtime = data, never instructions. Why: this single boundary is what stops indirect injection (LLM01).

Tool gating / least agency

Give every tool a profile: read / write / exec / send, the exact resources it may touch, and an allowlist (never a wildcard). Block destructive flags and secret paths at the tool boundary, not in the prompt — the prompt is advisory, the boundary is enforced.

# Bad: one wildcard tool = unbounded blast radius, runs anything the loop emits
def run_shell(cmd: str) -> str:
    return subprocess.run(cmd, shell=True, capture_output=True, text=True).stdout
# Good: narrow tool, allowlisted root, denied patterns, no shell
ALLOWED_ROOT = pathlib.Path("/srv/agent/workspace").resolve()
DENY = ("*.key", "*.pem", "*secret*", "*.env", "id_rsa*")

def read_file(path: str) -> str:
    p = (ALLOWED_ROOT / path).resolve()
    if not p.is_relative_to(ALLOWED_ROOT):           # no traversal out of scope
        raise PermissionError("path outside workspace")
    if any(p.match(g) for g in DENY):                # never read secrets
        raise PermissionError("denied pattern")
    return p.read_text()
  • Issue task-scoped, short-lived tokens — not the session's broad creds. A credential should be valid only for the specific tool and the duration of one task. Why: a hijacked loop cannot reuse a session-wide token it never held.
  • Prefer read-only by default; writes/sends/exec are separate, gated tools. Why: most steps only need to read, so most steps should be unable to mutate anything.

Injection defense

Treat all external data as untrusted: user messages, retrieved documents, API responses, emails, web pages, other agents' output. Sanitize and delimit before it enters context, and never let external text reach a privileged tool unmediated.

Source Trust level Required mediation before it can act
System / developer prompt Trusted none (this is the only instruction channel)
End-user chat message Untrusted delimit; treat as data, not commands
Retrieved RAG / KB document Untrusted delimit; strip instruction-like spans
Fetched web page / API JSON Untrusted parse to schema; no raw text → tool args
Inbound email / ticket body Untrusted delimit; HITL on any action it requests
Another agent's message Untrusted same as external user input
# Bad: retrieved chunk flows straight into a privileged action
chunk = retriever.search(q)[0].text          # attacker-controlled doc
agent.call_tool("send_email", to=extract_to(chunk), body=chunk)
# Good: external content is quarantined data; the action is schema-validated + gated
chunk = retriever.search(q)[0].text
ctx = f"<retrieved untrusted>\n{chunk}\n</retrieved untrusted>"   # delimited, labeled
proposal = agent.draft("send_email", context=ctx)                # model proposes
args = SendEmail.model_validate(proposal.args)                   # schema or reject
if args.to_domain not in ALLOWED_DOMAINS:                        # exfil guard
    raise PermissionError("recipient outside allowlist")
require_human_approval("send_email", args)                       # irreversible → HITL
  • Validate every tool-call argument against a strict schema before execution. Why: a schema rejects the surprise field, encoded payload, or off-allowlist recipient injection produces.
  • Watch for exfiltration shapes — unexpected outbound URLs, base64 blobs, recipients outside the allowlist. Why: data theft is the common payload of a successful injection.

Human-in-the-loop by risk class

Do not approve every action — reported ~93% of permission prompts get approved without being read, so blanket prompting trains a rubber stamp. Gate by risk class, keyed on reversibility × blast radius. Bind each approval to the exact parameters with a short-lived token so the approved action cannot be swapped after the click.

Action type (examples) Reversible? Blast radius Control
Read file, search KB, fetch page n/a none auto
Write to scratch workspace, internal draft yes local log-only
Mutate prod DB, deploy, change config hard system approve (HITL)
Send email/payment to external party, post live no external approve (HITL)
Delete backups, rotate prod creds, mass-delete no catastrophic block (or step-up auth)
  • Step up for the top row — high-value irreversible actions deserve fresh auth, not the ambient session. Why: a hijacked session should not also hold the keys to the worst action.

Memory hygiene

  • Validate and sanitize content before it is stored. Why: memory poisoning persists across sessions (OWASP Agentic T1) — unlike session-scoped injection, a poisoned memory re-attacks every future run until purged.
  • Isolate memory per user and per session; do not let one user's writes color another's reads. Why: shared memory is a cross-tenant injection channel.
  • Expire entries and cap memory size; redact PII (SSN, cards, API keys) before persist. Why: stale instructions and leaked secrets both age into liabilities.

Runtime kill-switches

A looping or hijacked agent must hit a wall on its own. Set hard caps, fail closed:

  • Tool-call rate cap (e.g. ~30 calls/min) — runaway loops trip it before they do damage.
  • Cost cap per session (e.g. ~$10) — a wallet attack stops at a known ceiling.
  • Loop / step cap — a fixed max iterations kills the infinite plan.
  • Wall-clock timeout — a stuck agent is terminated, not left running.

Log every tool call (arguments redacted) and alert on repeated approval-bypass attempts.

Anti-patterns

Anti-pattern Why it bites Do instead
Approve every action ~93% rubber-stamped; the real risky one slips through Gate by risk class; HITL only on irreversible/external
One broad session token shared by all tools Hijacked loop reuses it everywhere Task-scoped, short-lived per-tool tokens
Trust RAG / fetched / email content Indirect injection (LLM01) becomes direct tool execution Delimit as untrusted data; mediate before any tool
Wildcard run_shell(cmd) tool Unbounded blast radius Narrow tools, allowlisted resources, denied patterns
Raw external text piped into tool args Attacker controls the action's parameters Schema-validate args; allowlist recipients/domains
Scope/limits stated only in the prompt Prompt is advisory; the model can be talked out of it Enforce at the tool/harness boundary
No loop / cost / rate cap A hijacked or looping agent runs until it runs out of money Hard fail-closed kill-switches
Redact PII only in the UI The secret was already written to memory/logs Redact before persistence, at the source
Files (rsc-harness)
  • evals
    • cases.yaml 3.2 KB
      Loading…
    • README.md 910 B
      Loading…
  • references
    • threat-model.md 4 KB
      Loading…
  • SKILL.md 10.3 KB
    ---
    name: agent-safety
    description: "Use when bounding an LLM agent that already runs — scoping its task domain, gating tools to least privilege, defending against prompt injection in untrusted web/email/RAG text, requiring human approval on irreversible actions, capping runtime and cost, or triaging what it already did. NOT building the loop, tools, or RAG (that is `building-agents`)."
    tags: [agent-security, guardrails, prompt-injection, least-privilege, owasp-agentic]
    recommends: [building-agents, security-audit, agent-eval]
    origin: risco
    ---
    
    # Agent safety
    
    You are the security review for an agent's **agency**, not for its code. The loop works,
    tools are wired, memory persists — your job is to make that autonomy *bounded*. If you
    want to review ordinary endpoints, auth, or secrets handling, that is
    `../security-audit/SKILL.md` — this skill is the *Agentic* Top 10, the risks that exist only
    because a model has tools and autonomy. If the loop or tools do not exist yet, that is
    `../building-agents/SKILL.md`. You arrive *after* both.
    
    `references/threat-model.md` carries the OWASP Agentic Top 10 2026 risks mapped to the
    controls below, the pre-ship guardrail checklist, and the incident-triage flow for "the
    agent did X" — open it when you are reviewing before ship or reconstructing an incident.
    
    ## The ownership split
    
    Agent security splits into four layers — **Model · Harness · Tools · Environment**. The
    model provider owns only the Model layer (alignment, refusals). Everything else is yours:
    the Harness (loop, memory, context assembly), the Tools (what the agent can *do*), and the
    Environment (creds, network, blast radius). Do not outsource a layer you own to "the model
    is aligned."
    
    Three excesses cause almost every agentic incident. Cut all three:
    
    - **Excessive functionality** — tools the task never needs.
    - **Excessive permissions** — broader scopes/creds than the tool needs.
    - **Excessive autonomy** — acting without checking back when it should.
    
    The operating principle is **least agency**: autonomy is earned per task, not defaulted.
    
    ## Scope limits
    
    - **Declare the allowed task domain as a hard boundary in the system prompt.** Why: an
      undeclared scope is an infinite scope; "you are a refund assistant; you do not touch
      payroll" is a constraint a reviewer can check.
    - **Deny by default — the agent starts with zero tools.** Each tool earns its place by a
      task justification. Why: an opt-out tool list grows; an opt-in list stays minimal.
    - **Segregate the instruction channel from the data channel.** System/developer prompt =
      trusted instructions. Everything the agent reads at runtime = data, never instructions.
      Why: this single boundary is what stops indirect injection (LLM01).
    
    ## Tool gating / least agency
    
    Give every tool a profile: **read / write / exec / send**, the exact resources it may
    touch, and an **allowlist** (never a wildcard). Block destructive flags and secret paths
    at the tool boundary, not in the prompt — the prompt is advisory, the boundary is enforced.
    
    ```python
    # Bad: one wildcard tool = unbounded blast radius, runs anything the loop emits
    def run_shell(cmd: str) -> str:
        return subprocess.run(cmd, shell=True, capture_output=True, text=True).stdout
    ```
    
    ```python
    # Good: narrow tool, allowlisted root, denied patterns, no shell
    ALLOWED_ROOT = pathlib.Path("/srv/agent/workspace").resolve()
    DENY = ("*.key", "*.pem", "*secret*", "*.env", "id_rsa*")
    
    def read_file(path: str) -> str:
        p = (ALLOWED_ROOT / path).resolve()
        if not p.is_relative_to(ALLOWED_ROOT):           # no traversal out of scope
            raise PermissionError("path outside workspace")
        if any(p.match(g) for g in DENY):                # never read secrets
            raise PermissionError("denied pattern")
        return p.read_text()
    ```
    
    - **Issue task-scoped, short-lived tokens — not the session's broad creds.** A credential
      should be valid only for the specific tool and the duration of one task. Why: a hijacked
      loop cannot reuse a session-wide token it never held.
    - **Prefer read-only by default; writes/sends/exec are separate, gated tools.** Why: most
      steps only need to read, so most steps should be unable to mutate anything.
    
    ## Injection defense
    
    Treat **all** external data as untrusted: user messages, retrieved documents, API
    responses, emails, web pages, other agents' output. Sanitize and delimit before it enters
    context, and **never let external text reach a privileged tool unmediated.**
    
    | Source                         | Trust level | Required mediation before it can act        |
    | ------------------------------ | ----------- | ------------------------------------------- |
    | System / developer prompt      | Trusted     | none (this is the only instruction channel) |
    | End-user chat message          | Untrusted   | delimit; treat as data, not commands        |
    | Retrieved RAG / KB document    | Untrusted   | delimit; strip instruction-like spans       |
    | Fetched web page / API JSON    | Untrusted   | parse to schema; no raw text → tool args    |
    | Inbound email / ticket body    | Untrusted   | delimit; HITL on any action it requests     |
    | Another agent's message        | Untrusted   | same as external user input                 |
    
    ```python
    # Bad: retrieved chunk flows straight into a privileged action
    chunk = retriever.search(q)[0].text          # attacker-controlled doc
    agent.call_tool("send_email", to=extract_to(chunk), body=chunk)
    ```
    
    ```python
    # Good: external content is quarantined data; the action is schema-validated + gated
    chunk = retriever.search(q)[0].text
    ctx = f"<retrieved untrusted>\n{chunk}\n</retrieved untrusted>"   # delimited, labeled
    proposal = agent.draft("send_email", context=ctx)                # model proposes
    args = SendEmail.model_validate(proposal.args)                   # schema or reject
    if args.to_domain not in ALLOWED_DOMAINS:                        # exfil guard
        raise PermissionError("recipient outside allowlist")
    require_human_approval("send_email", args)                       # irreversible → HITL
    ```
    
    - **Validate every tool-call argument against a strict schema before execution.** Why: a
      schema rejects the surprise field, encoded payload, or off-allowlist recipient injection
      produces.
    - **Watch for exfiltration shapes** — unexpected outbound URLs, base64 blobs, recipients
      outside the allowlist. Why: data theft is the common payload of a successful injection.
    
    ## Human-in-the-loop by risk class
    
    Do **not** approve every action — reported ~93% of permission prompts get approved without
    being read, so blanket prompting trains a rubber stamp. Gate by **risk class**, keyed on
    reversibility × blast radius. Bind each approval to the **exact parameters** with a
    short-lived token so the approved action cannot be swapped after the click.
    
    | Action type (examples)                          | Reversible? | Blast radius | Control            |
    | ----------------------------------------------- | ----------- | ------------ | ------------------ |
    | Read file, search KB, fetch page                | n/a         | none         | **auto**           |
    | Write to scratch workspace, internal draft      | yes         | local        | **log-only**       |
    | Mutate prod DB, deploy, change config            | hard        | system       | **approve (HITL)** |
    | Send email/payment to external party, post live | no          | external     | **approve (HITL)** |
    | Delete backups, rotate prod creds, mass-delete  | no          | catastrophic | **block** (or step-up auth) |
    
    - **Step up for the top row** — high-value irreversible actions deserve fresh auth, not the
      ambient session. Why: a hijacked session should not also hold the keys to the worst action.
    
    ## Memory hygiene
    
    - **Validate and sanitize content before it is stored.** Why: memory poisoning persists
      across sessions (OWASP Agentic T1) — unlike session-scoped injection, a poisoned memory
      re-attacks every future run until purged.
    - **Isolate memory per user and per session; do not let one user's writes color another's
      reads.** Why: shared memory is a cross-tenant injection channel.
    - **Expire entries and cap memory size; redact PII (SSN, cards, API keys) before persist.**
      Why: stale instructions and leaked secrets both age into liabilities.
    
    ## Runtime kill-switches
    
    A looping or hijacked agent must hit a wall on its own. Set hard caps, fail closed:
    
    - **Tool-call rate cap** (e.g. ~30 calls/min) — runaway loops trip it before they do damage.
    - **Cost cap per session** (e.g. ~$10) — a wallet attack stops at a known ceiling.
    - **Loop / step cap** — a fixed max iterations kills the infinite plan.
    - **Wall-clock timeout** — a stuck agent is terminated, not left running.
    
    Log every tool call (arguments redacted) and alert on repeated approval-bypass attempts.
    
    ## Anti-patterns
    
    | Anti-pattern                                   | Why it bites                                                   | Do instead                                          |
    | ---------------------------------------------- | ------------------------------------------------------------- | --------------------------------------------------- |
    | Approve every action                           | ~93% rubber-stamped; the real risky one slips through         | Gate by risk class; HITL only on irreversible/external |
    | One broad session token shared by all tools    | Hijacked loop reuses it everywhere                            | Task-scoped, short-lived per-tool tokens            |
    | Trust RAG / fetched / email content            | Indirect injection (LLM01) becomes direct tool execution      | Delimit as untrusted data; mediate before any tool  |
    | Wildcard `run_shell(cmd)` tool                 | Unbounded blast radius                                        | Narrow tools, allowlisted resources, denied patterns |
    | Raw external text piped into tool args         | Attacker controls the action's parameters                     | Schema-validate args; allowlist recipients/domains  |
    | Scope/limits stated only in the prompt         | Prompt is advisory; the model can be talked out of it         | Enforce at the tool/harness boundary                |
    | No loop / cost / rate cap                       | A hijacked or looping agent runs until it runs out of money   | Hard fail-closed kill-switches                      |
    | Redact PII only in the UI                       | The secret was already written to memory/logs                 | Redact before persistence, at the source            |
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.