# Hook Evaluation Criteria

Detailed scoring rubric and quality gates for hook evaluation.

## Mathematical Foundation

This evaluation framework follows Multi-Criteria Decision Analysis (MCDA) best practices:

- **Normalization**: Vector normalization for scale invariance ([full methodology](../../skills-eval/modules/evaluation-criteria.md))
- **Weighting**: Security-first weights with stakeholder validation
- **Aggregation**: Weighted sum with penalty-based security scoring
- **Validation**: Sensitivity analysis on non-security weights

**Documentation**: See [Multi-Metric Evaluation Methodology](../../skills-eval/modules/evaluation-criteria.md) for complete mathematical foundation.

## Scoring System (100 points total)

### Security Analysis (30 points)

**Vulnerability Detection:**
- Critical vulnerabilities: -15 points each
- High-risk issues: -8 points each
- Medium-risk issues: -4 points each
- Low-risk issues: -1 point each

**Security Checklist:**

| Check | Severity | Points Lost |
|-------|----------|-------------|
| Dynamic code evaluation with user input | Critical | -15 |
| Command injection vulnerability | Critical | -15 |
| Unvalidated file path access | High | -8 |
| Secrets/credentials in code | High | -8 |
| Missing input validation | Medium | -4 |
| Overly permissive patterns | Medium | -4 |
| No rate limiting | Low | -1 |
| Verbose error messages exposing internals | Low | -1 |

### Performance Analysis (25 points)

| Metric | Max Points | Criteria |
|--------|------------|----------|
| Execution time efficiency | 10 | PreToolUse <100ms, PostToolUse <200ms |
| Memory usage optimization | 8 | <50MB for simple hooks, <100MB for complex |
| I/O operation efficiency | 4 | Minimal file/network operations |
| Resource cleanup | 3 | Proper cleanup of handles, connections |

**Performance Thresholds:**

```yaml
pre_tool_use:
  excellent: <50ms
  good: <100ms
  acceptable: <200ms
  poor: >200ms

post_tool_use:
  excellent: <100ms
  good: <200ms
  acceptable: <500ms
  poor: >500ms

memory:
  excellent: <25MB
  good: <50MB
  acceptable: <100MB
  poor: >100MB
```

### Compliance Analysis (20 points)

| Aspect | Max Points | Requirements |
|--------|------------|--------------|
| Structure compliance | 8 | Valid JSON/Python, correct schema |
| Documentation completeness | 6 | Purpose, parameters, return values documented |
| Error handling | 4 | All exceptions caught, meaningful messages |
| Best practices | 2 | Follows hook authoring guidelines |

**Structure Requirements:**

- JSON hooks: Valid JSON schema with required fields
- Python hooks: Type hints, async/await patterns
- Matcher patterns: Valid regex, appropriate scope

### Reliability Analysis (15 points)

| Aspect | Max Points | Requirements |
|--------|------------|--------------|
| Error handling robustness | 6 | Graceful handling of all error conditions |
| Timeout management | 4 | Appropriate timeouts configured |
| Idempotency | 3 | Safe to retry without side effects |
| Graceful degradation | 2 | Falls back safely on failure |

**Reliability Checklist:**

- [ ] Hook returns valid response on all code paths
- [ ] Exceptions are caught and handled
- [ ] Timeout is configured appropriately
- [ ] Hook can be called multiple times safely
- [ ] Failure doesn't break agent operation

### Maintainability (10 points)

| Aspect | Max Points | Requirements |
|--------|------------|--------------|
| Code structure | 4 | Clear, modular, single responsibility |
| Documentation clarity | 3 | Purpose and behavior well explained |
| Modularity | 2 | Reusable components, no duplication |
| Test coverage | 1 | Tests exist for key functionality |

## Quality Levels

| Score | Level | Description |
|-------|-------|-------------|
| 91-100 | Excellent | Production-ready, follows all best practices |
| 76-90 | Good | Minor improvements suggested |
| 51-75 | Acceptable | Some issues requiring attention |
| 26-50 | Poor | Significant issues need addressing |
| 0-25 | Critical | Major security or reliability issues |

## Quality Gates

Default thresholds for CI/CD integration:

```yaml
quality_gates:
  security_score: ">= 80"
  performance_score: ">= 70"
  compliance_score: ">= 85"
  reliability_score: ">= 85"
  overall_score: ">= 75"
  max_critical_issues: 0
  max_high_issues: 2
```

### Sensitivity Analysis Requirements

Security weights are non-negotiable, but other weights should be validated:

```yaml
sensitivity_analysis:
  # Security weights are fixed (non-negotiable)
  fixed_weights: ["security_analysis"]

  # Other weights tested for sensitivity
  test_weights: ["performance", "compliance", "reliability", "maintainability"]
  variation: 0.20  # ±20% weight variation

  requirements:
    stable_rankings: true  # Rankings shouldn't change (except security)
    critical_weights_identified: true  # Document sensitive weights
```

See [Sensitivity Analysis](../../skills-eval/modules/evaluation-criteria.md#sensitivity-analysis-requirements) for implementation details.

### Gate Behaviors

| Gate | Failure Action |
|------|----------------|
| `security_score` | Block deployment, require review |
| `performance_score` | Warn, suggest optimization |
| `compliance_score` | Block until documentation complete |
| `reliability_score` | Block deployment |
| `max_critical_issues` | Immediate block |

## Issue Classification

### Critical Issues (Immediate Action Required)

- Dynamic code evaluation with untrusted input
- Command injection vulnerabilities
- Credential exposure
- Unhandled exceptions that break agent

### High Issues (Address Before Release)

- Missing input validation
- Performance exceeds thresholds
- Missing error handling
- Insecure file operations

### Medium Issues (Address Soon)

- Missing documentation
- Suboptimal patterns
- Minor performance concerns
- Code style violations

### Low Issues (Nice to Fix)

- Minor documentation gaps
- Formatting inconsistencies
- Optimization opportunities
- Enhanced logging suggestions

## Evaluation Report Format

### Summary Format

```
=== Hooks Evaluation Report ===
Plugin: {name} (v{version})
Scope: {scope}
Total hooks: {count} ({json_count} JSON, {python_count} Python)

=== Scores ===
Security:      {score}/100 ({level})
Performance:   {score}/100 ({level})
Compliance:    {score}/100 ({level})
Reliability:   {score}/100 ({level})
Maintainability: {score}/100 ({level})
────────────────────────────────
Overall:       {score}/100 ({level})

=== Issues ===
Critical: {count}
High: {count}
Medium: {count}
Low: {count}
```

### Detailed Format

Includes per-hook breakdown:

```
=== Hook: {hook_path} ===
Type: {json|python}
Event: {PreToolUse|PostToolUse|...}
Matcher: {pattern|universal}

Security Issues:
  [{severity}] Line {n}: {description}

Performance:
  Estimated time: {ms}ms (threshold: {threshold}ms)
  Memory usage: {mb}MB (threshold: {threshold}MB)

Recommendations:
  1. {recommendation}
  2. {recommendation}
```

## Customization

### Per-Plugin Configuration

Create `.hooks-eval.yaml` in plugin root:

```yaml
hooks_eval:
  # Override security thresholds
  security_thresholds:
    critical_score: 80
    high_score: 70

  # Override performance thresholds
  performance_thresholds:
    pre_tool_use_max_ms: 100
    post_tool_use_max_ms: 200
    max_memory_mb: 50

  # Compliance requirements
  compliance_requirements:
    require_documentation: true
    require_error_handling: true
    require_timeout_config: true

  # Custom rules
  custom_rules:
    - name: "no-hardcoded-secrets"
      pattern: "password|secret|token"
      severity: "high"
    - name: "require-shebang"
      pattern: "^#!"
      file_types: [".sh", ".py"]
      severity: "medium"

  # Excluded paths
  exclude_paths:
    - "hooks/experimental/*"
    - "hooks/deprecated/*"
```

### Severity Overrides

Override default severity for specific patterns:

```yaml
severity_overrides:
  - pattern: "subprocess.run"
    default_severity: "high"
    override_severity: "medium"
    reason: "Safe usage verified in review"
```

# Python SDK Hook Types

Complete reference for Claude Agent SDK hook types, callbacks,
and matchers.

## Hook Events

### HookEvent

Supported hook event types in the Python SDK.

```python
from typing import Literal

HookEvent = Literal[
    "Setup",  # Called when plugin installed/enabled
    "SessionStart",  # Called when session begins
    "SessionEnd",  # Called when session ends normally
    "UserPromptSubmit",  # Called when user submits a prompt
    "PreToolUse",  # Called before tool execution
    "PostToolUse",  # Called after tool execution
    "PostToolUseFailure",  # Called when tool execution fails (2.1.20+)
    "PermissionRequest",  # Called when permission dialog would appear
    "Notification",  # Called on system notification (2.1.20+)
    "SubagentStart",  # Called when subagent spawns (2.1.20+)
    "SubagentStop",  # Called when a subagent stops
    "Stop",  # Called when stopping execution
    "TeammateIdle",  # Called when teammate agent becomes idle (2.1.33+)
    "TaskCompleted",  # Called when a task finishes execution (2.1.33+)
    "ConfigChange",  # Called when config is modified (2.1.49+)
    "InstructionsLoaded",  # Called when instructions are loaded (2.1.33+)
    "PreCompact",  # Called before message compaction
    "PostCompact",  # Called after compaction (2.1.76+)
    "WorktreeCreate",  # Called when git worktree is created (2.1.50+)
    "WorktreeRemove",  # Called when git worktree is removed (2.1.50+)
    "StopFailure",  # Called on error (2.1.78+)
    "TaskCreated",  # Called when task created (2.1.84+)
    "CwdChanged",  # Called on working dir change (2.1.83+)
    "FileChanged",  # Called on file change (2.1.83+)
    "Elicitation",  # MCP elicitation request (2.1.76+)
    "ElicitationResult",  # MCP elicitation response (2.1.76+)
]
```

**SDK vs CLI availability**: Most events work in both JSON
hooks (CLI) and Python SDK hooks. `PermissionRequest` is
CLI-only. `Setup`, `SessionStart`, `SessionEnd`, and
`Notification` are CLI-only (JSON hooks).
`WorktreeCreate` and `WorktreeRemove` are command-only
hooks (no Python SDK callback). They do not support
matchers.

### Event Summary

| Event | Trigger | Blockable | Matcher |
|-------|---------|-----------|---------|
| `Setup` | Plugin installed/enabled | No | No |
| `SessionStart` | Session begins | No | No |
| `SessionEnd` | Session ends normally | No | No |
| `UserPromptSubmit` | User submits input | No | No |
| `PreToolUse` | Before any tool runs | Yes | Tool name |
| `PostToolUse` | After tool completes | No | Tool name |
| `PostToolUseFailure` | Tool execution fails | No | Tool name |
| `PermissionRequest` | Permission dialog | Yes | Tool name |
| `SubagentStart` | Subagent spawns | No | No |
| `SubagentStop` | Subagent completes | No | No |
| `Stop` | Agent stops | No | No |
| `TeammateIdle` | Teammate idle | No | No |
| `TaskCompleted` | Task finishes | No | No |
| `ConfigChange` | Config modified | No | No |
| `InstructionsLoaded` | Instructions loaded | No | No |
| `PreCompact` | Before compaction | No | No |
| `PostCompact` | After compaction | No | No |
| `WorktreeCreate` | Worktree created | No | No |
| `WorktreeRemove` | Worktree removed | No | No |
| `StopFailure` | Error occurs | No | Error type |
| `TaskCreated` | Task created | Yes | No |
| `CwdChanged` | Directory changed | No | No |
| `FileChanged` | File changed | No | Filename |
| `Elicitation` | MCP elicitation | Yes | MCP server |
| `ElicitationResult` | Elicitation response | Yes | MCP server |

### Notable Version Changes

All hook events include `agent_id` and `agent_type` as
of 2.1.69+.

| Version | Change |
|---------|--------|
| 2.1.69 | `TeammateIdle`/`TaskCompleted` support `{"continue": false}` for graceful shutdown |
| 2.1.69 | Plugin WorktreeCreate/WorktreeRemove hooks fire correctly (were silently ignored) |
| 2.1.71 | New tools: `CronCreate`, `CronList`, `CronDelete` appear in PreToolUse/PostToolUse |
| 2.1.72 | `ExitWorktree` tool added; `lsof`/`pgrep`/`tput`/`ss`/`fd`/`fdfind` auto-approved |
| 2.1.72 | Skill hook double-fire fixed; `transcript_path` correct for resumed sessions |
| 2.1.72 | Failed Read/WebFetch/Glob no longer cancel sibling tool calls (only Bash cascades) |
| 2.1.73 | SessionStart no longer double-fires on `--resume`/`--continue` |
| 2.1.73 | JSON-output hooks no longer inject spurious system-reminder messages |
| 2.1.74 | SessionEnd hooks timeout now configurable via `CLAUDE_CODE_SESSIONEND_HOOKS_TIMEOUT_MS` |
| 2.1.75 | Hook source displayed in permission prompts; async hook messages suppressed by default |
| 2.1.76 | `Elicitation` and `ElicitationResult` events for MCP servers |
| 2.1.76 | `PostCompact` event fires after context compaction |
| 2.1.77 | PreToolUse "allow" no longer bypasses deny rules (security fix) |
| 2.1.83 | `CwdChanged` and `FileChanged` events added |
| 2.1.84 | `TaskCreated` event (blockable); HTTP hooks can return worktree path |
| 2.1.85 | `if` field for conditional hook execution; PreToolUse can match `AskUserQuestion` |

## Type Definitions

### HookCallback

```python
from typing import Any, Awaitable, Callable

HookCallback = Callable[
    [dict[str, Any], str | None, HookContext], Awaitable[dict[str, Any]]
]
```

| Parameter | Type | Description |
|-----------|------|-------------|
| `input_data` | `dict[str, Any]` | Hook-specific input data (varies by event) |
| `tool_use_id` | `str \| None` | Tool use identifier (for tool-related hooks) |
| `context` | `HookContext` | Additional context information |

**Returns:** `dict[str, Any]` with optional fields:
`decision` ("block"), `systemMessage` (str),
`hookSpecificOutput` (dict).

### HookMatcher

```python
@dataclass
class HookMatcher:
    matcher: str | None = None
    hooks: list[HookCallback] = field(default_factory=list)
    timeout: float | None = None  # Default: 60s
```

| Pattern | Matches |
|---------|---------|
| `"Bash"` | Only Bash tool |
| `"Write\|Edit"` | Write OR Edit tools |
| `None` | All tools (universal matcher) |

## Complete Usage Example

```python
from claude_agent_sdk import query, ClaudeAgentOptions, HookMatcher, HookContext
from typing import Any


async def validate_bash_command(
    input_data: dict[str, Any], tool_use_id: str | None, context: HookContext
) -> dict[str, Any]:
    """Block dangerous bash commands."""
    if input_data["tool_name"] == "Bash":
        command = input_data["tool_input"].get("command", "")
        if "rm -rf /" in command:
            return {
                "hookSpecificOutput": {
                    "hookEventName": "PreToolUse",
                    "permissionDecision": "deny",
                    "permissionDecisionReason": "Dangerous command blocked",
                }
            }
    return {}


async def log_tool_use(
    input_data: dict[str, Any], tool_use_id: str | None, context: HookContext
) -> dict[str, Any]:
    """Log all tool usage for auditing."""
    print(f"Tool used: {input_data.get('tool_name')}")
    return {}


options = ClaudeAgentOptions(
    hooks={
        "PreToolUse": [
            HookMatcher(matcher="Bash", hooks=[validate_bash_command], timeout=120),
            HookMatcher(hooks=[log_tool_use]),
        ],
        "PostToolUse": [HookMatcher(hooks=[log_tool_use])],
    }
)

async for message in query(prompt="Analyze this codebase", options=options):
    print(message)
```

## Input Data by Event Type

### PreToolUse / PostToolUse

```python
# PreToolUse
{"tool_name": "Bash", "tool_input": {"command": "ls -la"}}

# PostToolUse (adds result)
{
    "tool_name": "Bash",
    "tool_input": {"command": "ls -la"},
    "tool_result": "file1.txt\nfile2.txt",
    "error": None,
}
```

### PermissionRequest (CLI only)

```python
# Input
{
    "session_id": "abc123",
    "tool_name": "Bash",
    "tool_input": {"command": "npm install"},
    "permission_mode": "default",
    "cwd": "/path/to/project",
}

# Output: allow
{
    "hookSpecificOutput": {
        "hookEventName": "PermissionRequest",
        "decision": {"behavior": "allow"},
    }
}

# Output: deny
{
    "hookSpecificOutput": {
        "hookEventName": "PermissionRequest",
        "decision": {"behavior": "deny", "message": "Reason"},
    }
}
```

### Other Events

| Event | Key Fields |
|-------|------------|
| `UserPromptSubmit` | `prompt`, `conversation_id` |
| `TeammateIdle` | `agent_id`, `session_id` |
| `TaskCompleted` | `task_id`, `result`, `duration_ms`, `token_count` |
| `Stop` / `SubagentStop` | `reason`, `final_message` |
| `PreCompact` | `messages`, `token_count` |
| `PostCompact` | `trigger` ("manual"/"auto"), `compact_summary` |
| `WorktreeCreate` | `name` (must print worktree path to stdout) |
| `WorktreeRemove` | `worktree_path` (cannot block removal) |

## Hook Return Patterns

```python
# Allow (default)
return {}

# Block action
return {
    "hookSpecificOutput": {
        "hookEventName": "PreToolUse",
        "permissionDecision": "deny",
        "permissionDecisionReason": "Explanation",
    }
}

# Add system message
return {"systemMessage": "Important context added to conversation"}
```

## Best Practices

| Area | Guidance |
|------|----------|
| Performance | Keep hooks fast (<100ms PreToolUse, <200ms PostToolUse) |
| Performance | Use appropriate timeouts; cache expensive computations |
| Security | Validate all input; never use dynamic code eval with hook input |
| Security | Use allowlists over blocklists; sanitize log data |
| Reliability | Always return a dict (even empty `{}`); handle exceptions |
| Reliability | Design hooks to be idempotent; include meaningful block reasons |
| Testing | Test with various input patterns; verify timeout behavior |