{
  "schema_version": 1,
  "skill_name": "software-architecture-analysis",
  "evals": [
    {
      "id": "quality-characteristics-from-evidence",
      "prompt": "Review this repository's architecture for reliability, privacy, and maintainability. The README claims it is highly reliable and privacy-first, but the repository only contains request handlers, a retry helper, a hosted analytics SDK, and no dashboards or recovery tests. Produce an evidence-based assessment.",
      "expected_output": "A reverse-engineering assessment that separates README claims from observed repository evidence, analyzes reliability, privacy, and maintainability through concrete scenarios, marks unsupported conclusions as unknown, identifies affected components and owners, and proposes the smallest probes that would reduce uncertainty.",
      "assertions": [
        "The output distinguishes observed repository evidence from README-reported claims",
        "Reliability is assessed through failure behavior and recovery evidence rather than the presence of a retry helper alone",
        "The hosted analytics SDK is identified as a privacy-relevant data-flow dependency",
        "Unknown or unverified characteristics are explicitly labeled",
        "Each major finding includes a concrete follow-up observation or test"
      ]
    },
    {
      "id": "six-lens-boundary-assessment",
      "prompt": "We are considering splitting billing out of a modular monolith. Billing code has few imports from the rest of the system, but it shares customer tables, participates in a synchronous checkout request, must deploy with tax rules, and is owned by the same on-call team. Assess the boundary without assuming microservices are the answer.",
      "expected_output": "A candidate-boundary report using static, dynamic, data, temporal, deployment, and organizational coupling, plus change coupling. It explains why low import coupling is insufficient, compares a bounded extraction with retaining or strengthening the modular monolith, and gives a readiness verdict with a reversible probe and stop condition.",
      "assertions": [
        "All six requested coupling lenses are addressed separately",
        "Shared customer data and invariant ownership are treated as data coupling evidence",
        "Synchronous checkout and deployment-together requirements are treated as dynamic, temporal, or deployment coupling",
        "The analysis includes retaining a modular monolith as an explicit alternative",
        "The verdict includes a reversible probe and a stop condition rather than an unconditional service recommendation"
      ]
    },
    {
      "id": "data-ownership-and-reconciliation",
      "prompt": "Trace an order workflow that writes an order database, charges a payment provider, publishes an event, and updates a search index. The payment can succeed while event publication fails, and the index sometimes lags. Produce a data ownership and recovery analysis.",
      "expected_output": "A timeline-based analysis that identifies authoritative data and projections, transaction boundaries, the workflow's orchestration or choreography shape, partial completion states, duplicate and delayed outcomes, detection and repair ownership, and a concrete reconciliation design with precedence, idempotency, audit evidence, and completion proof.",
      "assertions": [
        "The output distinguishes authoritative order or payment records from the search projection",
        "It identifies the actual boundary of each transaction and does not call the whole workflow atomic",
        "It names the partial state where payment succeeds but publication fails",
        "It addresses duplicate, delayed, and unavailable outcomes with detection and recovery behavior",
        "The reconciliation path includes comparison keys, source precedence, safe repair, audit evidence, and completion proof"
      ]
    },
    {
      "id": "clean-room-health-template",
      "prompt": "Reverse-engineer a reference application's local-first replacement and deliver an architecture health assessment. The source repository includes a web client, hosted database, background sync, and third-party transcription service. Do not copy source code or private identifiers.",
      "expected_output": "A clean-room health assessment with scope, evidence ledger, architecture characteristics, coupling and modularity, data ownership and workflow behavior, privacy-relevant dependencies, decomposition readiness where relevant, prioritized findings, and neighboring-skill handoffs. It contains no source code or recognizable private identifiers from the reference.",
      "assertions": [
        "The output contains a scope and evidence ledger with claim classification and confidence",
        "Local versus hosted processing and the transcription dependency are mapped as privacy-relevant architecture evidence",
        "The output includes coupling, data/workflow, and decomposition sections or clearly states why one is not applicable",
        "The clean-room output contains no source code snippets or private identifiers",
        "The handoff names API, data, implementation, platform, security, or migration ownership where applicable"
      ]
    },
    {
      "id": "provider-contract-extraction",
      "prompt": "A codebase uses a relational database for graph storage, vector search, and traversal. Catalog the implicit storage contract and design a provider-agnostic abstraction that could also be implemented by a graph database. Preserve the project's documented constraint that graph edges are intentionally untyped.",
      "expected_output": "An interface-extraction analysis that reads the project's philosophy first, catalogs storage operations, identifies substrate workarounds as boundary leaks, preserves the untyped-edge constraint, and proposes a two-provider proof without copying source code.",
      "assertions": [
        "The analysis starts from the project's documented philosophy or design constraints",
        "It catalogs CRUD, vector, traversal, maintenance, lifecycle, and transaction needs",
        "It identifies workarounds such as fallback search or hand-written traversal as provider-boundary concerns",
        "The proposed abstraction preserves untyped graph edges",
        "A second-provider proof is used to test for contract leaks and missing operations"
      ]
    },
    {
      "id": "greenfield-boundary-routing",
      "prompt": "Design a new event-driven payments architecture from business requirements. Choose services, databases, message brokers, consistency rules, and deployment topology before any existing codebase has been provided.",
      "expected_output": "A boundary response that explains this skill is for reverse-engineering an existing implementation and clean-room redesign, not unconstrained greenfield architecture. It routes the proactive architecture decision to the appropriate architecture methodology and identifies the evidence this skill would need if a reference system later becomes available.",
      "assertions": [
        "The response does not invent a reverse-engineering report without an existing codebase",
        "It explicitly distinguishes clean-room analysis from greenfield architecture design",
        "It routes interface, data, platform, security, and migration details to neighboring owners where relevant",
        "It states the evidence needed before this skill could assess an existing system"
      ]
    }
  ]
}

# Architecture Characteristics Analysis

Use this reference when a reverse-engineering report must explain which qualities the system actually exhibits or needs to preserve. Do not start with a catalog of adjectives. Start with evidence about a user-visible or operator-visible scenario.

## Evidence loop

1. **Collect scenarios.** Use requirements, support cases, runbooks, tests, dashboards, incident records, configuration, and code paths. Record who needs what outcome, under which load or failure, and how success is recognized.
2. **Name the characteristic.** Translate the scenario into a quality concern such as latency stability, availability, recoverability, auditability, change isolation, portability, privacy, or operability. Keep the scenario and label separate; the label is a shorthand, not proof.
3. **Locate the responsibility.** Map the characteristic to components, data stores, queues, deployment units, and teams. A quality that has no owner is an architecture risk even if the design document names it.
4. **Test the claim.** Prefer measured behavior or an executable control. If the only support is a stakeholder statement, mark it reported. If no evidence exists, mark it unknown and state the smallest useful observation to obtain.
5. **Record tension.** Characteristics compete. State the tradeoff in the system's terms, for example, stronger isolation increasing operational work or synchronous confirmation reducing latency tolerance.

## Scenario record

Capture each important characteristic as:

| Field | What to record |
|---|---|
| Actor and trigger | Who starts the scenario and what changes? |
| Stimulus and boundary | Request, failure, load, deployment, or policy event; affected components and data |
| Response measure | Latency, correctness, recovery time, audit evidence, operator action, or user outcome |
| Current evidence | Artifact and date/version; classify as observed, reported, inferred, or unknown |
| Responsible elements | Components, stores, runtime/deployment units, and teams |
| Tension and risk | What another characteristic or dependency makes difficult |
| Next probe | Test, trace, metric, interview, or document needed to reduce uncertainty |

## Quality-characteristic cautions

- Do not call a system “scalable” because it has replicas. Identify the constrained resource, demand shape, scaling mechanism, and observed limit.
- Do not call a system “resilient” because it retries. Check retry scope, idempotency, backoff, timeout budgets, downstream overload, and recovery evidence.
- Do not call a system “secure” from architecture shape alone. Record the relevant threat evidence and route a security assessment to `secure-software-engineering`.
- Do not infer maintainability from folder structure. Use change history, dependency direction, test seams, ownership, and time-to-change evidence.
- Do not convert a desired quality into a current-state fact. Report “required,” “observed,” and “unverified” separately.

## Output

End the section with a prioritized table: characteristic, scenario, current evidence, affected boundary, risk if unchanged, confidence, and next observation. This keeps the analysis useful without pretending to design the future system.

# Coupling, Modularity, and Decomposition

Use this reference when the question is whether architecture boundaries contain change, failure, ownership, or deployment consequences. A boundary is not justified by a component name or by a preference for services.

## Six coupling lenses

Inspect the same candidate boundary through these lenses and keep the evidence separate:

| Lens | Questions and evidence |
|---|---|
| Static | Which modules import, call, inherit from, or directly reach into one another? Are there cycles, shared utilities with business meaning, or hidden side doors? |
| Dynamic | Which runtime calls, messages, callbacks, retries, and fan-out paths cross the boundary? What happens on timeout, duplication, or partial completion? |
| Data | Which tables, records, files, caches, indexes, and schemas are read or written by each part? Who defines invariants and can safely change the data? |
| Temporal | Which steps must occur in order, within one request, within a time window, or after an earlier event? Does a shared clock or sequence create a hidden dependency? |
| Deployment | Which parts must be released, scaled, configured, rolled back, or restored together? Do they share a process, image, database, secret, or maintenance window? |
| Organizational | Which team owns the code, data, on-call burden, and decision rights? Do team boundaries align with the proposed boundary or create a coordination tax? |

Also inspect change coupling: use version history, incident fixes, and release notes to see whether files or capabilities change together. A low import count does not prove low change coupling.

## Modularity signals

Positive signals include a coherent reason to change, explicit inputs and outputs, owned invariants, replaceable dependencies, failure containment, independent verification, and an owner able to operate the unit. Negative signals include cycles, shared mutable state, cross-boundary transactions, synchronous fan-out, duplicated policy, coordination-heavy releases, and an interface that exposes internal data shape.

Classify each signal as observed, reported, inferred, or unknown. Do not use a numeric score as a substitute for judgment; a single cross-boundary invariant can outweigh many clean imports.

## Decomposition readiness

Assess readiness in this order:

1. State the reason for considering a split: change isolation, scaling asymmetry, fault containment, team ownership, regulatory isolation, or another evidenced pressure.
2. Identify the smallest capability and its invariants, data authority, inbound/outbound dependencies, and operational responsibilities.
3. Test whether the boundary can tolerate asynchronous or independently deployed behavior. Name the consistency and recovery consequences rather than assuming a message solves them.
4. Estimate the new coordination surface: contracts, observability, deployment, access, testing, support, data migration, and reconciliation.
5. Compare alternatives: retain a modular monolith, isolate a process without a service boundary, extract a library/package, use a queue, or split a deployable unit. Select “not ready” when evidence does not support the added cost.

The output should state a readiness verdict such as `ready for a bounded experiment`, `needs seam work`, `retain current boundary`, or `insufficient evidence`. It should list reversible probes and a stop condition. A decomposition recommendation is outside this skill's execution scope; route an approved migration to `migration-engineering`.

## Boundary report

For each candidate boundary, report: purpose, six-lens evidence, change coupling, data ownership, consistency model, failure containment, team/operator fit, coordination cost, alternatives rejected, confidence, and the next reversible probe.

# Data Ownership and Workflow Analysis

Use this reference when architecture behavior depends on who may change data, how multi-step work completes, or how the system repairs partial outcomes.

## Data authority map

For every important entity or fact, identify:

- authoritative store and write path;
- schema or semantic owner;
- allowed writers and readers;
- derived copies, caches, indexes, exports, and event projections;
- invariant owner and validation point;
- retention, deletion, replay, and backfill behavior;
- evidence for the map and unresolved conflicts.

Distinguish “source of truth” from “most frequently queried copy.” A projection may be operationally critical without owning the fact. Shared tables with multiple business writers are a strong coupling signal and require explicit invariant ownership.

## Distributed transaction trace

Trace each multi-resource operation as a timeline, not just a component diagram. Record the trigger, writes, reads, emitted messages, acknowledgement points, timeout/retry behavior, and visible intermediate states. Then answer:

1. What must be atomic, and where is that atomicity actually enforced?
2. What can be repeated safely, and what idempotency key or deduplication evidence supports that claim?
3. What may commit in one resource while another fails?
4. Who detects and repairs the split state?
5. What does the user or downstream consumer observe while repair is pending?

Do not describe a workflow as transactional merely because one database call is transactional. Name the boundary of each transaction and the remaining business consistency mechanism.

## Orchestration and choreography

Classify the workflow from observed control flow:

- **Orchestration:** a named coordinator chooses steps, tracks progress, applies timeouts, and exposes completion or compensation state.
- **Choreography:** participants react to facts or commands without one central coordinator; trace implicit ordering, duplicate handling, and how operators discover a stuck process.
- **Hybrid:** central policy or admission with event-driven local reactions.

The labels are descriptive. Evaluate observability, ownership, coupling, and recovery rather than treating either shape as inherently superior.

## Failure and reconciliation

For each boundary crossing, enumerate lost, delayed, duplicated, reordered, rejected, malformed, and permanently unavailable outcomes. For each outcome record detection signal, retry policy, idempotency behavior, dead-letter or quarantine path, operator owner, user-visible state, and recovery evidence.

Reconciliation is a first-class workflow, not a batch apology. Define its comparison keys, source precedence, safe repair action, conflict policy, audit trail, rate limits, and completion proof. If no reconciliation path exists, mark the workflow as an unresolved integrity risk. Route detailed event/API contract design to `api-design-and-evolution` and implementation mechanics to `data-engineering` or `backend-engineering`.

# Interface Extraction — Worked Example

This reference demonstrates the Phase 3b interface extraction pattern using the [cashew thought-graph library](https://github.com/rajkripal/cashew) (MIT license) as the target codebase. The goal was to extract the implicit storage contract to design a provider-agnostic DAO interface enabling swappable backends (sqlite-vec, KuzuDB, DuckDB+vss, etc.).

This is a teaching example — the methodology, not the specific project, is what generalizes.

## The Five Steps

### Step 1 — Philosophy First

Read PHILOSOPHY.md, DESIGN.md, and README.md before touching code. The cashew repo's philosophy revealed a critical constraint: **"dumb graph, smart reasoning layer"** — edges carry no type labels, node types are descriptive hints for the LLM only, not load-bearing for graph engine operations. This constraint must be baked into the contract. The `Edge` dataclass has no `type` or `label` field.

### Step 2 — Catalog Every Storage Operation

Read all core modules and extracted every distinct database operation:

| Module | Operations | Current Implementation |
|--------|-----------|-----------------------|
| db.py | Node CRUD, connection mgmt, schema migration | Thin sqlite3 wrappers, schema in constants |
| embeddings.py | KNN search, dual-write, dim-mismatch, novelty check | vec0 fast path + numpy O(N) fallback + text Jaccard fallback |
| sleep.py | Cross-link candidates, dedup clusters, GC, core promotion | sklearn pairwise matrix, Bron-Kerbosch, random sampling via SQL |
| retrieval.py | Embedding search → BFS walk → hybrid score | vec0 query + recursive CTEs + manual scoring |
| traversal.py | Trace derivation paths, audit | Recursive CTEs, UNION BFS, DFS in Python |
| session.py | Context assembly, access tracking, node creation | db.py primitives + timestamp bumps |
| graph_utils.py | Load embeddings for batch operations | Full embeddings table → numpy array |

### Step 3 — Identify Boundary Leaks (Workarounds)

Four workarounds signaled the boundary was in the wrong place:

1. **Dual-write vector storage** — every embed writes to two tables (embeddings BLOB + vec_embeddings virtual table), with dim-mismatch detection and a three-layer fallback chain (vec0 → numpy → Jaccard). This is the cost of sqlite-vec not supporting native HNSW.

2. **O(N²) pairwise matrix in application memory** — the sleep cycle loads all embeddings into sklearn for cosine similarity, hitting ~160GB for 200K 1024-dim nodes. A native HNSW provider would do this incrementally.

3. **Recursive CTEs in Python strings** — graph traversal is hand-rolled via raw SQL recursive CTEs that a native graph DB would do with a single `MATCH` clause.

4. **Full-table load into numpy** — cross-link candidate discovery loads the entire embeddings table into memory instead of incremental approximate queries.

**Key lesson:** These are not bugs. They are signals telling you where the architectural boundary is currently leaking. Each one is a candidate to push behind the contract.

### Step 4 — Design the Contract

Produce a Python ABC (Abstract Base Class) with ~30 methods across 8 domains:

- `StorageProvider` base class with `initialize()`/`close()` lifecycle
- Node CRUD: `create_node`, `get_node`, `update_node`, `delete_node`, `scan_nodes`, `count_nodes`
- Edge CRUD (dumb graph — no edge types): `create_edge`, `get_neighbors`, `delete_incident_edges`, `redirect_edges`
- Vector KNN: `set_embedding`, `get_embedding`, `find_similar`, `find_similar_by_id`, `scan_embeddings`
- Graph traversal: `bfs`, `shortest_path`, `trace_derivation`
- Batch/maintenance: `find_cross_link_candidates`, `find_near_duplicates`, `random_sample`, `get_graph_metrics`
- Transactions and introspection

**Key decisions:**
- `find_cross_link_candidates` returns candidate pairs but does not specify *how* — sqlite-vec does O(N²) internally, HNSW providers query incrementally
- `find_near_duplicates` (Bron-Kerbosch algorithm) stays in application code as business logic, not storage
- `random_sample` is a provider operation — native sampling is far more efficient than loading everything to Python
- No edge type/label field anywhere — the dumb-graph constraint is enforced at the data-model level

### Step 5 — Two-Provider Proof Concept

Design two provider implementations to validate the contract:

- **Provider A (sqlite-vec):** wraps the existing dual-write, sklearn matrix, recursive CTEs, dim-mismatch detection into the contract. All the workarounds become *this provider's internal complexity*, not application core's.

- **Provider B (KuzuDB):** second implementation proving the contract is complete. Native HNSW means `set_embedding` is a column write, `find_similar` is `QUERY_VECTOR_INDEX`, `shortest_path` is `MATCH`. No fallback chains — the provider is simpler because the substrate handles it.

Both providers pass the same test suite, proving the abstraction is sound.

## Key Takeaways

1. **The philosophy is the constraint.** Read it before touching code — it tells you what the interface must respect.

2. **Workarounds are signals.** Dual-write, fallback chains, full-table loads, hand-rolled graph traversals in SQL — each one tells you where the boundary is leaking.

3. **Design against what the codebase *needs*, not what the current substrate *does*.** The interface represents the application's requirements, not the current provider's capabilities.

4. **The two-provider proof catches leaks.** If you can't implement a second provider against the same interface, the contract has holes.

5. **Expensive operations define the performance profile.** `find_similar`, `find_cross_link_candidates`, and `bfs` are the operations that determine whether the system is fast or slow. The contract must make them implementable efficiently on a native substrate.

# Architecture Health Assessment

Use this template for a repeatable, evidence-backed assessment of an existing system. Keep recommendations separate from reverse-engineered facts.

## Scope and confidence

- System/revision:
- Assessment date:
- Included surfaces:
- Excluded surfaces:
- Stakeholders and operators consulted:
- Evidence limitations:
- Confidence scale: high / medium / low

## Executive assessment

- Overall health statement:
- Strongest evidence-backed property:
- Highest-risk unresolved property:
- Immediate observation or containment:
- Boundary statement and neighboring skills:

## Evidence ledger

| ID | Claim | Class (observed/reported/inferred/unknown) | Evidence | Confidence | Follow-up |
|---|---|---|---|---|---|
| E-001 |  |  |  |  |  |

## Architecture characteristics

| Scenario | Characteristic | Current behavior | Evidence | Owner/boundary | Tension or risk | Next probe |
|---|---|---|---|---|---|---|
|  |  |  |  |  |  |  |

## Coupling and modularity

| Candidate boundary | Static | Dynamic | Data | Temporal | Deployment | Organizational | Change coupling | Verdict |
|---|---|---|---|---|---|---|---|---|
|  |  |  |  |  |  |  |  |  |

## Data ownership and workflows

- Entity/fact authority map:
- Shared invariants:
- Distributed transaction boundaries:
- Workflow shape: orchestration / choreography / hybrid / unknown
- Partial completion states:
- Failure detection and repair:
- Reconciliation key, precedence, and audit evidence:

## Decomposition readiness

- Pressure requiring a boundary change:
- Candidate capability and invariant owner:
- Independent deployment/scaling evidence:
- New coordination and operating cost:
- Alternatives considered, including retaining a modular monolith:
- Readiness verdict: ready for bounded experiment / needs seam work / retain current boundary / insufficient evidence
- Reversible probe and stop condition:

## Prioritized findings

| Priority | Finding | Evidence IDs | User/operator impact | Smallest safe next step | Owner |
|---|---|---|---|---|---|
|  |  |  |  |  |  |

## Clean-room and handoff checks

- [ ] No source code, copied implementation examples, private identifiers, or line-level derivative detail appears in the design output.
- [ ] Facts, inferences, reports, and unknowns are labeled.
- [ ] API/interface semantics are handed to `api-design-and-evolution`.
- [ ] Data-platform strategy is handed to `data-architect`.
- [ ] Implementation is handed to the relevant engineering skill.
- [ ] Approved migration execution is handed to `migration-engineering`.