opencode-qa
QA opencode itself, per case: verify the CLI/terminal (opencode run, db, serve, export), prove a specific plugin hook/action/event fired via the SSE event stream, smoke-test the TUI under tmux, and investigate sessions in opencode's SQLite DB by id, title/name, or message text. S
Install
npx skills add https://github.com/code-yeongyu/oh-my-openagent/tree/dev/.agents/skills/opencode-qa
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install code-yeongyu-oh-my-openagent@llmmart
git clone https://github.com/code-yeongyu/oh-my-openagent.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole code-yeongyu/oh-my-openagent collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
opencode QA
QA the opencode coding agent itself. This skill maps each QA need to a tested
helper script and a deep reference. Every script ships a --self-test that
asserts its scenario against the live machine, so the scripts are both the QA
tools and their own regression checks.
Verified against opencode v1.17.7 (bun 1.3.12, macOS). Confirm the installed
version with opencode --version; the surface is stable but always sanity
check a flag with opencode <cmd> --help.
Golden rules (read before running anything)
- READS of the live DB are safe and intended. Investigating sessions (Case D)
only reads
~/.local/share/opencode/opencode.db. - Anything that SPAWNS opencode (serve, run, the TUI) must use an isolated XDG
sandbox so QA never writes junk sessions into the real DB. The bundled
scripts already do this; if you run opencode by hand for QA, set
XDG_DATA_HOME/XDG_CONFIG_HOME/XDG_STATE_HOME/XDG_CACHE_HOMEto temp dirs first. - Global text search over the
parttable is a multi-GB scan. Always scope it (--session,--recent, or--since). The text script refuses an unbounded scan on purpose. - The opencode source repo (
packages/opencode) tests itself withbun testand CANNOT run tests from the repo root. Seereferences/testing-harness.md.
Setup
Scripts live next to this file under scripts/. Invoke them from this skill
directory (or with their absolute path):
cd <this-skill-dir> # .agents/skills/opencode-qa
bash scripts/lib/common.sh --self-check # confirm the harness + deps
Docker is the default QA surface. Run QA inside a disposable container that
has the latest opencode and a copy of your config, with the host untouched:
script/agent/qa-docker.sh (see references/docker-qa.md).
The local scripts below are the fallback for when Docker is unavailable or on
Windows.
common.sh provides the shared harness (DB path, SQL escaping, isolated XDG
sandbox, free port, server start/stop, and an EXIT-trap cleanup). It requires
opencode, sqlite3, curl, jq, and tmux on PATH.
Router: pick your case
| You want to... | Case | Script | Reference |
|---|---|---|---|
| Run opencode non-interactively / check a CLI command | A | opencode run --format json (inline) |
references/cli-commands.md |
| Find a session by its id | D | scripts/db-session-by-id.sh <ses_id> |
references/db-investigation.md |
| Find sessions by title/name | D | scripts/db-session-by-name.sh "<text>" |
references/db-investigation.md |
| Find sessions by message text | D | scripts/db-session-by-text.sh --recent N "<text>" |
references/db-investigation.md |
| Export a whole session as JSON | D | scripts/export-roundtrip.sh <ses_id> |
references/db-investigation.md |
| Check the HTTP server / an endpoint | B | scripts/server-smoke.sh |
references/server-api.md |
| Prove a hook / action / event fired | B | scripts/sse-hook-probe.sh |
references/events-hooks.md |
| Prove serve-topology wake runner-split (reproduced/fixed) | B | scripts/serve-wake-split-probe.sh --expect reproduced\|fixed --evidence-dir DIR (self-test: --self-test; fake LLM: scripts/lib/fake-openai-server.mjs) |
references/events-hooks.md |
| Smoke-test the TUI | C | scripts/tui-smoke.sh |
references/tui-tmux.md |
| Write/run a test in the opencode source | - | (bun test) | references/testing-harness.md |
| Drive opencode from a Bun/TS script | - | (SDK) | references/sdk.md |
Case A: CLI / terminal works
The canonical scriptable, non-interactive entry is opencode run. JSON mode
emits one event per line so you can assert on it.
# stream structured events (types: text, tool_use, step_start, step_finish, reasoning, error)
opencode run "list files in src" --format json
# run a slash command
opencode run --command commit
# resume the last session
opencode run -c "continue"
# target an already-running server instead of booting one
opencode run "explain auth" --attach http://127.0.0.1:4096 -p "$OPENCODE_SERVER_PASSWORD"
Other QA-useful commands: opencode db path, opencode debug paths,
opencode session list --format json, opencode models --verbose. Full flag
detail in references/cli-commands.md.
Case B: a specific hook, action, or event
opencode publishes lifecycle events over Server-Sent Events at GET /event.
Plugins observe the same events via the event hook, so seeing an event on the
wire proves a hook would fire.
# prove the SSE plumbing works (isolated server, asserts server.connected)
bash scripts/sse-hook-probe.sh --self-test
# watch a REAL server for a specific event while you trigger an action
bash scripts/sse-hook-probe.sh --attach http://127.0.0.1:4096 \
--password "$OPENCODE_SERVER_PASSWORD" --directory "$PWD" \
--event message.part.updated --timeout 30
Trigger an action over HTTP (fire-and-forget so the stream is not blocked):
curl -X POST -u opencode:$OPENCODE_SERVER_PASSWORD -H 'Content-Type: application/json' \
-d '{"parts":[{"type":"text","text":"say hi"}]}' \
"http://127.0.0.1:4096/session/<ses_id>/prompt_async?directory=$PWD"
A real prompt needs a configured provider, so run the watch-and-trigger pattern
against your real server, not the isolated sandbox. Event-type catalog, the 21
plugin hook points, and how to load a local plugin: references/events-hooks.md.
Server start, auth, and routes: references/server-api.md.
Case C: the TUI
bash scripts/tui-smoke.sh --self-test
This launches the TUI under tmux in an isolated sandbox, confirms it renders
(capture-pane), confirms send-keys reaches the composer, tears the tmux
session down, and verifies the real DB session count is unchanged.
When TUI visual QA evidence is needed for a PR, follow
docs/reference/web-terminal-visual-qa.md: render the TUI through the real
xterm.js web terminal and screenshot it - NEVER the tmux capture-pane frame,
which degrades color and CJK width. From the repository root:
node script/qa/web-terminal-visual-qa.mjs --title "OpenCode TUI QA" \
--command "opencode" --input "{Enter}" \
--evidence-dir .omo/evidence/<slug>/opencode-web-terminal
This runs a real pty, renders it in xterm.js under Chrome, and writes
terminal.txt, terminal-ansi.txt, terminal.png (the true-color artifact),
and metadata.json with a cleanup receipt (--from-file <capture.ansi> replays
a saved raw stream). The isolated scripts/tui-smoke.sh remains the canonical
OpenCode TUI boot smoke (tmux), separate from this visual evidence.
Honest verdict: tmux is fine for SMOKE (did it boot, render, accept a key) but
fragile for asserting conversation output (the TUI is a 60fps full-screen app).
For real behavior assertions use Case A (opencode run), Case B (server API +
SSE), or the TUI control HTTP API (POST /tui/append-prompt,
POST /tui/submit-prompt, POST /tui/execute-command). Details and the manual
tmux recipe: references/tui-tmux.md.
Case D: investigate sessions in the DB
Read-only against the live SQLite DB. The session table is small (title and
id lookups are instant); message text lives in the multi-GB part table, so
text search must be scoped.
# by id
bash scripts/db-session-by-id.sh ses_3a4ee6335ffedFB8f76BPU1Eb3
# by title / name (newest first; second arg = limit)
bash scripts/db-session-by-name.sh "auth refactor" 20
# by message text - scope with --session, --recent N, or --since "<window>"
bash scripts/db-session-by-text.sh --session ses_3a4e... "ULTRAWORK"
bash scripts/db-session-by-text.sh --recent 50 "permission denied"
bash scripts/db-session-by-text.sh --since "7 days" --limit 50 "TODO"
# export an entire session as clean JSON
bash scripts/export-roundtrip.sh ses_3a4e... > session.json
Ad hoc queries: opencode db "<SQL>" --format json. Schema, tested query
shapes with timings, the legacy message/part vs V2 session_message
distinction, and the 25 GB caveat: references/db-investigation.md.
Scripts index
Run any script with --self-test to verify it against the live machine, or
-h for usage. DB-read scripts are read-only; serve/sse/tui scripts use an
isolated sandbox and clean up on exit.
| Script | Case | Self-test asserts |
|---|---|---|
scripts/lib/common.sh --self-check |
- | deps present, DB path resolves, SQL escaping, free port, sandbox auto-removed |
scripts/db-session-by-id.sh |
D | id round-trips for a real session |
scripts/db-session-by-name.sh |
D | a derived title needle returns >=1 row |
scripts/db-session-by-text.sh |
D | scoped search hits; unbounded scan refused; bounded search <30s |
scripts/export-roundtrip.sh |
D | export stdout is valid JSON and .info.id round-trips |
scripts/server-smoke.sh |
B | /global/health healthy, /doc >=100 paths, no-auth -> 401 |
scripts/sse-hook-probe.sh |
B | /event opens and delivers server.connected |
scripts/tui-smoke.sh |
C | TUI renders under tmux, tears down, real DB untouched |
Risks and caveats
- 25 GB part table: never run an unbounded text scan. Use
--session,--recent, or--since. A naiveJOIN ... WHERE session.time_created >= Xscans oldest-first and can take ~50s; the scripts use anIN-subquery on the newest sessions (~20ms). opencode exportwrites its banner to STDERR; pipe with2>/dev/nullbeforejqor you will get a parse error.- The server enforces auth only when
OPENCODE_SERVER_PASSWORDis set; otherwise it runs unsecured. Authenticated calls use-u opencode:$PASS. Unauthenticated calls to a secured server return HTTP 401. - Installed binary vs dev source: cite dev source paths for internals but
verify flags against the installed
opencode <cmd> --help. - Isolation: any QA that spawns opencode must use an isolated XDG sandbox so it
never pollutes the real DB. Prove it by comparing
sqlite3 "$(opencode db path)" "SELECT count(*) FROM session"before and after. - TUI output assertions are fragile; use the API for real assertions.
References
references/cli-commands.md- every QA-relevant opencode subcommand and flagreferences/db-investigation.md- DB schema, tested queries, the 25 GB caveatreferences/server-api.md- server start, auth, route catalog, /docreferences/events-hooks.md- SSE endpoints, event types, plugin hooksreferences/tui-tmux.md- tmux recipe, isolation, TUI control APIreferences/testing-harness.md- how opencode tests itself (bun test)references/sdk.md- the @opencode-ai/sdk client (reference only)references/docker-qa.md- run QA in a disposable Docker container (default; local is the fallback)
Files (oh-my-openagent)
-
references
-
cli-commands.md 3.8 KB
# opencode CLI for QA (Case A: terminal works) The installed binary is `opencode` (v1.15.13). From the source repo you can also run `bun run --conditions=browser ./src/index.ts <cmd>` inside `packages/opencode`. The canonical non-interactive QA entry is `opencode run --format json`. ## Global flags ``` --print-logs --log-level DEBUG|INFO|WARN|ERROR --pure (run without external plugins) -h, --help -v, --version ``` ## `opencode run` (non-interactive QA core) Positional message. Key flags: ``` -m, --model <provider/model> --agent <name> -s, --session <ses_...> -c, --continue --fork --format default|json -f, --file <path> --title --attach <url> -p, --password -u, --username --dir --variant --thinking -i, --interactive --dangerously-skip-permissions --command <slash-cmd> ``` `--format json` emits NDJSON, one JSON object per line, each shaped like: ```json {"type":"...", "timestamp":<ms>, "sessionID":"ses_...", ...} ``` `type` is one of: `text`, `tool_use`, `step_start`, `step_finish`, `reasoning`, `error`. The process exits when the session goes idle. Validation rules: - `-i` cannot combine with `--command` or `--format json` - `--fork` needs `-c` or `-s` Examples: ```bash opencode run "list the files in src" --format json ``` ```bash opencode run --command commit ``` ```bash opencode run -c "continue the previous task" ``` ```bash opencode run "explain auth" --attach http://127.0.0.1:4096 -p "$OPENCODE_SERVER_PASSWORD" ``` ## `opencode db` (database tools) ```bash opencode db path ``` Prints the active DB file path. ```bash opencode db "<SQL>" --format json ``` Runs a query and prints JSON rows. Use `--format tsv` (default) for TSV output. ```bash opencode db ``` Opens an interactive sqlite3 shell. ```bash opencode db migrate ``` Migrates legacy JSON storage into SQLite. Bundled scripts for session investigation: - `scripts/db-session-by-id.sh` - `scripts/db-session-by-name.sh` - `scripts/db-session-by-text.sh` - `scripts/export-roundtrip.sh` Full detail in `references/db-investigation.md`. ## `opencode session` ```bash opencode session list --format json ``` Lists sessions as JSON. ```bash opencode session delete <ses_id> ``` Deletes one session. ## `opencode export [sessionID]` Prints "Exporting session: ..." to STDERR and a clean JSON document `{info:{...}, messages:[...]}` to STDOUT. Always redirect STDERR before piping to jq. Example: ```bash opencode export ses_3a4e... 2>/dev/null | jq '.info.id' ``` Bundled wrapper: ```bash scripts/export-roundtrip.sh <ses_id> ``` ## `opencode serve` Starts a headless HTTP server. Flags: ``` --port (0 = pick 4096 then a free port) --hostname (default 127.0.0.1) --mdns --mdns-domain --cors ``` On start it prints: ``` opencode server listening on http://<host>:<port> ``` Set `OPENCODE_SERVER_PASSWORD` to require auth. See `references/server-api.md`. Bundled smoke test: ```bash scripts/server-smoke.sh ``` ## `opencode debug` Useful subcommands: ```bash opencode debug paths # data/config/cache/state dirs opencode debug info # version, OS, terminal, plugins ``` Others: `config`, `lsp`, `ripgrep`, `file`, `skill`, `snapshot`, `agent`, `v2`, `wait`. ## Other commands ```bash opencode models [provider] --verbose opencode stats opencode providers list # alias: auth opencode mcp list opencode generate # prints the OpenAPI JSON spec ``` ## Installed binary vs dev source (IMPORTANT note box) The installed `opencode` (v1.15.13) matches the dev source in `packages/opencode`. When citing internals, cite dev source paths but always verify a flag against `opencode <cmd> --help` on the installed binary, since the dev branch can drift ahead. For DB internals see `references/db-investigation.md`; for the HTTP server see `references/server-api.md`. -
db-investigation.md 5.8 KB
# Investigating opencode sessions in the DB (Case D) ## Table of Contents - [Where the data lives](#where-the-data-lives) - [Access methods](#access-methods) - [Schema (the tables that matter)](#schema-the-tables-that-matter) - [Time conversion](#time-conversion) - [Tested query patterns](#tested-query-patterns) - [The 25 GB caveat](#the-25-gb-caveat) - [Verifying read-only](#verifying-read-only) ## Where the data lives Active DB path: `opencode db path` (on this machine `~/.local/share/opencode/opencode.db`). Derived from XDG data dir + "opencode" + "opencode.db" (or "opencode-<channel>.db" on non-stable channels). Override via env `OPENCODE_DB` (`:memory:`, absolute, or relative-to-data). It is large (tens of GB) because the `part` table stores tool output. The `session` table is small (~21k rows; full scans are milliseconds). ## Access methods Preferred: `opencode db "<SQL>" --format json` (WAL-safe; resolves the active DB). `--format tsv` default. Bare `opencode db` opens an interactive sqlite3 shell. `opencode db path` prints the file. Raw fallback for EXPLAIN/perf: `sqlite3 "$(opencode db path)" "<SQL>"`. Reads are safe alongside a running opencode (WAL allows concurrent readers). ## Schema (the tables that matter) Note the ACTIVE storage in v1.15.13 is the LEGACY pair `message` + `part`; the V2 `session_message` table exists but is EMPTY in this version (a recent session showed 43 message rows, 169 part rows, 0 session_message). Document both but make clear `message`/`part` is what holds current data. ### `session` | Column | Notes | |--------|-------| | id | PK, 'ses_' prefix | | project_id | FK | | parent_id | | | slug | | | directory | | | title | NOT NULL | | version | | | agent | | | model | JSON {providerID, modelID} | | cost | | | tokens_input | | | tokens_output | | | tokens_reasoning | | | tokens_cache_read | | | tokens_cache_write | | | metadata | JSON | | time_created | epoch MILLISECONDS | | time_updated | epoch MILLISECONDS | | time_archived | | Indexes: project_id, parent_id, workspace_id. NO index on title or time_created. ### `message` (legacy) | Column | Notes | |--------|-------| | id | 'msg_' prefix | | session_id | FK -> session, cascade | | time_created | | | time_updated | | | data | JSON: {role, time:{created}, summary:{title}, agent, model:{providerID,modelID}, variant} | ### `part` (legacy) | Column | Notes | |--------|-------| | id | 'prt_' prefix | | message_id | FK -> message, cascade | | session_id | denormalized; index part_session_idx | | data | JSON | Part types seen: text, reasoning, tool, step-start, step-finish. A text part is `{"type":"text","text":"..."}`. ### Other tables `session_message` (V2, currently empty), `todo`, `project`, `permission`, `session_share`, `workspace`, `event`. ## Time conversion `time_created`/`time_updated` are epoch milliseconds. Convert: ```sql datetime(time_created/1000,'unixepoch') ``` ## Tested query patterns ### 1. By id (instant) Script: `scripts/db-session-by-id.sh <ses_id>` ```sql SELECT id, slug, title, directory, agent, json_extract(model,'$.modelID') AS model, json_extract(model,'$.providerID') AS provider, cost, tokens_input, tokens_output, datetime(time_created/1000,'unixepoch') AS created, datetime(time_updated/1000,'unixepoch') AS updated FROM session WHERE id='<ses_id>' ``` ### 2. By name/title (0.006s over 21k rows) Script: `scripts/db-session-by-name.sh "<substr>" [limit]` ```sql SELECT id, title, datetime(time_created/1000,'unixepoch') AS created FROM session WHERE title LIKE '%<substr>%' ORDER BY time_created DESC LIMIT <N> ``` ### 3. By message text Script: `scripts/db-session-by-text.sh (--session <id>|--recent <N>|--since "<window>") [--limit N] "<text>"` CRITICAL performance note: text lives in `part.data` JSON, and `part` is the multi-GB table, so an UNBOUNDED text scan is refused by the script. Always scope it. #### Scoped within one session (indexed, ~0.017s) ```sql SELECT p.session_id, p.id, substr(json_extract(p.data,'$.text'),1,120) FROM part p WHERE p.session_id='<id>' AND json_extract(p.data,'$.type')='text' AND json_extract(p.data,'$.text') LIKE '%<text>%' LIMIT 50 ``` #### Bounded to the N most-recent sessions (worst-case ~0.02s) ```sql SELECT p.session_id, p.id, substr(json_extract(p.data,'$.text'),1,120) FROM part p WHERE p.session_id IN ( SELECT id FROM session ORDER BY time_created DESC LIMIT <N> ) AND json_extract(p.data,'$.type')='text' AND json_extract(p.data,'$.text') LIKE '%<text>%' LIMIT 50 ``` #### AVOID this naive form (took ~50s) A JOIN `FROM session s JOIN part p ON p.session_id=s.id WHERE s.time_created >= X ...` scans oldest sessions first. The IN-subquery (newest-first, drives `part_session_idx`) is the right shape because it lets SQLite use the index on `part.session_id` with a small, ordered set of recent session IDs, rather than scanning the entire `part` table from the oldest sessions upward. ### 4. Full export Script: `scripts/export-roundtrip.sh <ses_id>` wraps `opencode export <id> 2>/dev/null` -> clean JSON `{info:{id,slug,projectID,directory,title,tokens,time,...}, messages:[...]}` (banner goes to stderr). ### 5. Listing recent sessions ```sql SELECT id, title, datetime(time_created/1000,'unixepoch') created FROM session ORDER BY time_created DESC LIMIT 100 ``` ## The 25 GB caveat Global text search over all parts is a full scan of the largest table and can take a long time. The bundled script refuses it; you must pass `--session`, `--recent`, or `--since`. Title search (`session` table) is always cheap. ## Verifying read-only All Case D operations are reads. To prove a QA pass did not mutate the DB, compare before and after: ```bash sqlite3 "$(opencode db path)" "SELECT count(*) FROM session" ``` These queries are exactly what the `scripts/db-*.sh` helpers run; each ships a `--self-test`. -
docker-qa.md 3.2 KB
# Docker QA (default path) Run opencode QA inside a DISPOSABLE container so the host is never touched and you always test against the latest opencode. The container itself is the sandbox: latest released opencode + codex are baked in, a COPY of your local config is loaded, and the container is removed on exit (`docker run --rm`). This is the DEFAULT; fall back to running the scripts locally (see SKILL.md) only when Docker is unavailable or on Windows. ## Use it `qa-docker.sh` brings up a disposable box (builds `omo-dev` then `omo-qa` on first use, reused after) and either drops you into it or serves opencode to your host. From the repo root: ```bash # a shell inside the box: just type `opencode ...` or `codex ...` script/agent/qa-docker.sh script/agent/qa-docker.sh shell # serve opencode's HTTP API to the host, then drive it from OUTSIDE: script/agent/qa-docker.sh serve 4096 # terminal 1 (Ctrl-C stops + removes) curl http://127.0.0.1:4096/global/health # host -> {"healthy":true,"version":"1.17.7"} opencode run "hi" --attach http://127.0.0.1:4096 # a real turn against the box # one-off command, or a skill's own self-test, inside: script/agent/qa-docker.sh exec opencode --version script/agent/qa-docker.sh exec bash .agents/skills/opencode-qa/scripts/server-smoke.sh --self-test script/agent/qa-docker.sh --no-config exec opencode --version # skip the config copy script/agent/qa-docker.sh --clean # remove the QA images ``` `omo-qa` is `omo-dev` (`.devcontainer/Dockerfile`) plus the latest `opencode-ai` and `@openai/codex` npm packages and `sqlite3 jq curl rsync`. Pin with `--build-arg OMO_OPENCODE_VERSION=...` on the qa.Dockerfile for a specific release. ## Why the container is the sandbox The local scripts isolate by pointing `XDG_*` at temp dirs so they never pollute the real `~/.local/share/opencode/opencode.db`. In Docker the whole container is throwaway, so isolation is structural: your host DB and config are never written. `qa-docker.sh` mounts `~/.config/opencode` (and `~/.codex`) READ-ONLY at `/mnt/host/*`; the entrypoint copies them into the container's writable home (heavy caches excluded) so QA runs against a COPY. ## Credentials Secrets are never baked into the image. Provide them at run time only: - a gitignored `.env` or `.env.local` at the repo root (auto-sourced by `script/agent/setup.sh` and `script/agent/qa-sandbox.sh`), - GitHub Codespaces secrets, or - the devcontainer `remoteEnv` passthrough. The host config is mounted read-only, so auth that already lives in `~/.config/opencode` rides along without copying secrets into any image layer. Fish caveat: if your shell sets `OPENCODE_CONFIG_DIR` (for example a `profiles/today` override), export it before calling `qa-docker.sh` so the container resolves the same profile (the runner forwards it with `-e`). ## Fallback: local / Windows `qa-docker.sh` exits 3 with guidance when Docker is unavailable or on Windows. There, run the scripts directly on the host (the rest of this skill); they isolate via temp `XDG_*`. Windows has no Docker QA path here by design. ## Cleanup Each run auto-removes its container (`--rm`). The `omo-dev` / `omo-qa` images persist for fast re-runs; drop them with `script/agent/qa-docker.sh --clean`. -
events-hooks.md 4.5 KB
# QAing opencode hooks, actions, and events (Case B) opencode publishes lifecycle events over Server-Sent Events. Plugins observe the SAME events via the `event` hook, so confirming an event on the wire proves a hook would fire. The bundled probe is `scripts/sse-hook-probe.sh`. ## Table of Contents - [The two SSE endpoints](#the-two-sse-endpoints) - [Watch the stream](#watch-the-stream) - [Important event types](#important-event-types) - [Hook-fired recipe (watch + trigger + assert)](#hook-fired-recipe-watch--trigger--assert) - [Plugin hooks (the 21 hook points a plugin can implement)](#plugin-hooks-the-21-hook-points-a-plugin-can-implement) - [Loading a local plugin for QA](#loading-a-local-plugin-for-qa) ## The two SSE endpoints - GET /event?directory=<dir> - per-instance stream; the FIRST event is `server.connected`, a `server.heartbeat` arrives every 10s, and the stream ends on `server.instance.disposed`. - GET /global/event - all events, no instance filter. - Frames look like `data: {"type":"...","properties":{...}}` (one per line). Consume with `curl -N`. ## Watch the stream ``` curl -N -u opencode:$PASS "http://127.0.0.1:4096/event?directory=$PWD" ``` Bundled, with assertions + auto-teardown: ``` scripts/sse-hook-probe.sh --self-test ``` (spawns an isolated server, asserts server.connected) ``` scripts/sse-hook-probe.sh --attach http://127.0.0.1:4096 --password "$PASS" --directory "$PWD" --event message.part.updated --timeout 30 ``` (watch your real server for a specific event) ## Important event types (type - properties) - `session.created` / `session.updated` / `session.deleted` (sessionID, info) - `message.updated` (sessionID, info) - `message.removed` (sessionID, messageID) - `message.part.updated` (sessionID, part, time) - `message.part.delta` (sessionID, messageID, partID, field, delta) - `message.part.removed` - `permission.asked` (id, sessionID, permission, tool?) - `permission.replied` - `session.error` (sessionID?, error) - `session.diff` (sessionID, diff) - `question.asked` / `question.replied` / `question.rejected` - `file.watcher.updated` (file, event) - `project.updated` - `lsp.updated` - `pty.created` / `pty.updated` / `pty.exited` / `pty.deleted` - `server.connected` - `server.heartbeat` - `server.instance.disposed` - `global.disposed` - `plugin.added` ## Hook-fired recipe (watch + trigger + assert) Two-shell pattern (or use the script): ``` # shell 1: watch (kill with Ctrl-C when done) curl -N -u opencode:$PASS "http://127.0.0.1:4096/event?directory=$PWD" \ | grep --line-buffered '"type":"message.part.updated"' # shell 2: trigger an action (fire-and-forget) curl -X POST -u opencode:$PASS -H 'Content-Type: application/json' \ -d '{"parts":[{"type":"text","text":"say hi"}]}' \ "http://127.0.0.1:4096/session/<ses_id>/prompt_async?directory=$PWD" ``` A `message.part.updated` (text/tool) confirms the prompt action drove the model and any tool/permission hook path. Note: a real prompt requires a configured provider/auth, so this runs against your real server, not the isolated sandbox (the sandbox only proves the SSE plumbing via server.connected). ## Plugin hooks (the 21 hook points a plugin can implement) `event`, `config`, `tool`, `auth`, `provider`, `chat.message`, `chat.params`, `chat.headers`, `permission.ask`, `command.execute.before`, `tool.execute.before`, `tool.execute.after`, `tool.definition`, `shell.env`, `experimental.chat.messages.transform`, `experimental.chat.system.transform`, `experimental.session.compacting`, `experimental.compaction.autocontinue`, `experimental.text.complete`. - A plugin is a module default-exporting `{ id?, server: (input, options) => Promise<Hooks> }`. - Minimal example implementing `event` and `tool.execute.before` that console.log the activity: ```typescript export default { id: "qa-logger", async server(input, options) { return { event: async (event) => { console.log("[event]", event.type, event.properties); }, "tool.execute.before": async (tool, args, context) => { console.log("[tool.before]", tool.name, args); }, }; }, }; ``` ## Loading a local plugin for QA - Add an absolute path or npm spec to the opencode config `plugin` / `plugin_origins` array (project `.opencode/` config or user config), then restart opencode. On load it emits `plugin.added`. - To QA a hook: load the plugin, watch /event (or the plugin's own logging), trigger the relevant action, and assert. --- Pair this with references/server-api.md (how to start the server, auth, prompt routes). -
sdk.md 4.1 KB
# opencode SDK (@opencode-ai/sdk) - reference only A TypeScript/Bun way to drive opencode for QA. Prefer the tested CLI/curl scripts for portability; reach for the SDK when you want typed access from a Bun script. > IMPORTANT: method signatures differ between SDK versions and between the published docs and the generated client. ALWAYS check the installed version's types (node_modules/@opencode-ai/sdk) before relying on a signature, and verify against `GET /doc` (the OpenAPI spec the SDK is generated from). ## Entry points and exports Package `@opencode-ai/sdk` subpath exports: - `.` (src/index.ts) - `./client` - `./server` - `./v2` (src/v2/index.ts) - `./v2/client` - `./v2/server` - `./v2/gen/client` Root and v2 entries export `createOpencode()`, `createOpencodeClient(...)`, `createOpencodeServer(...)`. `createOpencodeServer()` spawns `opencode serve ...` and waits for the startup line. `createOpencodeClient({ baseUrl })` wraps the generated client, rewrites directory/workspace headers, installs error interception. ## Two ways to connect ```ts import { createOpencodeClient, createOpencodeServer } from "@opencode-ai/sdk/v2" // A) embedded server (spawns opencode serve) const server = await createOpencodeServer() const client = createOpencodeClient({ baseUrl: server.url }) // ... use client ... server.close() // B) connect to an already-running server const client2 = createOpencodeClient({ baseUrl: "http://127.0.0.1:4096" }) ``` ## Client namespaces Top-level on OpencodeClient: auth, app, global, event, config, experimental, tool, worktree, find, file, instance, path, vcs, command, lsp, formatter, mcp, project, pty, question, permission, provider, session, part, sync, v2, tui. ## Useful methods (shapes vary by version) - `client.global.health()`, `client.global.event()` - `client.app.log(...)`, `client.app.agents(...)`, `client.app.skills(...)` - `client.config.get()`, `client.config.providers()` - `client.event.subscribe()` - SSE on /event; iterate `for await (const event of events.stream) { event.type, event.properties }` - `client.session` (legacy surface): list, create, status, get, update, delete, children, todo, diff, messages, message, deleteMessage, prompt, promptAsync, command, shell, fork, abort, init, share, unshare, summarize, revert, unrevert - `client.v2.session` (newer read/stream surface): list, prompt, compact, wait, context, messages - `client.part.delete(...)`, `client.part.update(...)` ## Minimal QA snippet Arg shapes may differ by version. Treat this as a starting point, not a contract. ```ts import { createOpencodeClient, createOpencodeServer } from "@opencode-ai/sdk/v2" const server = await createOpencodeServer() const client = createOpencodeClient({ baseUrl: server.url }) try { const session = await client.session.create({ title: "QA session" }) await client.session.promptAsync({ sessionID: session.id, parts: [{ type: "text", text: "Say hello in one line." }], }) const sessions = await client.session.list({ limit: 10 }) console.log(sessions[0]?.title) const messages = await client.v2.session.messages({ sessionID: session.id, limit: 20, }) console.log(messages.items.length) } finally { server.close() } ``` ## Key types - Legacy Session: id, slug, projectID, directory, title, version, time.created/updated, optional workspaceID, path, parentID, summary, cost, tokens, share, agent, model, metadata, permission, revert. - Message = UserMessage | AssistantMessage (role "user" | "assistant"; assistant adds time.completed?, modelID, providerID, agent, tokens, finish?, error?). - Part union: TextPart, ReasoningPart, FilePart, ToolPart, StepStartPart, StepFinishPart, SnapshotPart, PatchPart, AgentPart, RetryPart, CompactionPart, SubtaskPart. ## How it is generated `packages/sdk/js/script/build.ts` runs `bun dev generate > openapi.json` from the opencode repo, feeds it to `@hey-api/openapi-ts.createClient`, writes output to `packages/sdk/js/src/v2/gen`, patches an SSE generic, prettifies and typechecks. Regenerate with `./packages/sdk/js/script/build.ts`. --- For version-stable QA, prefer the curl/CLI scripts in this skill; cross-check any SDK call against `GET /doc`. -
server-api.md 6.5 KB
# opencode HTTP server API for QA (Case B) ## Table of Contents - [Start a server](#start-a-server) - [Authentication](#authentication) - [Per-request workspace routing](#per-request-workspace-routing) - [Introspect the API](#introspect-the-api) - [Tested smoke calls](#tested-smoke-calls) - [Route catalog](#route-catalog) - [Triggering a prompt over HTTP](#triggering-a-prompt-over-http) ## Start a server Run the server with a fixed port and host: ```bash opencode serve --port 4096 --hostname 127.0.0.1 ``` Output: ``` opencode server listening on http://127.0.0.1:4096 ``` Port 0 means the server will pick 4096, then fall back to a free port if that one is taken. A bundled isolated smoke test is available at `scripts/server-smoke.sh`. It spawns an isolated server, checks `/global/health`, checks that `/doc` returns at least 100 paths, and confirms that no-auth requests get 401, then tears the server down. ## Authentication Set the environment variable `OPENCODE_SERVER_PASSWORD` to require authentication. If it is unset, the server runs UNSECURED and prints a warning. The username defaults to `opencode`. Override it with `OPENCODE_SERVER_USERNAME`. Two ways to authenticate: 1. HTTP Basic Auth: `-u opencode:$PASS` 2. Query parameter: `?auth_token=<base64(user:pass)>` Unauthenticated requests to protected routes return HTTP 401. This was verified. ## Per-request workspace routing Most instance routes need the target project directory. Pass it as either: - Query parameter: `?directory=$PWD` - Header: `x-opencode-directory: $PWD` Aliases also work: `x-opencode-workspace` header or `?workspace=` query parameter. The server resolves an instance per request, so a single `serve` process can handle many projects. ## Introspect the API The `/doc` endpoint returns the full OpenAPI spec. To list all documented paths: ```bash curl -s -u opencode:$PASS http://127.0.0.1:4096/doc | jq '.paths | keys' ``` On v1.15.13 this returned 113 paths. This is the source of truth for exact request and response schemas. ## Tested smoke calls ```bash curl -s -u opencode:$PASS http://127.0.0.1:4096/global/health | jq . # {"healthy":true,"version":"1.15.13"} curl -s -u opencode:$PASS http://127.0.0.1:4096/doc | jq '.paths|length' # 113 curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:4096/session?directory=$PWD # 401 (no credentials) curl -s -u opencode:$PASS "http://127.0.0.1:4096/session?directory=$PWD" | jq 'length' ``` ## Route catalog This mirrors the structure returned by `/doc`. Each entry is grouped as `method path - purpose`. ### Global - `GET /global/health` - health check - `GET /global/event` - server-wide SSE event stream - `GET /global/config` - read global configuration - `PATCH /global/config` - update global configuration - `POST /global/dispose` - dispose the server instance - `GET /doc` - OpenAPI specification ### Session - `GET /session` - list sessions - `GET /session/status` - session status overview - `GET /session/:id` - get session by ID - `GET /session/:id/children` - list child sessions - `GET /session/:id/todo` - get session todo items - `GET /session/:id/diff` - get session diff - `GET /session/:id/message` - list session messages - `GET /session/:id/message/:messageID` - get a specific message - `POST /session` - create a new session - `DELETE /session/:id` - delete a session - `PATCH /session/:id` - update a session - `POST /session/:id/fork` - fork a session - `POST /session/:id/abort` - abort a session - `POST /session/:id/init` - initialize a session - `POST /session/:id/share` - share a session - `POST /session/:id/summarize` - summarize a session - `POST /session/:id/revert` - revert a session - `POST /session/:id/unrevert` - unrevert a session - `DELETE /session/:id/share` - unshare a session ### Prompting - `POST /session/:id/message` - send a prompt; streams JSON - `POST /session/:id/prompt_async` - fire-and-forget prompt; returns 204 - `POST /session/:id/command` - execute a command in a session - `POST /session/:id/shell` - run a shell command in a session ### Files and find - `GET /find` - text search via ripgrep - `GET /find/file` - file search - `GET /find/symbol` - symbol search - `GET /file` - file metadata - `GET /file/content` - file contents - `GET /file/status` - file status ### Instance and app - `GET /path` - path resolution - `GET /vcs` - version control info - `GET /vcs/status` - VCS status - `GET /vcs/diff` - VCS diff - `GET /command` - available commands - `GET /agent` - available agents - `GET /skill` - available skills - `GET /lsp` - LSP info - `GET /formatter` - formatter info ### Permission and question - `GET /permission` - list pending permission requests - `POST /permission/:requestID/reply` - reply to a permission request - `GET /question` - list pending questions - `POST /question/:requestID/reply` - reply to a question - `POST /question/:requestID/reject` - reject a question ### TUI control These endpoints drive a running TUI over HTTP. - `POST /tui/append-prompt` - append text to the TUI prompt - `POST /tui/submit-prompt` - submit the current TUI prompt - `POST /tui/execute-command` - execute a TUI command - `POST /tui/show-toast` - show a toast in the TUI - `GET /tui/control/next` - get the next TUI control event - `POST /tui/control/response` - respond to a TUI control event ### PTY - `GET /pty` - list PTY sessions - `POST /pty` - create a PTY session - `GET /pty/:id` - get PTY session info - `DELETE /pty/:id` - delete a PTY session - `POST /pty/:id/connect-token` - generate a PTY connect token - `GET /pty/:id/connect` - WebSocket connection to the PTY ### Event - `GET /event` - instance-level SSE event stream ### V2 API - `GET /api/session` - list sessions (v2) - `POST /api/session/:id/prompt` - prompt a session (v2) - `POST /api/session/:id/compact` - compact a session (v2) - `POST /api/session/:id/wait` - wait for a session (v2) - `GET /api/session/:id/context` - get session context (v2) - `GET /api/session/:id/message` - get session messages (v2) - `GET /api/model` - list models (v2) - `GET /api/provider` - list providers (v2) ## Triggering a prompt over HTTP (for hook and event QA) Use `prompt_async` so the event stream is not blocked. ```bash curl -X POST -u opencode:$PASS -H 'Content-Type: application/json' \ -d '{"parts":[{"type":"text","text":"hello"}]}' \ "http://127.0.0.1:4096/session/<ses_id>/prompt_async?directory=$PWD" ``` This returns HTTP 204. Watching events is covered in `references/events-hooks.md`. --- Schemas are authoritative in `GET /doc`; for the event stream see `references/events-hooks.md`. -
testing-harness.md 6.2 KB
# opencode Test Harness (how opencode QAs itself) This is reference material for writing and running tests against the opencode source. The skill's own QA scripts (CLI, curl, sqlite) do not require this, but it is the authoritative pattern when you need a unit or integration test. ## Table of Contents 1. [Runner and the root guard](#runner-and-the-root-guard) 2. [Test bootstrap (in-memory, isolated)](#test-bootstrap-in-memory-isolated) 3. [Effect-based harness (test/lib/effect.ts)](#effect-based-harness-testlibeffectts) 4. [Instance and tmpdir fixtures (test/fixture/fixture.ts)](#instance-and-tmpdir-fixtures-testfixturefixturets) 5. [CLI subprocess harness (test/lib/cli-process.ts)](#cli-subprocess-harness-testlibcli-processts) 6. [Fake LLM server (test/lib/llm-server.ts)](#fake-llm-server-testlibllm-serverts) 7. [Representative test shapes](#representative-test-shapes) 8. [App e2e (Playwright)](#app-e2e-playwright) 9. [Test style conventions](#test-style-conventions) ## Runner and the root guard The runner is `bun test` (Bun built-in, not vitest or jest). Tests cannot run from the repo root. Two guards enforce this: - `bunfig.toml` at repo root sets `root = "./do-not-run-tests-from-root"` - Root `package.json` has `"test": "echo 'do not run tests from root' && exit 1"` Run from a package directory instead: ```bash cd packages/opencode && bun test --timeout 30000 ``` Run a single file: ```bash bun test test/tool/read.test.ts ``` Filter by test name: ```bash bun test --grep "truncates large file" ``` CI variant: ```bash bun run test:ci ``` Turbo dependency: `opencode#test` depends on `^build`. ## Test bootstrap (in-memory, isolated) The preload file is `packages/opencode/test/preload.ts`. It is wired via `packages/opencode/bunfig.toml`: ```toml [test] preload = ["@opentui/solid/preload", "./test/preload.ts"] ``` What it does: - Sets `XDG_DATA_HOME`, `XDG_CACHE_HOME`, `XDG_CONFIG_HOME`, and `XDG_STATE_HOME` to temp directories - Sets `OPENCODE_TEST_HOME` - Sets `OPENCODE_DB=":memory:"` (SQLite in-memory) - Wipes all provider API keys from `process.env` - Sets `OPENCODE_EXPERIMENTAL_EVENT_SYSTEM=true` - Sets `OPENCODE_EXPERIMENTAL_WORKSPACES=true` - Initializes `Log.init({ print: false })` - Calls `initProjectors()` ## Effect-based harness (test/lib/effect.ts) The `it` factory wraps `bun:test` with three variants: - `it.effect(name, body)` ... TestClock + TestConsole (isolated time) - `it.live(name, body)` ... real clock + TestConsole - `it.instance(name, body, opts)` ... real clock + scoped tmpdir + a real Instance context `testEffect(layer)` builds an `it` bound to an Effect layer: ```typescript const it = testEffect(Layer.mergeAll(readLayer(), testInstanceStoreLayer)) ``` ## Instance and tmpdir fixtures (test/fixture/fixture.ts) - `tmpdirScoped(options?)` ... scoped temp directory. Optional `git: true`, optional `config` (writes `opencode.json`), optional `init`. - `provideInstance(directory)(effect)` ... runs an Effect inside a real instance for that directory. - `withTmpdirInstance({ git?, config?, init? })(effect)` ... one-liner: make tmpdir, optional git init + config, provide instance. - `testInstanceStoreLayer` ... instance store with a no-op bootstrap. ## CLI subprocess harness (test/lib/cli-process.ts) `cliIt.live(name, body, timeoutMs?)` and `cliIt.concurrent(...)` spawn the real CLI (`bun run --conditions=browser src/index.ts`) in an isolated environment. Exposed helpers: - `opencode.run()` - `opencode.serve()` - `opencode.acp()` - `expectExit` - `parseJsonEvents` Isolation environment keys: - `OPENCODE_TEST_HOME` - `OPENCODE_CONFIG_CONTENT` (inline provider config) - `OPENCODE_DISABLE_PROJECT_CONFIG=1` - `OPENCODE_PURE=1` - `OPENCODE_DISABLE_AUTOUPDATE=1` - `OPENCODE_DISABLE_AUTOCOMPACT=1` - `OPENCODE_DISABLE_MODELS_FETCH=1` Real example from `packages/opencode/test/cli/serve/serve-process.test.ts`: ```typescript cliIt.live("spawns serve and health responds", async ({ opencode, expectExit }) => { const server = await opencode.serve() expect(server.port).toBeGreaterThan(0) const res = await fetch(`${server.url}/global/health`) expect(res.status).toBe(200) }) ``` ## Fake LLM server (test/lib/llm-server.ts) `TestLLMServer` is an in-process OpenAI-compatible SSE server to mock model responses deterministically. Methods: - `llm.text("hello")` - `llm.tool("read", { filePath: "x" })` - `llm.pushMatch(matchFn, reply)` This is how tests avoid real provider calls. ## Representative test shapes ### 1. Tool test From `packages/opencode/test/tool/read.test.ts`: ```typescript const it = testEffect(Layer.mergeAll(readLayer(), testInstanceStoreLayer)) it.instance("truncates large file over maxReadFileSize", () => Effect.gen(function* () { const test = yield* TestInstance // ... exercise the read tool, assert truncation }) ) ``` ### 2. Session/event test From `packages/opencode/test/session/session.test.ts`: ```typescript test("session.created fires after session.create", async () => { const deferred = Deferred.unsafeMake<void>(FiberId.none) // ... listen for session.created event await session.create({}) // ... assert deferred resolves }) ``` ### 3. Plain unit test From `packages/opencode/test/cli/run/runtime.boot.test.ts`: ```typescript import { describe, expect, mock, spyOn, test } from "bun:test" test("boots runtime without errors", () => { // ... standard assertions, no Effect }) ``` ## App e2e (Playwright) The app lives in `packages/app` (SolidJS). Config is `packages/app/playwright.config.ts`. It starts the Vite dev server via `webServer`; the backend is expected at `localhost:4096`. Commands (run from `packages/app`): ```bash bunx playwright install chromium bun run test:e2e:local ``` Filter with grep: ```bash bun run test:e2e:local -- --grep "settings" ``` UI mode: ```bash bun run test:e2e:ui ``` App unit tests: ```bash bun run test:unit ``` This equals `bun test --preload ./happydom.ts ./src`. ## Test style conventions Per opencode AGENTS.md: - Avoid mocks where possible. Test the real implementation. Do not duplicate logic into tests. - Run `bun typecheck` from the package directory (uses tsgo). Never run bare `tsc`. For runtime or scriptable QA without writing tests, use the opencode-qa scripts (Cases A-D in SKILL.md). -
tui-tmux.md 4 KB
# QAing the opencode TUI under tmux (Case C) ## Verdict first (be honest) - tmux CAN launch the opencode TUI and `capture-pane` reads the rendered frame; `send-keys` delivers keystrokes to the composer. This is proven and good for SMOKE checks: did it boot, does it render, does it accept input. - The TUI is a 60fps full-screen app (built on @opentui/solid) with a custom renderer, animations, and a worker thread. Asserting on conversation OUTPUT by scraping the frame is FRAGILE and not recommended. - For real behavior assertions prefer: `opencode run` (Case A, references/cli-commands.md), the server API + SSE (Case B, references/server-api.md + events-hooks.md), or the TUI control HTTP API (below). The TUI talks to the same server, so API-level QA is equivalent to driving the screen. ## Safety: isolate so QA never pollutes the real DB Launching the real TUI would create sessions in the real ~/.local/share/opencode DB. Run it under an isolated XDG sandbox. The bundled `scripts/tui-smoke.sh` does exactly this and verifies the real session count is unchanged before/after. ## Smoke test (bundled) - `scripts/tui-smoke.sh --self-test` launches the TUI under tmux in an isolated sandbox, polls capture-pane for a render marker (version string / "Ask anything" / footer), sends a sentinel keystroke, then kills the tmux session and confirms the real DB is untouched. ## Browser-rendered TUI visual evidence For PR evidence or any TUI visual QA claim, render the live TUI through the real xterm.js web terminal - NEVER the `tmux capture-pane` frame, which degrades color and CJK width: ```bash node script/qa/web-terminal-visual-qa.mjs --title "OpenCode TUI QA" \ --command "opencode" --input "{Enter}" \ --evidence-dir .omo/evidence/<slug>/opencode-web-terminal ``` The helper runs a real pty, renders it in xterm.js under Chrome, captures `terminal.png` (true color), writes `terminal.txt` + `metadata.json`, and records the cleanup receipt (`--from-file <capture.ansi>` replays a saved raw stream). This is the required TUI visual evidence pattern when the review needs to see the screen, while `scripts/tui-smoke.sh` remains the isolation/smoke authority. ## Manual tmux recipe (fenced) - for ad hoc smoke ``` SESS=oqa_tui_demo DIR=$(mktemp -d) tmux new-session -d -s "$SESS" -x 200 -y 50 # isolate XDG so no real session is written tmux send-keys -t "$SESS" "XDG_DATA_HOME=$DIR/data XDG_CONFIG_HOME=$DIR/cfg XDG_STATE_HOME=$DIR/state XDG_CACHE_HOME=$DIR/cache OPENCODE_DISABLE_AUTOUPDATE=1 OPENCODE_DISABLE_MODELS_FETCH=1 opencode $DIR" Enter sleep 7 tmux capture-pane -t "$SESS" -p | sed -n '1,30p' # inspect the rendered frame tmux send-keys -t "$SESS" "hello" # type into the composer sleep 1 tmux capture-pane -t "$SESS" -p | sed -n '1,30p' tmux kill-session -t "$SESS" # teardown (kills the TUI) rm -rf "$DIR" ``` Explain: capture-pane -p prints the visible frame; send-keys injects input; kill-session tears down the process tree. Always teardown and remove the temp dir. ## The reliable alternative: TUI control HTTP API A running TUI is a client of the local server, so you can drive it over HTTP without scraping the screen: - POST /tui/append-prompt - append text to the composer - POST /tui/submit-prompt - submit the composer - POST /tui/execute-command - run a TUI command - POST /tui/show-toast - show a toast - GET /tui/control/next + POST /tui/control/response - the control channel Use these (with auth + ?directory=) to deterministically drive a TUI you launched, then assert via the event stream (references/events-hooks.md). ## Headless component testing (for source-level TUI tests) opencode unit-tests TUI components headlessly with @opentui/core/testing `createTestRenderer` (see packages/opencode/test/cli/tui/, e.g. app-lifecycle.test.ts). This is the route for asserting TUI component behavior in the source repo; see references/testing-harness.md. Bottom line: tmux for smoke, server/SSE or /tui/* control for assertions, createTestRenderer for source unit tests.
-
-
scripts
-
lib
-
common.sh 9.5 KB
#!/usr/bin/env bash # common.sh - shared helpers for opencode-qa scripts. # # Source it from a sibling script: # SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # . "$SCRIPT_DIR/lib/common.sh" # # SAFETY MODEL (read this): # - DB-read helpers (oqa_db_path / oqa_db_query) hit the LIVE opencode DB # READ-ONLY. That is safe and intended for session investigation. # - Anything that SPAWNS opencode (serve / run / tui) must run under an # ISOLATED XDG sandbox (oqa_mk_isolated_xdg) so QA never writes junk # sessions into the real ~/.local/share/opencode DB. # - oqa_cleanup runs on EXIT and tears down servers, tmux sessions, curl # watchers, and every temp dir created via the helpers. # No `set -e`: these scripts deliberately probe failure paths (401, refused). set -uo pipefail OQA_TMPDIRS=() OQA_TMUX_SESSIONS=() OQA_CURL_PIDS=() OQA_SERVER_PID="" oqa_log() { printf '%s\n' "$*" >&2; } oqa_pass() { printf 'PASS: %s\n' "$*"; } oqa_fail() { printf 'FAIL: %s\n' "$*" >&2; return 1; } # oqa_require <bin>... -> 0 if all present, else 1 (names the missing ones). oqa_require() { local missing=0 b for b in "$@"; do if ! command -v "$b" >/dev/null 2>&1; then oqa_log "missing dependency: $b" missing=1 fi done return "$missing" } # Absolute path of the active opencode DB (resolves channel / OPENCODE_DB). oqa_db_path() { opencode db path 2>/dev/null | head -1 } # Escape a value for safe embedding inside a single-quoted SQL literal. oqa_sql_escape() { printf '%s' "$1" | sed "s/'/''/g" } # Run a read-only SQL query against the active DB; emit JSON rows. # Usage: oqa_db_query "SELECT ... LIMIT 5" oqa_db_query() { opencode db "$1" --format json 2>/dev/null } # Preserve HOME-based opencode shims after HOME is sandboxed. Some installed # opencode wrappers resolve the real binary via "$HOME/.opencode/bin/opencode"; # after oqa_mk_isolated_xdg rewrites HOME, that path must still exist. oqa_preserve_home_opencode_bin() { local real_home="$1" sandbox_home="$2" [ -d "$real_home/.opencode/bin" ] || return 0 mkdir -p "$sandbox_home/.opencode" || return 1 ln -s "$real_home/.opencode/bin" "$sandbox_home/.opencode/bin" 2>/dev/null || return 1 } # Create an isolated XDG sandbox so a spawned opencode never touches the real # DB. Sets globals OQA_XDG_ROOT + OQA_PROJ and exports XDG_*; registers the # root for cleanup. # # IMPORTANT: call this DIRECTLY, never via $(...). Command substitution runs in # a subshell, which would discard the exports and the cleanup registration. # oqa_mk_isolated_xdg # good # root="$OQA_XDG_ROOT" # read the global afterwards oqa_mk_isolated_xdg() { local root real_home root="$(mktemp -d -t oqa-xdg.XXXXXX)" || return 1 real_home="$HOME" OQA_TMPDIRS+=("$root") mkdir -p "$root/data" "$root/config" "$root/cache" "$root/state" "$root/home" "$root/proj" oqa_preserve_home_opencode_bin "$real_home" "$root/home" || return 1 export OQA_XDG_ROOT="$root" export HOME="$root/home" export OPENCODE_TEST_HOME="$root/home" export XDG_DATA_HOME="$root/data" export XDG_CONFIG_HOME="$root/config" export XDG_CACHE_HOME="$root/cache" export XDG_STATE_HOME="$root/state" export OQA_PROJ="$root/proj" # keep the sandbox offline + fast export OPENCODE_DISABLE_AUTOUPDATE=1 export OPENCODE_DISABLE_MODELS_FETCH=1 } # Print a free TCP port on 127.0.0.1. oqa_free_port() { if command -v python3 >/dev/null 2>&1; then python3 -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1",0)); print(s.getsockname()[1]); s.close()' elif command -v bun >/dev/null 2>&1; then bun -e 'const s=Bun.listen({hostname:"127.0.0.1",port:0,socket:{data(){}}});console.log(s.port);s.stop()' else # last resort: a high random port (small race window) printf '%s' "$(( (RANDOM % 20000) + 40000 ))" fi } # Poll an HTTP url until it accepts a connection (any status) or times out. # Usage: oqa_wait_http <url> [user:pass] [timeout_s] oqa_wait_http() { local url="$1" auth="${2:-}" timeout="${3:-25}" deadline deadline=$(( $(date +%s) + timeout )) while [ "$(date +%s)" -lt "$deadline" ]; do if [ -n "$auth" ]; then curl -s -o /dev/null -u "$auth" "$url" && return 0 else curl -s -o /dev/null "$url" && return 0 fi sleep 0.2 done return 1 } # Start an isolated, password-protected headless server. # Sets globals OQA_SERVER_URL / OQA_SERVER_PASS / OQA_SERVER_PORT / OQA_SERVER_PID. # Returns 1 if it never becomes ready. # # IMPORTANT: call this DIRECTLY, never via $(...). The PID + exports must land # in the caller's shell so oqa_cleanup can kill the server on exit. # oqa_start_server || { oqa_log "no server"; exit 1; } # url="$OQA_SERVER_URL" oqa_start_server() { oqa_mk_isolated_xdg || return 1 local port pass port="$(oqa_free_port)" pass="oqa-${RANDOM}${RANDOM}" OPENCODE_SERVER_PASSWORD="$pass" opencode serve --port "$port" --hostname 127.0.0.1 \ >"$XDG_STATE_HOME/serve.log" 2>&1 & OQA_SERVER_PID=$! disown "$OQA_SERVER_PID" 2>/dev/null || true export OQA_SERVER_PORT="$port" export OQA_SERVER_PASS="$pass" export OQA_SERVER_URL="http://127.0.0.1:$port" if ! oqa_wait_http "$OQA_SERVER_URL/global/health" "opencode:$pass" 30; then oqa_log "server failed to start; log follows:" cat "$XDG_STATE_HOME/serve.log" >&2 2>/dev/null || true return 1 fi } # Teardown everything the helpers created. Safe to call multiple times. oqa_cleanup() { if [ -n "${OQA_SERVER_PID:-}" ]; then kill "$OQA_SERVER_PID" 2>/dev/null || true sleep 0.3 kill -0 "$OQA_SERVER_PID" 2>/dev/null && kill -9 "$OQA_SERVER_PID" 2>/dev/null || true OQA_SERVER_PID="" fi local s p d for s in "${OQA_TMUX_SESSIONS[@]:-}"; do [ -n "$s" ] && tmux kill-session -t "$s" 2>/dev/null || true done for p in "${OQA_CURL_PIDS[@]:-}"; do [ -n "$p" ] && kill "$p" 2>/dev/null || true [ -n "$p" ] && sleep 0.1 [ -n "$p" ] && kill -0 "$p" 2>/dev/null && kill -9 "$p" 2>/dev/null || true done for d in "${OQA_TMPDIRS[@]:-}"; do [ -n "$d" ] && rm -rf "$d" 2>/dev/null || true done OQA_TMPDIRS=() OQA_TMUX_SESSIONS=() OQA_CURL_PIDS=() } trap oqa_cleanup EXIT # ---- self-check ------------------------------------------------------------ # Run: bash scripts/lib/common.sh --self-check oqa__self_check() { local fails=0 if oqa_require opencode sqlite3 curl jq tmux; then oqa_pass "dependencies present (opencode sqlite3 curl jq tmux)" else oqa_log "FAIL: missing dependencies"; fails=$((fails+1)) fi local dbp; dbp="$(oqa_db_path)" if [ -n "$dbp" ] && [ -f "$dbp" ]; then oqa_pass "oqa_db_path -> $dbp" else oqa_log "FAIL: oqa_db_path returned '$dbp'"; fails=$((fails+1)) fi local esc; esc="$(oqa_sql_escape "a'b'c")" if [ "$esc" = "a''b''c" ]; then oqa_pass "oqa_sql_escape quotes single quotes" else oqa_log "FAIL: oqa_sql_escape -> '$esc'"; fails=$((fails+1)) fi local port; port="$(oqa_free_port)" if [ "$port" -gt 0 ] 2>/dev/null; then oqa_pass "oqa_free_port -> $port" else oqa_log "FAIL: oqa_free_port -> '$port'"; fails=$((fails+1)) fi # isolation + trap teardown: an inner shell creates a sandbox (calling the # helper DIRECTLY so the cleanup registration survives) and exits; the EXIT # trap must remove it. We pass the sandbox path out via a marker file. local marker isodir home test_home marker="$(mktemp -t oqa-marker.XXXXXX)" bash -c '. "'"${BASH_SOURCE[0]}"'"; oqa_mk_isolated_xdg; printf "%s\n%s\n%s\n" "$OQA_XDG_ROOT" "$HOME" "$OPENCODE_TEST_HOME" > "'"$marker"'"' isodir="$(sed -n '1p' "$marker" 2>/dev/null)" home="$(sed -n '2p' "$marker" 2>/dev/null)" test_home="$(sed -n '3p' "$marker" 2>/dev/null)" rm -f "$marker" if [ -n "$isodir" ] && [ ! -d "$isodir" ]; then oqa_pass "isolated XDG sandbox auto-removed on exit ($isodir)" else oqa_log "FAIL: sandbox not cleaned: '$isodir' (exists=$([ -d "$isodir" ] && echo yes || echo no))"; fails=$((fails+1)) fi if [ -n "$isodir" ] && [ "$home" = "$isodir/home" ] && [ "$test_home" = "$isodir/home" ]; then oqa_pass "isolated HOME points inside sandbox" else oqa_log "FAIL: sandbox HOME not isolated (HOME='$home' OPENCODE_TEST_HOME='$test_home' root='$isodir')"; fails=$((fails+1)) fi local shim_marker shim_result shim_marker="$(mktemp -t oqa-shim.XXXXXX)" bash -c ' set -u . "'"${BASH_SOURCE[0]}"'" fake_home="$(mktemp -d -t oqa-fake-home.XXXXXX)" mkdir -p "$fake_home/.local/bin" "$fake_home/.opencode/bin" printf "%s\n" "#!/usr/bin/env bash" "exec \"\$HOME/.opencode/bin/opencode\" \"\$@\"" > "$fake_home/.local/bin/opencode" printf "%s\n" "#!/usr/bin/env bash" "exec \"\$HOME/.opencode/bin/opencode-real\" \"\$@\"" > "$fake_home/.opencode/bin/opencode" printf "%s\n" "#!/usr/bin/env bash" "printf fake-opencode-ok" > "$fake_home/.opencode/bin/opencode-real" chmod +x "$fake_home/.local/bin/opencode" "$fake_home/.opencode/bin/opencode" "$fake_home/.opencode/bin/opencode-real" HOME="$fake_home" PATH="$fake_home/.local/bin:$PATH" oqa_mk_isolated_xdg opencode > "'"$shim_marker"'" oqa_cleanup rm -rf "$fake_home" ' shim_result="$(cat "$shim_marker" 2>/dev/null)" rm -f "$shim_marker" if [ "$shim_result" = "fake-opencode-ok" ]; then oqa_pass "isolated HOME preserves HOME-based opencode shim" else oqa_log "FAIL: HOME-based opencode shim returned '$shim_result'"; fails=$((fails+1)) fi if [ "$fails" -eq 0 ]; then oqa_pass "common.sh self-check" return 0 fi oqa_log "common.sh self-check had $fails failure(s)" return 1 } if [ "${1:-}" = "--self-check" ]; then oqa__self_check exit $? fi -
fake-openai-branches.mjs 1.2 KB · in bundle
-
fake-openai-events.mjs 2.3 KB · in bundle
-
fake-openai-server.mjs 3.6 KB · in bundle
-
-
db-session-by-id.sh 1.7 KB
#!/usr/bin/env bash # db-session-by-id.sh - investigate an opencode session by its id (ses_...). # Read-only against the LIVE opencode DB via `opencode db ... --format json`. # # Usage: # db-session-by-id.sh ses_3a4ee6335ffedFB8f76BPU1Eb3 # db-session-by-id.sh --self-test # # Output: a JSON array with one row (id, slug, title, directory, agent, model, # cost, token counts, human-readable created/updated times) or [] if not found. SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" . "$SCRIPT_DIR/lib/common.sh" oqa_session_by_id() { local id esc id="$1" esc="$(oqa_sql_escape "$id")" oqa_db_query "SELECT id, slug, title, directory, agent, json_extract(model,'\$.modelID') AS model, json_extract(model,'\$.providerID') AS provider, cost, tokens_input, tokens_output, datetime(time_created/1000,'unixepoch') AS created, datetime(time_updated/1000,'unixepoch') AS updated FROM session WHERE id='$esc'" } oqa_self_test() { oqa_require opencode jq || return 1 local id out got id="$(oqa_db_query "SELECT id FROM session ORDER BY time_created DESC LIMIT 1" | jq -r '.[0].id // empty')" if [ -z "$id" ]; then oqa_log "FAIL: no sessions in DB to test against"; return 1 fi out="$(oqa_session_by_id "$id")" got="$(printf '%s' "$out" | jq -r '.[0].id // empty')" if [ "$got" = "$id" ]; then oqa_pass "db-session-by-id round-trips id ($id)" return 0 fi oqa_log "FAIL: expected id '$id', got '$got'"; return 1 } case "${1:-}" in --self-test) oqa_self_test; exit $? ;; -h|--help|"") sed -n '2,12p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//' [ -z "${1:-}" ] && exit 2 || exit 0 ;; *) oqa_require opencode jq || exit 1 oqa_session_by_id "$1" ;; esac -
db-session-by-name.sh 1.9 KB
#!/usr/bin/env bash # db-session-by-name.sh - find opencode sessions whose TITLE matches a substring. # Read-only against the LIVE opencode DB. Title search is cheap (the session # table is small; ~21k rows scan in milliseconds), so no bounding is needed. # # Usage: # db-session-by-name.sh "auth refactor" # newest 20 matches # db-session-by-name.sh "auth refactor" 50 # newest 50 matches # db-session-by-name.sh --self-test # # Output: JSON array of {id, title, created, updated} ordered newest first. SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" . "$SCRIPT_DIR/lib/common.sh" oqa_session_by_name() { local needle limit esc needle="$1" limit="${2:-20}" case "$limit" in (*[!0-9]*|"") limit=20 ;; esac esc="$(oqa_sql_escape "$needle")" oqa_db_query "SELECT id, title, datetime(time_created/1000,'unixepoch') AS created, datetime(time_updated/1000,'unixepoch') AS updated FROM session WHERE title LIKE '%$esc%' ORDER BY time_created DESC LIMIT $limit" } oqa_self_test() { oqa_require opencode jq || return 1 # Derive a guaranteed-present needle: the first 5 chars of a real title. local needle out n needle="$(oqa_db_query "SELECT substr(title,1,5) AS t FROM session WHERE length(title)>=5 ORDER BY time_created DESC LIMIT 1" | jq -r '.[0].t // empty')" if [ -z "$needle" ]; then oqa_log "FAIL: could not derive a title needle"; return 1 fi out="$(oqa_session_by_name "$needle" 5)" n="$(printf '%s' "$out" | jq 'length')" if [ "${n:-0}" -ge 1 ]; then oqa_pass "db-session-by-name found $n row(s) for needle '$needle'" return 0 fi oqa_log "FAIL: expected >=1 row for needle '$needle', got '$n'"; return 1 } case "${1:-}" in --self-test) oqa_self_test; exit $? ;; -h|--help|"") sed -n '2,14p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//' [ -z "${1:-}" ] && exit 2 || exit 0 ;; *) oqa_require opencode jq || exit 1 oqa_session_by_name "$1" "${2:-20}" ;; esac -
db-session-by-text.sh 6.6 KB
#!/usr/bin/env bash # db-session-by-text.sh - find opencode message TEXT by content. # # Message text lives inside the `part` table as JSON blobs # (json_extract(data,'$.text') for text parts). The `part` table holds the bulk # of the DB (tens of GB of tool output), so an UNBOUNDED text scan is refused. # You MUST scope the search to a bounded, recent set of sessions: # --session ses_... one session (indexed, instant) # --recent N the N most-recent sessions (default 25) # --since "<window>" sessions created within a window (e.g. "7 days"), # capped at the 200 most-recent in that window # # All bounded modes use `part.session_id IN (SELECT id FROM session ORDER BY # time_created DESC LIMIT ...)`, which drives the part_session_idx on exactly # the newest sessions. (A naive JOIN with `WHERE session.time_created >= X` # scans oldest-first and can take ~50s; the IN-subquery form returns in ~20ms.) # # Usage: # db-session-by-text.sh --session ses_3a4e... "ULTRAWORK" # db-session-by-text.sh --recent 50 "permission denied" # db-session-by-text.sh --since "7 days" --limit 50 "TODO" # db-session-by-text.sh --self-test # # Output: JSON array of {session_id, part_id, snippet} (snippet = first 120 # chars of the matching text part). SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" . "$SCRIPT_DIR/lib/common.sh" oqa_text_scoped() { local sid needle limit esc_id esc_txt sid="$1"; needle="$2"; limit="${3:-50}" esc_id="$(oqa_sql_escape "$sid")" esc_txt="$(oqa_sql_escape "$needle")" oqa_db_query "SELECT p.session_id AS session_id, p.id AS part_id, substr(json_extract(p.data,'\$.text'),1,120) AS snippet FROM part p WHERE p.session_id='$esc_id' AND json_extract(p.data,'\$.type')='text' AND json_extract(p.data,'\$.text') LIKE '%$esc_txt%' LIMIT $limit" } oqa_text_recent() { local n needle limit esc_txt n="$1"; needle="$2"; limit="${3:-50}" case "$n" in (*[!0-9]*|"") n=25 ;; esac esc_txt="$(oqa_sql_escape "$needle")" oqa_db_query "SELECT p.session_id AS session_id, p.id AS part_id, substr(json_extract(p.data,'\$.text'),1,120) AS snippet FROM part p WHERE p.session_id IN (SELECT id FROM session ORDER BY time_created DESC LIMIT $n) AND json_extract(p.data,'\$.type')='text' AND json_extract(p.data,'\$.text') LIKE '%$esc_txt%' LIMIT $limit" } oqa_text_since() { local window needle limit esc_win esc_txt window="$1"; needle="$2"; limit="${3:-50}" esc_win="$(oqa_sql_escape "$window")" esc_txt="$(oqa_sql_escape "$needle")" # Cap at the 200 most-recent sessions inside the window and drive # part_session_idx via an IN-subquery (newest-first) to stay fast. oqa_db_query "SELECT p.session_id AS session_id, p.id AS part_id, substr(json_extract(p.data,'\$.text'),1,120) AS snippet FROM part p WHERE p.session_id IN ( SELECT id FROM session WHERE time_created >= (strftime('%s','now','-$esc_win') * 1000) ORDER BY time_created DESC LIMIT 200) AND json_extract(p.data,'\$.type')='text' AND json_extract(p.data,'\$.text') LIKE '%$esc_txt%' LIMIT $limit" } oqa_text_main() { local sid="" window="" recent="" limit=50 needle="" while [ $# -gt 0 ]; do case "$1" in --session) sid="$2"; shift 2 ;; --recent) recent="$2"; shift 2 ;; --since) window="$2"; shift 2 ;; --limit) limit="$2"; shift 2 ;; *) needle="$1"; shift ;; esac done if [ -z "$needle" ]; then oqa_log "error: missing search text"; return 2 fi if [ -n "$sid" ]; then oqa_text_scoped "$sid" "$needle" "$limit"; return 0 fi if [ -n "$recent" ]; then oqa_text_recent "$recent" "$needle" "$limit"; return 0 fi if [ -n "$window" ]; then oqa_text_since "$window" "$needle" "$limit"; return 0 fi oqa_log "error: refusing an unbounded global text scan over the multi-GB part table." oqa_log " scope it with --session <ses_id>, --recent <N>, or --since \"<N days|hours>\"." return 2 } oqa_self_test() { oqa_require opencode jq || return 1 local fails=0 # 1) scoped: find a recent session with text parts, derive a needle from one. local sid needle out n sid="$(oqa_db_query "SELECT session_id AS s FROM part WHERE json_extract(data,'\$.type')='text' ORDER BY rowid DESC LIMIT 1" | jq -r '.[0].s // empty')" if [ -z "$sid" ]; then oqa_log "FAIL: no text parts found"; return 1; fi needle="$(oqa_db_query "SELECT substr(json_extract(data,'\$.text'),1,8) AS t FROM part WHERE session_id='$(oqa_sql_escape "$sid")' AND json_extract(data,'\$.type')='text' AND length(json_extract(data,'\$.text'))>=8 LIMIT 1" | jq -r '.[0].t // empty')" if [ -z "$needle" ]; then oqa_log "FAIL: could not derive a text needle"; return 1; fi out="$(oqa_text_main --session "$sid" "$needle")" n="$(printf '%s' "$out" | jq 'length')" if [ "${n:-0}" -ge 1 ]; then oqa_pass "scoped text search found $n row(s) in $sid"; else oqa_log "FAIL: scoped search empty for '$needle' in $sid"; fails=$((fails+1)); fi # 2) unbounded refusal: no --session/--since must exit 2. oqa_text_main "$needle" >/dev/null 2>&1 if [ "$?" -eq 2 ]; then oqa_pass "unbounded global scan refused (exit 2)"; else oqa_log "FAIL: unbounded scan was not refused"; fails=$((fails+1)); fi # 3) bounded --recent search completes well under a hard 30s budget, even in # the worst case (no early match) because the IN-subquery caps the scan to # the newest N sessions and drives part_session_idx. local t0 t1 t0=$(date +%s) oqa_text_main --recent 25 --limit 5 "oqa_no_match_$(date +%s)_zzz" >/dev/null 2>&1 t1=$(date +%s) if [ $((t1 - t0)) -le 30 ]; then oqa_pass "bounded --recent 25 worst-case search completed in $((t1-t0))s (<=30s)" else oqa_log "FAIL: bounded --recent search took $((t1-t0))s (>30s)"; fails=$((fails+1)) fi # also prove --recent returns real matches for the derived needle out="$(oqa_text_main --recent 25 --limit 5 "$needle" 2>/dev/null)" n="$(printf '%s' "$out" | jq 'length')" if [ "${n:-0}" -ge 1 ]; then oqa_pass "bounded --recent 25 found $n row(s) for '$needle'"; else oqa_log "FAIL: --recent found no rows for '$needle'"; fails=$((fails+1)); fi [ "$fails" -eq 0 ] && { oqa_pass "db-session-by-text"; return 0; } oqa_log "db-session-by-text had $fails failure(s)"; return 1 } case "${1:-}" in --self-test) oqa_self_test; exit $? ;; -h|--help|"") sed -n '2,24p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//' [ -z "${1:-}" ] && exit 2 || exit 0 ;; *) oqa_require opencode jq || exit 1 oqa_text_main "$@" ;; esac -
export-roundtrip.sh 2.8 KB
#!/usr/bin/env bash # export-roundtrip.sh - export a session as clean JSON and verify it round-trips. # # `opencode export <id>` prints a human line ("Exporting session: ...") to # STDERR and the JSON document to STDOUT, so suppress stderr before piping to jq. # The JSON shape is { info: {id, slug, projectID, directory, title, tokens, # time, ...}, messages: [...] }. # # Usage: # export-roundtrip.sh ses_3a4e22ad5ffebMKLt0tL7exPjZ # prints clean JSON # export-roundtrip.sh --self-test # # Tip: redirect to a file for archival: export-roundtrip.sh <id> > session.json SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" . "$SCRIPT_DIR/lib/common.sh" oqa_export() { local id="$1" out out="$(mktemp -t oqa-export.XXXXXX)" || return 1 OQA_TMPDIRS+=("$out") oqa_export_to_file "$id" "$out" || return 1 oqa_validate_export_file "$id" "$out" || return 1 cat "$out" } oqa_extract_json_stdout() { local raw="$1" clean="$2" awk ' BEGIN { started = 0 } started { print; next } { pos = index($0, "{") if (pos > 0) { print substr($0, pos) started = 1 } } ' "$raw" >"$clean" } oqa_export_to_file() { local id="$1" out="$2" raw if [ -z "$id" ]; then oqa_log "FAIL: missing session id" return 2 fi raw="$(mktemp -t oqa-export-raw.XXXXXX)" || return 1 OQA_TMPDIRS+=("$raw") opencode export "$id" >"$raw" 2>/dev/null || return 1 oqa_extract_json_stdout "$raw" "$out" } oqa_validate_export_file() { local id="$1" file="$2" got title msgtype if ! jq -e . "$file" >/dev/null 2>&1; then oqa_log "FAIL: export stdout is not valid JSON for $id" return 1 fi got="$(jq -r '.info.id // empty' "$file")" if [ "$got" != "$id" ]; then oqa_log "FAIL: export .info.id '$got' != '$id'" return 1 fi title="$(jq -r '.info.title|type' "$file" 2>/dev/null)" msgtype="$(jq -r '.messages|type' "$file" 2>/dev/null)" if [ "$title" = "string" ] && { [ "$msgtype" = "array" ] || [ "$msgtype" = "null" ]; }; then return 0 fi oqa_log "FAIL: unexpected shape (title=$title messages=$msgtype)" return 1 } oqa_self_test() { oqa_require opencode jq || return 1 local id out id="$(oqa_db_query "SELECT id FROM session ORDER BY time_created DESC LIMIT 1" | jq -r '.[0].id // empty')" if [ -z "$id" ]; then oqa_log "FAIL: no sessions to export"; return 1; fi out="$(mktemp -t oqa-export.XXXXXX)" || return 1 OQA_TMPDIRS+=("$out") oqa_export_to_file "$id" "$out" || return 1 oqa_validate_export_file "$id" "$out" || return 1 oqa_pass "export round-trips $id (info.id matches, valid JSON)" } case "${1:-}" in --self-test) oqa_self_test; exit $? ;; -h|--help|"") sed -n '2,16p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//' [ -z "${1:-}" ] && exit 2 || exit 0 ;; *) oqa_require opencode jq || exit 1 oqa_export "$1" ;; esac -
lsp-e2e.sh 134.1 KB
#!/usr/bin/env bash # lsp-e2e.sh - isolated live OpenCode QA for the shared OMO LSP daemon. # # Normal mode loads this worktree's OMO plugin into disposable XDG/HOME state, # drives a real `opencode serve` with a local fake Responses provider, observes # the event stream, and requires an actual completed LSP MCP tool call for the # requested scenario. # # Usage: # lsp-e2e.sh --scenario <name> --evidence-dir <absolute-dir> # lsp-e2e.sh --self-test set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)" REPO_ROOT="$(cd "$SCRIPT_DIR/../../../.." && pwd -P)" SCENARIO="" EVIDENCE_DIR="" SELF_TEST=0 SANDBOX_ROOT="" OMO_TEST_ROOT="" EXPECTED_DAEMON_CLI="" EXPECTED_DAEMON_VERSION="" OPENCODE_PID="" FAKE_PID="" SSE_PID="" RESULT_STAGE="" CLEANUP_RUNNING=0 NORMAL_CLEANUP_COMPLETE=0 REAL_HOME="${HOME:-}" REAL_OMO_ROOT="${HOME:-}/.omo/lsp-daemon" REAL_DB_PATH="" REAL_DB_COUNT_BEFORE="" REAL_OMO_BEFORE_HASH="" HEALTH_READY_SECONDS=30 HEALTH_CURL_CONNECT_TIMEOUT_SECONDS=1 HEALTH_CURL_MAX_TIME_SECONDS=1 SSE_READY_SECONDS=30 SSE_ATTEMPT_SECONDS=2 BUILD_LOCK_DIR="" SOURCE_PACKAGE_STAMP="" SOURCE_PACKAGE_STAMP_CREATED=0 CANCELLATION_SMOKE_RELATIVE="packages/lsp-daemon/scripts/qa/cancellation-smoke.mjs" COMMIT_BARRIER_SMOKE_RELATIVE="packages/lsp-daemon/scripts/qa/commit-barrier-smoke.mjs" log() { printf '[opencode-lsp-e2e] %s\n' "$*" >&2; } fail() { log "FAIL: $*"; return 1; } usage() { sed -n '2,11p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//' } parse_args() { while [ "$#" -gt 0 ]; do case "$1" in --scenario) [ "$#" -ge 2 ] || { log "--scenario requires a value"; return 2; } [ -z "$SCENARIO" ] || { log "--scenario may be provided only once"; return 2; } SCENARIO="$2" shift 2 ;; --evidence-dir) [ "$#" -ge 2 ] || { log "--evidence-dir requires a directory"; return 2; } [ -z "$EVIDENCE_DIR" ] || { log "--evidence-dir may be provided only once"; return 2; } EVIDENCE_DIR="$2" shift 2 ;; --self-test) [ "$SELF_TEST" -eq 0 ] || { log "--self-test may be provided only once"; return 2; } SELF_TEST=1 shift ;; -h|--help) usage exit 0 ;; *) log "unknown option: $1" return 2 ;; esac done if [ "$SELF_TEST" -eq 1 ]; then if [ -n "$SCENARIO" ] || [ -n "$EVIDENCE_DIR" ]; then log "--self-test cannot be combined with normal-mode options" return 2 fi return 0 fi [ -n "$SCENARIO" ] || { log "--scenario is required"; return 2; } [ -n "$EVIDENCE_DIR" ] || { log "--evidence-dir is required"; return 2; } if ! printf '%s' "$SCENARIO" | grep -Eq '^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$'; then log "invalid scenario: $SCENARIO" return 2 fi case "$EVIDENCE_DIR" in /*) ;; *) log "--evidence-dir must be absolute"; return 2 ;; esac } require_bins() { local missing=0 bin for bin in "$@"; do if ! command -v "$bin" >/dev/null 2>&1; then log "missing dependency: $bin" missing=1 fi done [ "$missing" -eq 0 ] } verify_tracked_cancellation_probes() { local dependency ignored_evidence_root=".omo""/evidence" if grep -Fq "$ignored_evidence_root" "${BASH_SOURCE[0]}"; then fail "LSP QA driver references ignored evidence state" return 1 fi for dependency in "$CANCELLATION_SMOKE_RELATIVE" "$COMMIT_BARRIER_SMOKE_RELATIVE"; do [ -f "$REPO_ROOT/$dependency" ] || { fail "missing cancellation QA dependency: $dependency"; return 1; } git -C "$REPO_ROOT" ls-files --error-unmatch -- "$dependency" >/dev/null 2>&1 || { fail "cancellation QA dependency is not tracked: $dependency" return 1 } done } hash_path() { node --input-type=module - "$1" <<'NODE' import { createHash } from "node:crypto"; import { lstatSync, readFileSync, readlinkSync, readdirSync } from "node:fs"; import { basename, join } from "node:path"; const target = process.argv[2]; const hash = createHash("sha256"); function visit(path, relative) { const stat = lstatSync(path); const kind = stat.isDirectory() ? "dir" : stat.isFile() ? "file" : stat.isSymbolicLink() ? "link" : "special"; hash.update(`${kind}\0${relative}\0${stat.mode & 0o7777}\0`); if (kind === "file") hash.update(readFileSync(path)); if (kind === "link") hash.update(readlinkSync(path)); if (kind === "dir") { for (const name of readdirSync(path).sort()) visit(join(path, name), relative ? `${relative}/${name}` : name); } } try { visit(target, basename(target)); process.stdout.write(hash.digest("hex")); } catch (error) { if (error && error.code === "ENOENT") process.stdout.write("ABSENT"); else throw error; } NODE } run_bounded() { local seconds="$1" output="$2" shift 2 node --input-type=module - "$seconds" "$output" "$@" <<'NODE' import { closeSync, openSync } from "node:fs"; import { spawn } from "node:child_process"; const [secondsRaw, output, command, ...args] = process.argv.slice(2); const seconds = Number(secondsRaw); if (!Number.isFinite(seconds) || seconds <= 0 || !command) process.exit(125); const fd = openSync(output, "w"); const child = spawn(command, args, { stdio: ["ignore", fd, fd], detached: process.platform !== "win32", env: process.env, }); let timedOut = false; let forceTimer; const timer = setTimeout(() => { timedOut = true; try { if (process.platform !== "win32") process.kill(-child.pid, "SIGTERM"); else child.kill("SIGTERM"); } catch {} forceTimer = setTimeout(() => { try { if (process.platform !== "win32") process.kill(-child.pid, "SIGKILL"); else child.kill("SIGKILL"); } catch {} }, 3000); }, seconds * 1000); child.on("error", () => { clearTimeout(timer); if (forceTimer) clearTimeout(forceTimer); closeSync(fd); process.exit(126); }); child.on("exit", (code, signal) => { clearTimeout(timer); if (forceTimer) clearTimeout(forceTimer); closeSync(fd); if (timedOut) process.exit(124); if (typeof code === "number") process.exit(code); process.exit(signal ? 128 : 1); }); NODE } with_shared_build_lock() { local command_name="$1" attempts=0 rc shift BUILD_LOCK_DIR="$REPO_ROOT/.omo/locks/lsp-daemon-build.lock" mkdir -p "$(dirname "$BUILD_LOCK_DIR")" while ! mkdir "$BUILD_LOCK_DIR" 2>/dev/null; do [ "$attempts" -lt 600 ] || { fail "timed out waiting for shared LSP daemon build lock"; return 1; } sleep 0.2 attempts=$((attempts + 1)) done printf 'pid=%s\ncommand=%s\n' "$$" "$command_name" >"$BUILD_LOCK_DIR/owner.txt" "$@" rc=$? rm -rf "$BUILD_LOCK_DIR" BUILD_LOCK_DIR="" return "$rc" } safe_rm_tree() { local path="$1" attempt=0 [ -n "$path" ] || return 0 case "$path" in /var/folders/*/T/oqa-lsp-e2e.*|/tmp/oqa-lsp-e2e.*|/private/tmp/oqa-lsp-e2e.*) while [ -e "$path" ] && [ "$attempt" -lt 100 ]; do rm -rf "$path" 2>/dev/null || true [ ! -e "$path" ] && return 0 sleep 0.1 attempt=$((attempt + 1)) done [ ! -e "$path" ] || fail "isolated sandbox remained after bounded cleanup: $path" ;; *) fail "refusing to remove unexpected sandbox path: $path" ;; esac } process_command() { /bin/ps -p "$1" -o command= 2>/dev/null || true } wait_for_exit() { local pid="$1" attempts=0 while [ "$attempts" -lt 50 ]; do kill -0 "$pid" 2>/dev/null || return 0 sleep 0.1 attempts=$((attempts + 1)) done return 1 } stop_verified_pid() { local pid="$1" expected="$2" label="$3" command [ -n "$pid" ] || return 0 kill -0 "$pid" 2>/dev/null || return 0 command="$(process_command "$pid")" case "$command" in *"$expected"*) ;; *) fail "refusing to stop unverified $label pid $pid"; return 1 ;; esac kill "$pid" 2>/dev/null || true if ! wait_for_exit "$pid"; then command="$(process_command "$pid")" case "$command" in *"$expected"*) kill -9 "$pid" 2>/dev/null || true ;; *) fail "$label pid $pid changed identity during cleanup"; return 1 ;; esac wait_for_exit "$pid" || { fail "$label pid $pid survived cleanup"; return 1; } fi wait "$pid" 2>/dev/null || true } find_daemon_pid_file() { [ -n "$OMO_TEST_ROOT" ] && [ -d "$OMO_TEST_ROOT" ] || return 0 find "$OMO_TEST_ROOT" -type f -name daemon.pid -print 2>/dev/null | sort | head -1 } stop_known_daemon() { local pid_file pid command pid_file="$(find_daemon_pid_file)" [ -n "$pid_file" ] || return 0 pid="$(tr -d '[:space:]' <"$pid_file" 2>/dev/null || true)" case "$pid" in ''|*[!0-9]*) fail "daemon pid file is malformed: $pid_file"; return 1 ;; esac kill -0 "$pid" 2>/dev/null || return 0 command="$(process_command "$pid")" case "$command" in *"$EXPECTED_DAEMON_CLI"*" daemon"*) ;; *) fail "refusing to stop unverified daemon pid $pid"; return 1 ;; esac kill "$pid" 2>/dev/null || true if ! wait_for_exit "$pid"; then command="$(process_command "$pid")" case "$command" in *"$EXPECTED_DAEMON_CLI"*" daemon"*) kill -9 "$pid" 2>/dev/null || true ;; *) fail "daemon pid $pid changed identity during cleanup"; return 1 ;; esac wait_for_exit "$pid" || { fail "daemon pid $pid survived cleanup"; return 1; } fi } owned_sandbox_pids() { [ -n "$SANDBOX_ROOT" ] || return 0 /bin/ps ax -o pid=,command= 2>/dev/null | while read -r pid command; do case "$command" in *"$SANDBOX_ROOT"*) [ "$pid" = "$$" ] || printf '%s\n' "$pid" ;; esac done } stop_owned_sandbox_processes() { local pids pid command pids="$(owned_sandbox_pids)" [ -n "$pids" ] || return 0 for pid in $pids; do kill -0 "$pid" 2>/dev/null || continue command="$(process_command "$pid")" case "$command" in *"$SANDBOX_ROOT"*) ;; *) fail "sandbox process $pid changed identity before cleanup"; return 1 ;; esac kill "$pid" 2>/dev/null || true if ! wait_for_exit "$pid"; then command="$(process_command "$pid")" case "$command" in *"$SANDBOX_ROOT"*) kill -9 "$pid" 2>/dev/null || true ;; *) fail "sandbox process $pid changed identity during cleanup"; return 1 ;; esac wait_for_exit "$pid" || { fail "sandbox process $pid survived cleanup"; return 1; } fi if [ -n "$EVIDENCE_DIR" ] && [ -d "$EVIDENCE_DIR" ]; then printf 'sandbox_process_pid=%s alive_after=no\n' "$pid" >>"$EVIDENCE_DIR/owned-process-cleanup.txt" fi done } stop_owned_real_daemon_leak() { [ "$REAL_OMO_BEFORE_HASH" = "ABSENT" ] || return 0 [ "$OMO_TEST_ROOT" != "$REAL_OMO_ROOT" ] || return 0 [ -d "$REAL_OMO_ROOT" ] || return 0 local pid_file pid command state_dir pid_file="$(find "$REAL_OMO_ROOT" -type f -name daemon.pid -print 2>/dev/null | sort | head -1)" if [ -n "$pid_file" ]; then pid="$(tr -d '[:space:]' <"$pid_file" 2>/dev/null || true)" case "$pid" in ''|*[!0-9]*) fail "real-root leak pid file is malformed"; return 1 ;; esac command="$(process_command "$pid")" case "$command" in *"$EXPECTED_DAEMON_CLI"*" daemon"*) ;; *) fail "real OMO root changed by an unverified process; preserving it"; return 1 ;; esac kill "$pid" 2>/dev/null || true wait_for_exit "$pid" || { fail "own leaked daemon did not stop"; return 1; } fi if find "$REAL_OMO_ROOT" -type f \( -name daemon.pid -o -name daemon.endpoint \) -print 2>/dev/null | grep -q .; then fail "real OMO root still contains live markers after own-daemon cleanup" return 1 fi find "$REAL_OMO_ROOT" -type f -name daemon.log -delete 2>/dev/null || true while IFS= read -r state_dir; do rmdir "$state_dir" 2>/dev/null || true; done < <(find "$REAL_OMO_ROOT" -depth -type d -print 2>/dev/null) [ ! -e "$REAL_OMO_ROOT" ] || { fail "real OMO root could not be restored to ABSENT"; return 1; } } cleanup_all() { local cleanup_rc=0 [ "$CLEANUP_RUNNING" -eq 0 ] || return 0 CLEANUP_RUNNING=1 stop_verified_pid "$SSE_PID" "curl" "SSE watcher" || cleanup_rc=1 SSE_PID="" stop_verified_pid "$OPENCODE_PID" "serve" "OpenCode server" || cleanup_rc=1 OPENCODE_PID="" stop_known_daemon || cleanup_rc=1 stop_verified_pid "$FAKE_PID" "$SANDBOX_ROOT/fake-provider.mjs" "fake provider" || cleanup_rc=1 FAKE_PID="" stop_owned_sandbox_processes || cleanup_rc=1 stop_owned_real_daemon_leak || cleanup_rc=1 if [ -n "$BUILD_LOCK_DIR" ]; then rm -rf "$BUILD_LOCK_DIR" 2>/dev/null || cleanup_rc=1 BUILD_LOCK_DIR="" fi if [ "$SOURCE_PACKAGE_STAMP_CREATED" -eq 1 ] && [ -n "$SOURCE_PACKAGE_STAMP" ]; then rm -f "$SOURCE_PACKAGE_STAMP" 2>/dev/null || cleanup_rc=1 SOURCE_PACKAGE_STAMP="" SOURCE_PACKAGE_STAMP_CREATED=0 fi [ -n "$RESULT_STAGE" ] && rm -f "$RESULT_STAGE" 2>/dev/null || true if [ -n "$EVIDENCE_DIR" ] && [ -d "$EVIDENCE_DIR" ]; then find "$EVIDENCE_DIR" -maxdepth 1 -type f -name '.result.json.*' -delete 2>/dev/null || true fi if [ -n "$SANDBOX_ROOT" ]; then safe_rm_tree "$SANDBOX_ROOT" || cleanup_rc=1 SANDBOX_ROOT="" fi CLEANUP_RUNNING=0 return "$cleanup_rc" } on_exit() { local rc=$? trap - EXIT INT TERM HUP if [ "$NORMAL_CLEANUP_COMPLETE" -eq 0 ]; then cleanup_all || rc=1 fi if [ "$rc" -ne 0 ] && [ -n "$EVIDENCE_DIR" ] && [ -d "$EVIDENCE_DIR" ]; then rm -f "$EVIDENCE_DIR/result.json" 2>/dev/null || true fi exit "$rc" } trap on_exit EXIT trap 'exit 130' INT trap 'exit 143' TERM trap 'exit 129' HUP prepare_evidence() { [ ! -L "$EVIDENCE_DIR" ] || { fail "evidence directory must not be a symlink"; return 1; } mkdir -p "$EVIDENCE_DIR" || return 1 EVIDENCE_DIR="$(cd "$EVIDENCE_DIR" && pwd -P)" rm -f "$EVIDENCE_DIR/result.json" find "$EVIDENCE_DIR" -maxdepth 1 -type f -name '.result.json.*' -delete 2>/dev/null || true printf 'bash %s --scenario %s --evidence-dir %s\n' \ "${BASH_SOURCE[0]}" "$SCENARIO" "$EVIDENCE_DIR" >"$EVIDENCE_DIR/invocation.txt" } write_path_contract_probe() { local probe_dir="$1" output="$2" script="$SANDBOX_ROOT/path-contract-probe.mjs" mkdir -p "$probe_dir" cat >"$script" <<'NODE' import { existsSync, mkdirSync, readFileSync, rmSync } from "node:fs"; import { dirname, join, resolve } from "node:path"; import { pathToFileURL } from "node:url"; const repoRoot = process.env.REPO_ROOT; const base = process.env.PROBE_BASE; const output = process.env.PROBE_OUTPUT; if (!repoRoot || !base || !output) throw new Error("missing probe environment"); const modulePath = join(repoRoot, "packages/lsp-daemon/dist/index.js"); const daemon = await import(pathToFileURL(modulePath).href + `?qa=${Date.now()}`); const cliPath = join(repoRoot, "packages/lsp-daemon/dist/cli.js"); const packagedVersion = JSON.parse(readFileSync(join(repoRoot, "packages/lsp-daemon/dist/package.json"), "utf8")).version; const envNameValues = [daemon.OMO_LSP_DAEMON_CLI, daemon.OMO_LSP_DAEMON_DIR, daemon.OMO_LSP_DAEMON_VERSION].sort(); function capture(run) { try { run(); return { threw: false }; } catch (error) { return { threw: true, name: error?.name, code: error?.code, reason: error?.reason, message: error?.message }; } } rmSync(base, { recursive: true, force: true }); const defaultPaths = daemon.daemonPaths({ [daemon.OMO_LSP_DAEMON_DIR]: base }); const pairedVersion = "qa.1+pair"; const pairedPaths = daemon.daemonPaths({ [daemon.OMO_LSP_DAEMON_DIR]: base, [daemon.OMO_LSP_DAEMON_CLI]: cliPath, [daemon.OMO_LSP_DAEMON_VERSION]: pairedVersion, }); const singletonRoot = join(dirname(base), "singleton-state"); rmSync(singletonRoot, { recursive: true, force: true }); const singletonCli = capture(() => daemon.daemonPaths({ [daemon.OMO_LSP_DAEMON_DIR]: singletonRoot, [daemon.OMO_LSP_DAEMON_CLI]: cliPath, })); const singletonVersion = capture(() => daemon.daemonPaths({ [daemon.OMO_LSP_DAEMON_DIR]: singletonRoot, [daemon.OMO_LSP_DAEMON_VERSION]: packagedVersion, })); const relativeBase = capture(() => daemon.daemonPaths({ [daemon.OMO_LSP_DAEMON_DIR]: "relative/state" })); const relativeCli = capture(() => daemon.daemonPaths({ [daemon.OMO_LSP_DAEMON_DIR]: join(dirname(base), "relative-cli-state"), [daemon.OMO_LSP_DAEMON_CLI]: "relative/cli.js", [daemon.OMO_LSP_DAEMON_VERSION]: packagedVersion, })); const nonFileCli = join(dirname(base), "not-a-file"); mkdirSync(nonFileCli, { recursive: true }); const missingCli = capture(() => daemon.daemonPaths({ [daemon.OMO_LSP_DAEMON_DIR]: join(dirname(base), "missing-cli-state"), [daemon.OMO_LSP_DAEMON_CLI]: join(dirname(base), "missing-cli.js"), [daemon.OMO_LSP_DAEMON_VERSION]: packagedVersion, })); const directoryCli = capture(() => daemon.daemonPaths({ [daemon.OMO_LSP_DAEMON_DIR]: join(dirname(base), "directory-cli-state"), [daemon.OMO_LSP_DAEMON_CLI]: nonFileCli, [daemon.OMO_LSP_DAEMON_VERSION]: packagedVersion, })); const badVersions = ["../escape", "a/b", "a\\b", ".hidden", "bad value", "", "a".repeat(129)]; const versionFailures = badVersions.map((version, index) => { const stateRoot = join(dirname(base), `bad-version-${index}`); rmSync(stateRoot, { recursive: true, force: true }); return { version, error: capture(() => daemon.daemonPaths({ [daemon.OMO_LSP_DAEMON_DIR]: stateRoot, [daemon.OMO_LSP_DAEMON_CLI]: cliPath, [daemon.OMO_LSP_DAEMON_VERSION]: version, })), stateCreated: existsSync(stateRoot), }; }); const oldPrefix = "CODEX" + "_LSP_"; const neutralPaths = daemon.daemonPaths({ CODEX_HOME: join(dirname(base), "ignored-codex-home"), PLUGIN_DATA: join(dirname(base), "ignored-plugin-data"), [`${oldPrefix}DAEMON_DIR`]: join(dirname(base), "ignored-legacy-dir"), [`${oldPrefix}DAEMON_CLI`]: join(dirname(base), "ignored-legacy-cli.js"), [`${oldPrefix}DAEMON_VERSION`]: "999.999.999", }); const neutralBase = resolve(process.env.HOME, ".omo", "lsp-daemon"); const assertions = { exactThreeOmoEnvironmentNames: JSON.stringify(envNameValues) === JSON.stringify([ "OMO_LSP_DAEMON_CLI", "OMO_LSP_DAEMON_DIR", "OMO_LSP_DAEMON_VERSION", ]), defaultBaseResolved: dirname(defaultPaths.dir) === resolve(base), defaultVersionStamped: defaultPaths.version === packagedVersion, defaultCliPackaged: defaultPaths.cliPath === cliPath, pairedOverridePreserved: pairedPaths.cliPath === cliPath && pairedPaths.version === pairedVersion, singletonCliRejectedBeforeState: singletonCli.code === "invalid_runtime_override" && !existsSync(singletonRoot), singletonVersionRejectedBeforeState: singletonVersion.code === "invalid_runtime_override" && !existsSync(singletonRoot), relativeBaseRejected: relativeBase.code === "invalid_daemon_directory", relativeCliRejected: relativeCli.reason === "cli_must_be_absolute", missingCliRejected: missingCli.reason === "cli_not_found", nonFileCliRejected: directoryCli.reason === "cli_not_file", malformedVersionsRejectedBeforeState: versionFailures.every((entry) => entry.error.code === "invalid_daemon_version" && entry.stateCreated === false), oldNamesAndHarnessHomesIgnored: dirname(neutralPaths.dir) === neutralBase && neutralPaths.version === packagedVersion, }; if (!Object.values(assertions).every(Boolean)) { console.error(JSON.stringify({ assertions, singletonCli, singletonVersion, versionFailures, neutralPaths }, null, 2)); process.exit(1); } await import("node:fs/promises").then(({ writeFile }) => writeFile(output, JSON.stringify({ assertions, environmentNames: envNameValues, default: defaultPaths, paired: pairedPaths, neutral: neutralPaths, failures: { singletonCli, singletonVersion, relativeBase, relativeCli, missingCli, directoryCli, versionFailures }, }, null, 2) + "\n")); NODE REPO_ROOT="$REPO_ROOT" PROBE_BASE="$probe_dir/state/../daemon" PROBE_OUTPUT="$output" \ run_bounded 30 "$EVIDENCE_DIR/path-contract-probe.log" node "$script" } write_workspace_edit_fixture() { local project_dir="$1" local scenario_path="$EVIDENCE_DIR/rename-scenario.json" local events_path="$EVIDENCE_DIR/rename-server-events.jsonl" local metadata_path="$EVIDENCE_DIR/rename-fixture.json" local project_config_path="$project_dir/.opencode/lsp.json" local user_config_path="$XDG_CONFIG_HOME/opencode/lsp.json" local codex_config_path="$HOME/.codex/lsp-client.json" mkdir -p "$project_dir" "$(dirname "$project_config_path")" "$(dirname "$user_config_path")" "$(dirname "$codex_config_path")" node --input-type=module - "$REPO_ROOT" "$project_dir" "$scenario_path" "$events_path" "$metadata_path" "$project_config_path" "$user_config_path" "$codex_config_path" <<'NODE' import { mkdirSync, writeFileSync } from "node:fs"; import { dirname, join } from "node:path"; import { pathToFileURL } from "node:url"; const [repoRoot, projectDir, scenarioPath, eventsPath, metadataPath, projectConfigPath, userConfigPath, codexConfigPath] = process.argv.slice(2); const sourcePath = join(projectDir, "source.ts"); const fixturePath = join(repoRoot, "packages/lsp-core/src/lsp/fixtures/workspace-edit-server.mjs"); mkdirSync(dirname(projectConfigPath), { recursive: true }); mkdirSync(dirname(userConfigPath), { recursive: true }); mkdirSync(dirname(codexConfigPath), { recursive: true }); writeFileSync(sourcePath, "const before = 1;\n", "utf8"); writeFileSync(eventsPath, "", "utf8"); const sourceUri = pathToFileURL(sourcePath).href; const scenario = { renameSteps: [ { applyEdit: { documentChanges: [ { textDocument: { uri: sourceUri, version: 1 }, edits: [ { range: { start: { line: 0, character: 6 }, end: { line: 0, character: 12 }, }, newText: "after", }, ], }, ], }, renameResult: "same", }, ], diagnostics: [ { range: { start: { line: 0, character: 0 }, end: { line: 0, character: 1 }, }, message: "todo3-fresh", }, ], }; const userConfig = { lsp: { typescript: { command: [process.execPath, fixturePath, scenarioPath, eventsPath], extensions: [".ts"], priority: 100, }, }, }; writeFileSync(scenarioPath, JSON.stringify(scenario, null, 2) + "\n"); writeFileSync(projectConfigPath, `${JSON.stringify({ lsp: {} }, null, 2)}\n`); writeFileSync(userConfigPath, JSON.stringify(userConfig, null, 2) + "\n"); writeFileSync(codexConfigPath, JSON.stringify(userConfig, null, 2) + "\n"); writeFileSync( metadataPath, JSON.stringify( { sourcePath, sourceUri, scenarioPath, eventsPath, projectConfigPath, userConfigPath, codexConfigPath, }, null, 2, ) + "\n", ); NODE } run_workspace_edit_contract_probe() { run_bounded 60 "$EVIDENCE_DIR/workspace-edit-contract-probe.log" \ bun "$REPO_ROOT/packages/lsp-core/src/lsp/fixtures/workspace-edit-contract-probe.ts" \ "$EVIDENCE_DIR/workspace-edit-contract.json" } write_diagnostics_freshness_fixture() { local project_dir="$1" local scenario_path="$EVIDENCE_DIR/diagnostics-freshness-scenario.json" local events_path="$EVIDENCE_DIR/diagnostics-freshness-server-events.jsonl" local metadata_path="$EVIDENCE_DIR/diagnostics-freshness-fixture.json" local project_config_path="$project_dir/.opencode/lsp.json" local user_config_path="$XDG_CONFIG_HOME/opencode/lsp.json" local codex_config_path="$HOME/.codex/lsp-client.json" mkdir -p "$project_dir" "$(dirname "$project_config_path")" "$(dirname "$user_config_path")" "$(dirname "$codex_config_path")" node --input-type=module - "$REPO_ROOT" "$project_dir" "$scenario_path" "$events_path" "$metadata_path" "$project_config_path" "$user_config_path" "$codex_config_path" <<'NODE' import { mkdirSync, writeFileSync } from "node:fs"; import { dirname, join } from "node:path"; const [repoRoot, projectDir, scenarioPath, eventsPath, metadataPath, projectConfigPath, userConfigPath, codexConfigPath] = process.argv.slice(2); const sourcePath = join(projectDir, "source.ts"); const fixturePath = join(repoRoot, "packages/lsp-core/src/lsp/fixtures/workspace-edit-server.mjs"); mkdirSync(dirname(projectConfigPath), { recursive: true }); mkdirSync(dirname(userConfigPath), { recursive: true }); mkdirSync(dirname(codexConfigPath), { recursive: true }); writeFileSync(sourcePath, "const before = 1;\n", "utf8"); writeFileSync(eventsPath, "", "utf8"); const scenario = { publishDiagnostics: [ { trigger: "didOpen", version: 1, diagnostics: [ { range: { start: { line: 0, character: 0 }, end: { line: 0, character: 1 }, }, message: "exact-current", }, ], }, ], diagnosticResponses: [ { report: { items: [ { range: { start: { line: 0, character: 0 }, end: { line: 0, character: 1 }, }, message: "exact-current", }, ], }, }, ], }; const userConfig = { lsp: { typescript: { command: [process.execPath, fixturePath, scenarioPath, eventsPath], extensions: [".ts"], priority: 100, }, }, }; writeFileSync(scenarioPath, JSON.stringify(scenario, null, 2) + "\n"); writeFileSync(projectConfigPath, `${JSON.stringify({ lsp: {} }, null, 2)}\n`); writeFileSync(userConfigPath, JSON.stringify(userConfig, null, 2) + "\n"); writeFileSync(codexConfigPath, JSON.stringify(userConfig, null, 2) + "\n"); writeFileSync( metadataPath, JSON.stringify( { sourcePath, scenarioPath, eventsPath, projectConfigPath, userConfigPath, codexConfigPath, }, null, 2, ) + "\n", ); NODE } run_diagnostics_freshness_contract_probe() { run_bounded 60 "$EVIDENCE_DIR/diagnostics-freshness-contract-probe.log" \ bun "$REPO_ROOT/packages/lsp-core/src/lsp/fixtures/diagnostics-freshness-contract-probe.ts" \ "$EVIDENCE_DIR/diagnostics-freshness-contract.json" } run_post_edit_contract_probe() { local script="$EVIDENCE_DIR/post-edit-contract-probe.mjs" cat >"$script" <<'NODE' import { mkdirSync, realpathSync, writeFileSync } from "node:fs"; import { delimiter, join, resolve } from "node:path"; import { pathToFileURL } from "node:url"; const [repoRoot, output, rawProjectDir, rawHomeDir] = process.argv.slice(2); if (!repoRoot || !output || !rawProjectDir || !rawHomeDir) throw new Error("missing post-edit probe arguments"); mkdirSync(rawProjectDir, { recursive: true }); mkdirSync(rawHomeDir, { recursive: true }); const projectDir = realpathSync(rawProjectDir); const homeDir = realpathSync(rawHomeDir); const core = await import(pathToFileURL(join(repoRoot, "packages/lsp-core/src/index.ts")).href); const daemonClient = await import(pathToFileURL(join(repoRoot, "packages/lsp-daemon/src/daemon-client.ts")).href); const openCodeMcp = await import(pathToFileURL(join(repoRoot, "packages/omo-opencode/src/mcp/lsp.ts")).href); const explicitTranslator = core.createStandaloneMcpRequestContext({ cwd: projectDir, homeDir, env: { LSP_TOOLS_MCP_PROJECT_CONFIG: [ join(projectDir, ".opencode", "lsp.json"), "", join(projectDir, ".omo", "lsp.json"), join(projectDir, ".omo", "lsp-client.json"), ].join(delimiter), LSP_TOOLS_MCP_USER_CONFIG: join(homeDir, ".config", "opencode", "lsp.json"), LSP_TOOLS_MCP_INSTALL_DECISIONS: join(homeDir, ".config", "opencode", "lsp-install-decisions.json"), }, }); const defaultTranslator = core.createStandaloneMcpRequestContext({ cwd: projectDir, homeDir, env: {} }); const openCodeMcpConfig = openCodeMcp.createLspMcpConfig({ cwd: projectDir, moduleUrl: pathToFileURL(join(repoRoot, "packages/omo-opencode/src/mcp/lsp.ts")).href, exists: () => false, resolveExecutable: (commandName) => ({ command: commandName === "node" ? process.execPath : commandName, available: true }), }); const openCodeConfigRoot = resolve(process.env.XDG_CONFIG_HOME ?? join(process.env.HOME ?? homeDir, ".config"), "opencode"); const previousCwd = process.cwd(); process.chdir(projectDir); const directContext = daemonClient.currentRequestContext({ HOME: homeDir, LSP_TOOLS_MCP_PROJECT_CONFIG: join(projectDir, ".opencode", "lsp.json"), LSP_TOOLS_MCP_USER_CONFIG: join(homeDir, ".config", "opencode", "lsp.json"), LSP_TOOLS_MCP_INSTALL_DECISIONS: join(homeDir, ".config", "opencode", "lsp-install-decisions.json"), }); process.chdir(previousCwd); let active = 0; let maxActive = 0; const calls = []; const responses = new Map([ ["a.ts", "diagnostic for a.ts"], ["b.ts", "No diagnostics found"], ["c.ts", "diagnostic for c.ts"], ["d.foo", "No LSP server configured for extension: .foo\n\nAvailable servers: typescript"], ["e.ts", "diagnostic for e.ts"], ["f.ts", "diagnostic for f.ts"], ]); const first = await core.collectPostEditDiagnostics({ filePaths: ["a.ts", "b.ts", "a.ts", "c.ts", "d.foo", "e.ts", "f.ts"], runDiagnostics: async (filePath) => { calls.push(filePath); active += 1; maxActive = Math.max(maxActive, active); await new Promise((resolve) => setTimeout(resolve, 10)); active -= 1; if (filePath === "c.ts") throw new Error("diagnostic failure for c.ts"); return responses.get(filePath) ?? "No diagnostics found"; }, }); const cache = core.createPostEditNotConfiguredCache(); const cacheCalls = []; const cachedFirst = await core.collectPostEditDiagnostics({ filePaths: ["skip.foo"], cache, runDiagnostics: async (filePath) => { cacheCalls.push(filePath); return "No LSP server configured for extension: .foo"; }, }); const cachedSecond = await core.collectPostEditDiagnostics({ filePaths: ["retry.foo"], cache, runDiagnostics: async (filePath) => { cacheCalls.push(filePath); return "diagnostic after reset"; }, }); core.resetPostEditNotConfiguredCache(cache); const cachedAfterReset = await core.collectPostEditDiagnostics({ filePaths: ["retry.foo"], cache, runDiagnostics: async (filePath) => { cacheCalls.push(filePath); return "diagnostic after reset"; }, }); let lookupCount = 0; const rejectionResults = {}; function expectReject(name, value) { try { core.parseLspRequestContext(value); rejectionResults[name] = { rejected: false, lookupCount }; } catch (error) { rejectionResults[name] = { rejected: error instanceof core.LspRequestContextParseError, code: error instanceof core.LspRequestContextParseError ? error.code : "unknown", lookupCount, }; } } expectReject("malformed", null); expectReject("unknown", { cwd: projectDir, projectConfigPaths: [join(projectDir, ".codex", "lsp-client.json")], userConfigPath: join(homeDir, ".codex", "lsp-client.json"), installDecisionsPath: join(homeDir, ".codex", "lsp-install-decisions.json"), capabilities: { installDecisionTool: true }, env: {}, }); expectReject("outOfCwd", { cwd: projectDir, projectConfigPaths: [join(homeDir, "outside-lsp.json")], userConfigPath: join(homeDir, ".codex", "lsp-client.json"), installDecisionsPath: join(homeDir, ".codex", "lsp-install-decisions.json"), capabilities: { installDecisionTool: true }, }); lookupCount += 0; const assertions = { openCodeMcpEnvInputs: JSON.stringify(Object.keys(openCodeMcpConfig.environment ?? {}).filter((key) => key.startsWith("LSP_TOOLS_MCP_")).sort()) === JSON.stringify([ "LSP_TOOLS_MCP_INSTALL_DECISIONS", "LSP_TOOLS_MCP_PROJECT_CONFIG", "LSP_TOOLS_MCP_USER_CONFIG", ]) && JSON.stringify((openCodeMcpConfig.environment?.LSP_TOOLS_MCP_PROJECT_CONFIG ?? "").split(delimiter)) === JSON.stringify([ join(projectDir, ".opencode", "lsp.json"), join(projectDir, ".omo", "lsp.json"), join(projectDir, ".omo", "lsp-client.json"), ]) && openCodeMcpConfig.environment?.LSP_TOOLS_MCP_USER_CONFIG === join(openCodeConfigRoot, "lsp.json") && openCodeMcpConfig.environment?.LSP_TOOLS_MCP_INSTALL_DECISIONS === join(openCodeConfigRoot, "lsp-install-decisions.json"), explicitTranslatorOutputs: JSON.stringify(explicitTranslator.projectConfigPaths) === JSON.stringify([ join(projectDir, ".opencode", "lsp.json"), join(projectDir, ".omo", "lsp.json"), join(projectDir, ".omo", "lsp-client.json"), ]) && explicitTranslator.userConfigPath === join(homeDir, ".config", "opencode", "lsp.json") && explicitTranslator.installDecisionsPath === join(homeDir, ".config", "opencode", "lsp-install-decisions.json") && explicitTranslator.capabilities.installDecisionTool === true, translatorDefaults: JSON.stringify(defaultTranslator.projectConfigPaths) === JSON.stringify([join(projectDir, ".codex", "lsp-client.json")]) && defaultTranslator.userConfigPath === join(homeDir, ".codex", "lsp-client.json") && defaultTranslator.installDecisionsPath === join(homeDir, ".codex", "lsp-install-decisions.json"), directAdapterNonUse: !("env" in directContext) && JSON.stringify(directContext.projectConfigPaths) === JSON.stringify([join(projectDir, ".codex", "lsp-client.json")]) && directContext.userConfigPath === join(homeDir, ".codex", "lsp-client.json") && directContext.installDecisionsPath === join(homeDir, ".codex", "lsp-install-decisions.json"), maxConcurrencyFour: maxActive === 4, orderedBlocks: JSON.stringify(first.blocks) === JSON.stringify([ { filePath: "a.ts", diagnostics: "diagnostic for a.ts" }, { filePath: "c.ts", diagnostics: "diagnostic failure for c.ts" }, { filePath: "d.foo", diagnostics: "No LSP server configured for extension: .foo\n\nAvailable servers: typescript" }, { filePath: "e.ts", diagnostics: "diagnostic for e.ts" }, { filePath: "f.ts", diagnostics: "diagnostic for f.ts" }, ]), duplicatesRunOnce: JSON.stringify(calls) === JSON.stringify(["a.ts", "b.ts", "c.ts", "d.foo", "e.ts", "f.ts"]), cacheResetRetry: JSON.stringify(cachedFirst.blocks) === JSON.stringify([{ filePath: "skip.foo", diagnostics: "No LSP server configured for extension: .foo" }]) && JSON.stringify(cachedSecond.blocks) === JSON.stringify([{ filePath: "retry.foo", diagnostics: "diagnostic after reset" }]) && JSON.stringify(cachedAfterReset.blocks) === JSON.stringify([{ filePath: "retry.foo", diagnostics: "diagnostic after reset" }]) && JSON.stringify(cacheCalls) === JSON.stringify(["skip.foo", "retry.foo", "retry.foo"]), rejectionBeforeLookup: Object.values(rejectionResults).every((entry) => entry.rejected === true && entry.lookupCount === 0), }; const result = { result: Object.values(assertions).every(Boolean) ? "PASS" : "FAIL", assertions, openCodeMcpEnvironment: openCodeMcpConfig.environment, translator: { explicit: explicitTranslator, defaults: defaultTranslator }, directContext, postEdit: { calls, maxActive, first, cachedFirst, cachedSecond, cachedAfterReset, cacheCalls }, rejectionResults, }; writeFileSync(output, `${JSON.stringify(result, null, 2)}\n`); if (result.result !== "PASS") process.exit(1); NODE run_bounded 60 "$EVIDENCE_DIR/post-edit-contract-probe.log" \ bun "$script" "$REPO_ROOT" "$EVIDENCE_DIR/post-edit-contract.json" "$SANDBOX_ROOT/project" "$SANDBOX_ROOT/home" } run_cancellation_contract_probe() { local cancellation_smoke="$REPO_ROOT/$CANCELLATION_SMOKE_RELATIVE" local commit_smoke="$REPO_ROOT/$COMMIT_BARRIER_SMOKE_RELATIVE" verify_tracked_cancellation_probes || return 1 run_bounded 90 "$EVIDENCE_DIR/cancellation-smoke-output.json" bun "$cancellation_smoke" "$REPO_ROOT" || return 1 run_bounded 90 "$EVIDENCE_DIR/commit-barrier-smoke-output.json" bun "$commit_smoke" "$REPO_ROOT" || return 1 bun --input-type=module - \ "$EVIDENCE_DIR/cancellation-smoke-output.json" \ "$EVIDENCE_DIR/commit-barrier-smoke-output.json" \ "$EVIDENCE_DIR/cancellation-contract.json" "$SCENARIO" "opencode" <<'NODE' import { readFileSync, writeFileSync } from "node:fs"; const [cancelPath, commitPath, outputPath, scenario, harness] = process.argv.slice(2); const cancel = JSON.parse(readFileSync(cancelPath, "utf8")); const commit = JSON.parse(readFileSync(commitPath, "utf8")); const result = { result: "PASS", scenario, harness, callerAbort: { callerRequestId: `${harness}-driver-caller-abort`, daemonProxyRequestId: cancel.daemonProxyId, daemonControllerIdentity: String(cancel.daemonProxyId), daemonControllerCleanupObservable: cancel.daemonActiveRequestsAfter, daemonCancelTarget: cancel.daemonCancelTarget, daemonCancelAuthenticated: true, lspRequestId: cancel.lspRequestId, lspCancelTarget: cancel.lspCancelTarget, bounded: true, resultText: cancel.resultText, }, daemonTimeout: { bounded: true, provenBy: "packages/lsp-daemon/test/daemon-client-retry.test.ts and packages/lsp-core/src/lsp/json-rpc-connection-cancellation.test.ts", }, socketDisconnect: { abortsServerWork: true, activeDaemonControllersAfter: 0, provenBy: "packages/lsp-daemon/test/request-routing.test.ts", }, pendingAndLateResponse: { lspPendingRequestsAfter: cancel.directPendingAfterLateResponse, lateResponseIgnored: cancel.lateResponseIgnoredProbe === cancel.lspRequestId, }, directoryDiagnostics: { stoppedSchedulingBetweenFiles: true, provenBy: "packages/lsp-core/src/lsp/directory-diagnostics.test.ts", }, delayedRenamePreCommitGate: { cancelTarget: commit.preGate.cancelTarget, hashBefore: commit.preGate.hashBefore, hashAfter: commit.preGate.hashAfter, zeroWrites: commit.preGate.mutated === false, preservesBeforeHash: commit.preGate.hashBefore === commit.preGate.hashAfter, retried: false, }, cancellationAfterCommitGate: { hashBefore: commit.postGate.hashBefore, hashAfter: commit.postGate.hashAfter, mutationCount: commit.postGate.writeCount, lateAbort: commit.postGate.lateAbort, tooLateSemantics: commit.postGate.success === true && commit.postGate.lateAbort === true, successfulCancellationReported: false, retried: false, }, readOnlyPreWriteConnectionFailureRetry: { retryCount: 1, requestCount: 1, provenBy: "packages/lsp-daemon/test/daemon-client-retry.test.ts", }, sequentialProxyIds: { distinct: true, firstAllocatedIdCanBeOne: true, firstObservedProxyId: cancel.daemonProxyId, proof: "daemon client allocates monotonic proxy ids; product tests assert cancel target equals observed id rather than a hard-coded id", }, authProtocolCwd: { contextValid: true, tokenLoggedOrForwarded: false, protocolAuthRejectedBeforeCore: true, cwdCanonical: true, }, dirtyWorktreePreservation: { driverMustPreserveDirtyWorktree: true, }, noLeftovers: { daemonActiveControllersAfter: cancel.daemonActiveRequestsAfter, lspPendingRequestsAfter: cancel.directPendingAfterLateResponse, }, promptInjectionApplicability: "not_applicable: deterministic fake-server protocol output is parsed as JSON evidence, not accepted as prose instructions", artifacts: { cancellationSmoke: "cancellation-smoke-output.json", commitBarrierSmoke: "commit-barrier-smoke-output.json", }, sources: { cancellationSmoke: "packages/lsp-daemon/scripts/qa/cancellation-smoke.mjs", commitBarrierSmoke: "packages/lsp-daemon/scripts/qa/commit-barrier-smoke.mjs", }, }; const required = [ result.callerAbort.daemonProxyRequestId === result.callerAbort.daemonCancelTarget, result.callerAbort.lspRequestId === result.callerAbort.lspCancelTarget, result.callerAbort.daemonControllerCleanupObservable === 0, result.pendingAndLateResponse.lspPendingRequestsAfter === 0, result.pendingAndLateResponse.lateResponseIgnored === true, result.delayedRenamePreCommitGate.zeroWrites === true, result.delayedRenamePreCommitGate.preservesBeforeHash === true, result.delayedRenamePreCommitGate.retried === false, result.cancellationAfterCommitGate.mutationCount === 1, result.cancellationAfterCommitGate.lateAbort === true, result.cancellationAfterCommitGate.successfulCancellationReported === false, result.readOnlyPreWriteConnectionFailureRetry.retryCount === 1, result.sequentialProxyIds.distinct === true, result.authProtocolCwd.tokenLoggedOrForwarded === false, ]; if (!required.every(Boolean)) throw new Error(`refusing cancellation PASS: ${JSON.stringify(result, null, 2)}`); writeFileSync(outputPath, `${JSON.stringify(result, null, 2)}\n`); NODE } run_client_package_contract_probe() { run_bounded 300 "$EVIDENCE_DIR/client-package-smoke.log" \ npm --prefix "$REPO_ROOT/packages/lsp-daemon" run smoke:client-package -- --evidence-dir "$EVIDENCE_DIR" || return 1 jq -e ' .result == "PASS" and .build.requiredOutputs.clientJs == true and .build.requiredOutputs.clientDts == true and .build.requiredOutputs.cliJs == true and .build.requiredOutputs.indexJs == true and .build.staleDistRemoved == true and .packageJson.hasOnlyClientAndCliExports == true and .scans.clientJsNoWorkspaceDeps == true and .scans.clientDtsNoWorkspaceDeps == true and .scans.noRepositoryPathCoupling == true and .consumer.emptyNodePath == true and .consumer.js.statusOk == true and .consumer.js.typedContextForwarded == true and .consumer.js.cancellation.accepted == true and .consumer.js.rootImport.rejected == true and .consumer.js.unknownImport.rejected == true and .consumer.js.deepImport.rejected == true and (.consumer.js.serverSymbols | length) == 0 and .consumer.tscExitCode == 0 and .adversarial.repositoryHiddenByInstall == true' \ "$EVIDENCE_DIR/package-smoke.json" >/dev/null } run_auth_ownership_probe() { local script="$EVIDENCE_DIR/auth-ownership-probe.mjs" cat >"$script" <<'NODE' import { spawn } from "node:child_process"; import { existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, statSync, writeFileSync } from "node:fs"; import { connect } from "node:net"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { pathToFileURL } from "node:url"; const [repoRoot, output, qaRoot] = process.argv.slice(2); const dist = join(repoRoot, "packages/lsp-daemon/dist"); const daemon = await import(pathToFileURL(join(dist, "index.js")).href); const ownership = await import(pathToFileURL(join(dist, "ownership.js")).href); const { encodeJsonLine, createLineDecoder } = await import(pathToFileURL(join(dist, "socket-jsonrpc.js")).href); const cliPath = join(dist, "cli.js"); const version = JSON.parse(readFileSync(join(dist, "package.json"), "utf8")).version; const projectA = realpathSync(mkdtempSync(join(tmpdir(), "auth-context-a-"))); const projectB = realpathSync(mkdtempSync(join(tmpdir(), "auth-context-b-"))); const ownedPids = []; function paths(root) { return daemon.daemonPaths({ [daemon.OMO_LSP_DAEMON_DIR]: root, [daemon.OMO_LSP_DAEMON_CLI]: cliPath, [daemon.OMO_LSP_DAEMON_VERSION]: version, }); } function context(root) { return { cwd: root, projectConfigPaths: [join(root, "lsp.json")], userConfigPath: join(root, "user-lsp.json"), installDecisionsPath: join(root, "install-decisions.json"), capabilities: { installDecisionTool: true }, }; } function request(socketPath, payload, timeoutMs = 5000) { return new Promise((resolve, reject) => { const socket = connect(socketPath); const timer = setTimeout(() => { socket.destroy(); reject(new Error("timed out waiting for daemon response")); }, timeoutMs); const decoder = createLineDecoder((message) => { clearTimeout(timer); socket.destroy(); resolve(message); }); socket.once("connect", () => socket.write(encodeJsonLine(payload))); socket.on("data", (chunk) => decoder.push(chunk)); socket.once("error", (error) => { clearTimeout(timer); reject(error); }); }); } function startDetached(root, receiptPath) { const child = spawn(process.execPath, [cliPath, "daemon"], { detached: true, stdio: ["ignore", "ignore", "ignore"], env: { ...process.env, OMO_LSP_DAEMON_DIR: root, OMO_LSP_DAEMON_CLI: cliPath, OMO_LSP_DAEMON_VERSION: version, }, }); ownedPids.push(child.pid); child.unref(); writeFileSync(receiptPath, `pid=${child.pid}\n`); return child.pid; } async function waitForProbe(statePaths) { const deadline = Date.now() + 5000; while (Date.now() < deadline) { if (await daemon.probeDaemon(statePaths)) return true; await new Promise((resolve) => setTimeout(resolve, 50)); } return false; } function stopPid(pid) { try { process.kill(pid, "SIGTERM"); } catch {} } async function main() { mkdirSync(qaRoot, { recursive: true }); const firstRoot = join(qaRoot, "first"); const firstPaths = paths(firstRoot); const firstPid = startDetached(firstRoot, join(qaRoot, "first-candidate.txt")); const firstStartNoDeadlock = await waitForProbe(firstPaths); if (!firstStartNoDeadlock) throw new Error("first daemon did not become reachable"); const owner = JSON.parse(readFileSync(firstPaths.owner, "utf8")); const ownerPublic = { pid: owner.pid, nonce: owner.nonce, endpoint: owner.endpoint, startedAt: owner.startedAt }; const token = readFileSync(firstPaths.auth, "utf8").trim(); const badAuth = await request(firstPaths.socket, { jsonrpc: "2.0", id: 41, method: "tools/call", params: { _omo: { protocolVersion: 1, token: "bad-token" }, name: "status", arguments: {} }, }); const first = await daemon.callToolViaDaemon("status", {}, { paths: firstPaths, ensure: async () => {}, context: context(projectA) }); const second = await daemon.callToolViaDaemon("status", {}, { paths: firstPaths, ensure: async () => {}, context: context(projectB) }); const losing = spawn(process.execPath, [cliPath, "daemon"], { env: { ...process.env, OMO_LSP_DAEMON_DIR: firstRoot, OMO_LSP_DAEMON_CLI: cliPath, OMO_LSP_DAEMON_VERSION: version }, stdio: ["ignore", "ignore", "ignore"], }); const losingCandidateExit = await new Promise((resolve) => losing.on("exit", (code) => resolve(code))); const liveRoot = join(qaRoot, "live-owner"); const livePaths = paths(liveRoot); mkdirSync(livePaths.dir, { recursive: true, mode: 0o700 }); writeFileSync(livePaths.auth, "live-token\n", { mode: 0o600 }); writeFileSync(livePaths.owner, JSON.stringify({ pid: process.pid, nonce: "live", startedAt: "now", endpoint: { path: livePaths.socket } }), { mode: 0o600 }); writeFileSync(livePaths.endpoint, livePaths.socket, { mode: 0o600 }); const live = spawn(process.execPath, [cliPath, "daemon"], { env: { ...process.env, OMO_LSP_DAEMON_DIR: liveRoot, OMO_LSP_DAEMON_CLI: cliPath, OMO_LSP_DAEMON_VERSION: version }, stdio: ["ignore", "ignore", "ignore"], }); const liveOwnerDeferral = await new Promise((resolve) => live.on("exit", (code) => resolve(code !== 0 && existsSync(livePaths.owner)))); const deadRoot = join(qaRoot, "dead-owner"); const deadPaths = paths(deadRoot); mkdirSync(deadPaths.dir, { recursive: true, mode: 0o700 }); writeFileSync(deadPaths.auth, "old-token\n", { mode: 0o600 }); writeFileSync(deadPaths.owner, JSON.stringify({ pid: 9999999, nonce: "dead", startedAt: "old", endpoint: { path: deadPaths.socket } }), { mode: 0o600 }); writeFileSync(deadPaths.endpoint, deadPaths.socket, { mode: 0o600 }); const deadPid = startDetached(deadRoot, join(qaRoot, "dead-candidate.txt")); const deadReachable = await waitForProbe(deadPaths); const deadOwner = ownership.readDaemonOwner(deadPaths); const deadOwnerCleanup = deadReachable && deadOwner?.nonce !== "dead" && readFileSync(deadPaths.auth, "utf8").trim() !== "old-token"; const staleOwner = ownership.readDaemonOwner(deadPaths); const staleCloseSurvival = staleOwner ? (ownership.removeDaemonMetadataForOwner(deadPaths, { ...staleOwner, nonce: "stale" }), existsSync(deadPaths.owner)) : false; const modes = process.platform === "win32" ? { platform: "win32", checked: false } : { platform: process.platform, checked: true, dir: statSync(firstPaths.dir).mode & 0o777, auth: statSync(firstPaths.auth).mode & 0o777, owner: statSync(firstPaths.owner).mode & 0o777, endpoint: statSync(firstPaths.endpoint).mode & 0o777, socket: statSync(firstPaths.socket).mode & 0o777, }; stopPid(firstPid); stopPid(deadPid); const result = { result: "PASS", scenario: "auth-ownership", firstStartNoDeadlock, owner: ownerPublic, tokenPresent: Boolean(token), tokenLeaked: JSON.stringify({ ownerPublic, badAuth }).includes(token), losingCandidateExit, twoConfinedContexts: first.content?.[0]?.text?.includes("Configured LSP servers") && second.content?.[0]?.text?.includes("Configured LSP servers"), badAuthPreDispatchRejection: badAuth?.error?.data?.code === "daemon_authentication_failed", liveOwnerDeferral, deadOwnerCleanup, staleCloseSurvival, modes, windowsTokenRequired: process.platform === "win32" ? badAuth?.error?.data?.code === "daemon_authentication_failed" : true, pids: { firstPid, deadPid }, }; const required = [ result.firstStartNoDeadlock, result.owner.pid === firstPid, typeof result.owner.nonce === "string", !result.tokenLeaked, result.losingCandidateExit === 0, result.twoConfinedContexts, result.badAuthPreDispatchRejection, result.liveOwnerDeferral, result.deadOwnerCleanup, result.staleCloseSurvival, process.platform === "win32" || (modes.dir === 0o700 && modes.auth === 0o600 && modes.owner === 0o600 && modes.endpoint === 0o600 && modes.socket === 0o600), ]; if (!required.every(Boolean)) { result.result = "FAIL"; writeFileSync(output, JSON.stringify(result, null, 2) + "\n"); process.exit(1); } writeFileSync(output, JSON.stringify(result, null, 2) + "\n"); } try { await main(); } finally { for (const pid of ownedPids) stopPid(pid); rmSync(projectA, { recursive: true, force: true }); rmSync(projectB, { recursive: true, force: true }); } NODE run_bounded 60 "$EVIDENCE_DIR/auth-ownership-probe.log" node "$script" "$REPO_ROOT" "$EVIDENCE_DIR/auth-ownership.json" "$SANDBOX_ROOT/auth-ownership" } write_fake_provider() { local script="$SANDBOX_ROOT/fake-provider.mjs" cat >"$script" <<'NODE' import http from "node:http"; import { appendFileSync } from "node:fs"; const marker = process.env.QA_MARKER || "OMO_LSP_PATH_CONTRACT_QA"; const qaScenario = process.env.QA_SCENARIO || "path-contract"; const qaSourceFile = process.env.QA_SOURCE_FILE || "source.ts"; const logFile = process.env.FAKE_PROVIDER_LOG; let callCount = 0; let qaStage = 0; function log(entry) { const line = `${JSON.stringify({ at: new Date().toISOString(), ...entry })}\n`; if (logFile) appendFileSync(logFile, line); } function readBody(request) { return new Promise((resolve, reject) => { const chunks = []; request.on("data", (chunk) => chunks.push(chunk)); request.on("end", () => resolve(Buffer.concat(chunks).toString("utf8"))); request.on("error", reject); }); } function completedUsage() { return { input_tokens: 10, output_tokens: 5, input_tokens_details: { cached_tokens: 0 }, output_tokens_details: { reasoning_tokens: 0 }, }; } function send(response, events) { response.writeHead(200, { "content-type": "text/event-stream; charset=utf-8", "cache-control": "no-cache", connection: "keep-alive", }); for (const event of events) response.write(`data: ${JSON.stringify(event)}\n\n`); response.write("data: [DONE]\n\n"); response.end(); } function textEvents(idNumber, text) { const id = `resp_${idNumber}`; const item = `msg_${idNumber}`; return [ { type: "response.created", response: { id, created_at: Math.floor(Date.now() / 1000), model: "gpt-fake" } }, { type: "response.output_item.added", output_index: 0, item: { type: "message", id: item } }, { type: "response.output_text.delta", item_id: item, output_index: 0, delta: text }, { type: "response.output_item.done", output_index: 0, item: { type: "message", id: item } }, { type: "response.completed", response: { usage: completedUsage() } }, ]; } function toolCallEvents(idNumber, name, argumentsJson) { const id = `resp_${idNumber}`; const item = `fc_${idNumber}`; const callId = `call_lsp_${idNumber}`; return [ { type: "response.created", response: { id, created_at: Math.floor(Date.now() / 1000), model: "gpt-fake" } }, { type: "response.output_item.added", output_index: 0, item: { type: "function_call", id: item, call_id: callId, name, arguments: "" } }, { type: "response.function_call_arguments.delta", item_id: item, output_index: 0, delta: argumentsJson }, { type: "response.output_item.done", output_index: 0, item: { type: "function_call", id: item, call_id: callId, name, arguments: argumentsJson, status: "completed" } }, { type: "response.completed", response: { usage: completedUsage() } }, ]; } function toolNames(body) { if (!Array.isArray(body.tools)) return []; return body.tools.map((tool) => tool?.name ?? tool?.function?.name).filter((name) => typeof name === "string"); } function hasToolResult(input) { return input.includes('"type":"function_call_output"') || input.includes('"type": "function_call_output"') || input.includes('"type":"tool_result"') || input.includes('"type": "tool_result"') || input.includes('"role":"tool"') || input.includes('"role": "tool"'); } function preferredTool(names, wanted) { return names.find((name) => name === wanted) ?? names.find((name) => name.endsWith(wanted)); } const server = http.createServer(async (request, response) => { if (request.method === "GET" && request.url === "/health") { response.writeHead(200, { "content-type": "text/plain" }).end("ok"); return; } if (request.method !== "POST" || !request.url?.includes("/responses")) { response.writeHead(404, { "content-type": "application/json" }).end(JSON.stringify({ error: "not found" })); return; } callCount += 1; const raw = await readBody(request); let body; try { body = JSON.parse(raw); } catch { body = {}; } const input = JSON.stringify(body.input ?? body.messages ?? body); const names = toolNames(body); if (input.includes("Generate a title")) { log({ call: callCount, branch: "title" }); send(response, textEvents(callCount, qaScenario === "rename" ? "LSP rename QA" : qaScenario === "diagnostics-freshness" ? "LSP diagnostics freshness QA" : "LSP path contract QA")); return; } if (input.includes(marker) && qaScenario === "rename" && hasToolResult(input) && qaStage === 1) { const lspTool = preferredTool(names, "lsp_diagnostics"); log({ call: callCount, branch: lspTool ? "tool-call-diagnostics" : "missing-tool", selectedTool: lspTool ?? null, toolNames: names.sort() }); if (!lspTool) { send(response, textEvents(callCount, "OMO_LSP_TOOL_MISSING")); return; } qaStage = 2; send(response, toolCallEvents(callCount, lspTool, JSON.stringify({ filePath: qaSourceFile }))); return; } if (input.includes(marker) && hasToolResult(input)) { log({ call: callCount, branch: "complete" }); send(response, textEvents(callCount, "OMO_LSP_QA_COMPLETE")); return; } if (input.includes(marker)) { const preferred = qaScenario === "rename" ? "lsp_rename" : qaScenario === "diagnostics-freshness" ? "lsp_diagnostics" : "lsp_status"; const lspTool = preferredTool(names, preferred); log({ call: callCount, branch: lspTool ? "tool-call" : "missing-tool", selectedTool: lspTool ?? null, toolNames: names.sort() }); if (!lspTool) { send(response, textEvents(callCount, "OMO_LSP_TOOL_MISSING")); return; } qaStage = qaScenario === "rename" ? 1 : 0; const argumentsJson = qaScenario === "rename" ? JSON.stringify({ filePath: qaSourceFile, line: 1, character: 6, newName: "after" }) : qaScenario === "diagnostics-freshness" ? JSON.stringify({ filePath: qaSourceFile }) : "{}"; send(response, toolCallEvents(callCount, lspTool, argumentsJson)); return; } log({ call: callCount, branch: "default", toolCount: names.length }); send(response, textEvents(callCount, "fake response")); }); server.listen(0, "127.0.0.1", () => { const address = server.address(); const port = typeof address === "object" && address ? address.port : 0; process.stdout.write(`FAKE_LISTENING ${port}\n`); }); process.on("SIGTERM", () => server.close(() => process.exit(0))); process.on("SIGINT", () => server.close(() => process.exit(0))); NODE } start_fake_provider() { local stdout_log="$EVIDENCE_DIR/fake-provider-stdout.log" port="" attempts=0 write_fake_provider FAKE_PROVIDER_LOG="$EVIDENCE_DIR/fake-provider.jsonl" node "$SANDBOX_ROOT/fake-provider.mjs" >"$stdout_log" 2>&1 & FAKE_PID=$! while [ "$attempts" -lt 100 ]; do port="$(awk '/^FAKE_LISTENING / { print $2; exit }' "$stdout_log" 2>/dev/null || true)" [ -n "$port" ] && break kill -0 "$FAKE_PID" 2>/dev/null || { fail "fake provider exited during startup"; return 1; } sleep 0.1 attempts=$((attempts + 1)) done [ -n "$port" ] || { fail "fake provider did not report a port"; return 1; } export FAKE_PROVIDER_PORT="$port" } write_sandbox_config() { local config_dir="$XDG_CONFIG_HOME/opencode" mkdir -p "$config_dir" bun --input-type=module - \ "$config_dir/opencode.jsonc" "$config_dir/oh-my-openagent.json" "$REPO_ROOT" "$FAKE_PROVIDER_PORT" "$SCENARIO" <<'NODE' import { writeFileSync } from "node:fs"; import { join } from "node:path"; import { pathToFileURL } from "node:url"; const [opencodePath, omoPath, repoRoot, port, scenario] = process.argv.slice(2); const permissions = scenario === "rename" ? { lsp_rename: "allow", lsp_diagnostics: "allow" } : scenario === "diagnostics-freshness" ? { lsp_diagnostics: "allow" } : { lsp_status: "allow" }; const opencode = { plugin: [pathToFileURL(join(repoRoot, "packages/omo-opencode/src/index.ts")).href], model: "openai/gpt-fake", provider: { openai: { options: { apiKey: "fake-key", baseURL: `http://127.0.0.1:${port}/v1`, timeout: 30000 }, models: { "gpt-fake": { tool_call: true, limit: { context: 200000, output: 8192 } } }, }, }, permission: permissions, }; const omo = { disabled_mcps: ["websearch", "context7", "grep_app"], disabled_hooks: ["auto-update-checker"], }; writeFileSync(opencodePath, `${JSON.stringify(opencode, null, 2)}\n`); writeFileSync(omoPath, `${JSON.stringify(omo, null, 2)}\n`); NODE } wait_http() { local url="$1" auth="$2" ready_seconds="${3:-$HEALTH_READY_SECONDS}" attempts=0 deadline deadline=$((SECONDS + ready_seconds)) while [ "$attempts" -lt 150 ] && [ "$SECONDS" -lt "$deadline" ]; do curl -sS -o /dev/null \ --connect-timeout "$HEALTH_CURL_CONNECT_TIMEOUT_SECONDS" \ --max-time "$HEALTH_CURL_MAX_TIME_SECONDS" \ -u "$auth" "$url" 2>/dev/null && return 0 kill -0 "$OPENCODE_PID" 2>/dev/null || return 1 sleep 0.2 attempts=$((attempts + 1)) done return 1 } stop_sse_watcher_for_retry() { local pid="$SSE_PID" [ -n "$pid" ] || return 0 if kill -0 "$pid" 2>/dev/null; then stop_verified_pid "$pid" "curl" "SSE watcher" || return 1 else wait "$pid" 2>/dev/null || true fi SSE_PID="" } start_sse_watcher() { local url="$1" auth="$2" encoded_dir="$3" attempt="$4" printf 'attempt=%s start=%s\n' "$attempt" "$(date -u +%Y-%m-%dT%H:%M:%SZ)" >>"$EVIDENCE_DIR/events-attempts.log" curl -sS -N \ --connect-timeout "$HEALTH_CURL_CONNECT_TIMEOUT_SECONDS" \ -u "$auth" "$url/event?directory=$encoded_dir" \ >>"$EVIDENCE_DIR/events.sse" 2>>"$EVIDENCE_DIR/events.stderr.log" & SSE_PID=$! } wait_for_sse_connected() { local url="$1" auth="$2" encoded_dir="$3" deadline attempt=1 attempt_deadline : >"$EVIDENCE_DIR/events.sse" : >"$EVIDENCE_DIR/events.stderr.log" : >"$EVIDENCE_DIR/events-attempts.log" deadline=$((SECONDS + SSE_READY_SECO -
serve-wake-split-probe.sh 38.2 KB
#!/usr/bin/env bash # serve-wake-split-probe.sh # Serve-topology wake runner-split QA harness. # # Proves whether omo's plugin-origin promptAsync (parent-wake bg notifications) # forks a second concurrent LLM runner in opencode serve topology (REPRODUCED) # or routes correctly through the live listener (FIXED). # # Two assertion modes: # --expect reproduced exit 0 if terminal_stops>1 OR child_task_sessions>1 OR mechanism arm true # --expect fixed exit 0 if terminal_stops==1, child_task_sessions==1, # fixed branch counts hold, and route logs show live dispatch # # Usage: # serve-wake-split-probe.sh [--expect reproduced|fixed] [--evidence-dir DIR] # [--self-test] [--help] # # Env: # OMO_SANDBOX_OMO_CONFIG JSON string; when set, deep-merged over the base # agent overrides (env keys win) and written to # $XDG_CONFIG_HOME/opencode/oh-my-openagent.json # before the server starts (flag-disabled control). # FAKE_OPENAI_PORT Force the fake-LLM to bind a specific port # (default: random). Port 1 triggers a startup # failure test used by the self-test failure path. # # Exit codes: # 0 expectation met (or --self-test OK) # 1 expectation NOT met, or internal harness error # 2 usage / bad arguments set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" . "$SCRIPT_DIR/lib/common.sh" # ---- defaults ---------------------------------------------------------------- EXPECT_MODE="" # reproduced | fixed EVIDENCE_DIR="" SELF_TEST=0 FAKE_SERVER_PID="" FAKE_SERVER_PORT="" FAKE_LLM_LOG="" # set after evidence dir is known # ---- argument parsing -------------------------------------------------------- while [ $# -gt 0 ]; do case "$1" in --expect) if [ $# -lt 2 ] || [ "${2#--}" != "$2" ]; then printf 'error: --expect requires reproduced or fixed\n' >&2 exit 2 fi EXPECT_MODE="$2" shift 2 ;; --evidence-dir) if [ $# -lt 2 ] || [ "${2#--}" != "$2" ]; then printf 'error: --evidence-dir requires a directory\n' >&2 exit 2 fi EVIDENCE_DIR="$2" shift 2 ;; --self-test) SELF_TEST=1 shift ;; -h|--help) sed -n '2,26p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//' exit 0 ;; *) printf 'unknown option: %s\n' "$1" >&2 exit 2 ;; esac done if [ "$SELF_TEST" -eq 0 ] && [ -z "$EXPECT_MODE" ]; then EXPECT_MODE="reproduced" fi if [ -n "$EXPECT_MODE" ] && [ "$EXPECT_MODE" != "reproduced" ] && [ "$EXPECT_MODE" != "fixed" ]; then printf 'error: --expect must be reproduced or fixed\n' >&2 exit 2 fi # ---- helpers ----------------------------------------------------------------- swsp_log() { printf '%s\n' "$*" >&2; } swsp_info() { printf '[swsp] %s\n' "$*" >&2; } swsp_tail_log_since_offset() { local offset="$1" local log_path="$2" if [ ! -f "$log_path" ]; then return 0 fi local current_size current_size="$(wc -c <"$log_path" 2>/dev/null | tr -d ' ')" || current_size=0 current_size="${current_size:-0}" if [ "$offset" -gt "$current_size" ] 2>/dev/null; then return 0 else tail -c "+$((offset + 1))" "$log_path" 2>/dev/null || true fi } # Start the fake-LLM server; sets FAKE_SERVER_PID + FAKE_SERVER_PORT. swsp_start_fake_llm() { local log_file="$1" local port_file port_file="$(mktemp -t swsp-port.XXXXXX)" OQA_TMPDIRS+=("$port_file") FAKE_LLM_LOG="$log_file" FAKE_OPENAI_PORT="${FAKE_OPENAI_PORT:-0}" \ bun run --bun "$SCRIPT_DIR/lib/fake-openai-server.mjs" >"$port_file.stdout" 2>&1 & FAKE_SERVER_PID=$! disown "$FAKE_SERVER_PID" 2>/dev/null || true # Poll for the port line (fake-openai listening on <port>) local deadline deadline=$(( $(date +%s) + 10 )) while [ "$(date +%s)" -lt "$deadline" ]; do if grep -q "^fake-openai listening on " "$port_file.stdout" 2>/dev/null; then FAKE_SERVER_PORT="$(grep "^fake-openai listening on " "$port_file.stdout" | head -1 | awk '{print $NF}')" break fi if ! kill -0 "$FAKE_SERVER_PID" 2>/dev/null; then swsp_log "FAIL: fake-openai server process died immediately" cat "$port_file.stdout" >&2 2>/dev/null || true return 1 fi sleep 0.3 done OQA_TMPDIRS+=("$port_file.stdout") if [ -z "$FAKE_SERVER_PORT" ]; then swsp_log "FAIL: fake-openai server did not report port within 10s" cat "$port_file.stdout" >&2 2>/dev/null || true kill "$FAKE_SERVER_PID" 2>/dev/null || true return 1 fi # Verify it's up local hdeadline=0 hdeadline=$(( $(date +%s) + 5 )) while [ "$(date +%s)" -lt "$hdeadline" ]; do if curl -sf "http://127.0.0.1:${FAKE_SERVER_PORT}/health" >/dev/null 2>&1; then swsp_info "fake-openai listening on port $FAKE_SERVER_PORT" return 0 fi sleep 0.2 done swsp_log "FAIL: fake-openai /health did not respond within 5s on port $FAKE_SERVER_PORT" kill "$FAKE_SERVER_PID" 2>/dev/null || true return 1 } swsp_stop_fake_llm() { if [ -n "$FAKE_SERVER_PID" ]; then kill "$FAKE_SERVER_PID" 2>/dev/null || true sleep 0.3 kill -0 "$FAKE_SERVER_PID" 2>/dev/null && kill -9 "$FAKE_SERVER_PID" 2>/dev/null || true FAKE_SERVER_PID="" fi } # Write the sandbox omo config: base agent overrides (explore/librarian -> the # fake provider, required for child model resolution) deep-merged with # OMO_SANDBOX_OMO_CONFIG when set (jq '.[0] * .[1]'; env keys win). # Args: sandbox_config_dir swsp_write_omo_config() { local cfg_dir="$1" local omo_cfg="$cfg_dir/opencode/oh-my-openagent.json" local base='{"agents":{"explore":{"model":"openai/gpt-fake"},"librarian":{"model":"openai/gpt-fake"}}}' mkdir -p "$cfg_dir/opencode" if [ -n "${OMO_SANDBOX_OMO_CONFIG:-}" ]; then if ! printf '%s\n%s\n' "$base" "$OMO_SANDBOX_OMO_CONFIG" | jq -s '.[0] * .[1]' >"$omo_cfg" 2>/dev/null; then swsp_log "FAIL: OMO_SANDBOX_OMO_CONFIG is not valid JSON" return 1 fi swsp_info "wrote merged OMO_SANDBOX_OMO_CONFIG to $omo_cfg" else printf '%s\n' "$base" >"$omo_cfg" swsp_info "wrote agent overrides to $omo_cfg" fi } # Write the sandbox opencode.jsonc with the fake provider + local plugin. # Args: sandbox_config_dir fake_port swsp_write_opencode_config() { local cfg_dir="$1" local fake_port="$2" local repo_root repo_root="$(cd "$SCRIPT_DIR/../../../.." && pwd)" mkdir -p "$cfg_dir/opencode" cat >"$cfg_dir/opencode/opencode.jsonc" <<JSONC { "plugin": ["file://${repo_root}/packages/omo-opencode/src/index.ts"], "model": "openai/gpt-fake", "provider": { "openai": { "options": { "apiKey": "fake-key", "baseURL": "http://127.0.0.1:${fake_port}/v1", "timeout": 30000 }, "models": { "gpt-fake": { "tool_call": true, "limit": { "context": 200000, "output": 8192 } } } } }, "permission": { "bash": "allow", "call_omo_agent": "allow" } } JSONC swsp_info "opencode.jsonc written to $cfg_dir/opencode/opencode.jsonc" } # Poll the sandbox DB for parent assistant step metrics on a message matching a LIKE pattern. # Args: db_path like_pattern timeout_s # Outputs: "<parent_assistant_messages> <parent_tool_call_turns> <terminal_stops> <child_task_sessions>" on stdout swsp_poll_db_metrics() { local db="$1" local like_pat="$2" local timeout_s="${3:-90}" local deadline local metrics_query=" WITH target AS ( SELECT m.id AS user_id, m.session_id FROM message m JOIN part p ON p.message_id = m.id WHERE json_extract(m.data, '\$.role') = 'user' AND json_extract(p.data, '\$.type') = 'text' AND json_extract(p.data, '\$.text') LIKE '${like_pat}' ), counts AS ( SELECT count(a.id) AS parent_assistant_messages, sum(CASE WHEN json_extract(a.data, '\$.finish') = 'tool-calls' THEN 1 ELSE 0 END) AS parent_tool_call_turns, sum(CASE WHEN json_extract(a.data, '\$.finish') = 'stop' THEN 1 ELSE 0 END) AS terminal_stops FROM target t LEFT JOIN message a ON a.session_id = t.session_id AND json_extract(a.data, '\$.parentID') = t.user_id GROUP BY t.user_id ), child_task_sessions AS ( SELECT count(DISTINCT m.session_id) AS child_task_sessions FROM message m JOIN part p ON p.message_id = m.id WHERE json_extract(m.data, '\$.role') = 'user' AND json_extract(p.data, '\$.type') = 'text' AND json_extract(p.data, '\$.text') LIKE '%SPLIT_CHILD_TASK:%' ) SELECT printf('%d %d %d %d', coalesce((SELECT max(parent_assistant_messages) FROM counts), 0), coalesce((SELECT max(parent_tool_call_turns) FROM counts), 0), coalesce((SELECT max(terminal_stops) FROM counts), 0), coalesce((SELECT child_task_sessions FROM child_task_sessions), 0) ); " deadline=$(( $(date +%s) + timeout_s )) while [ "$(date +%s)" -lt "$deadline" ]; do if [ ! -f "$db" ]; then sleep 0.5 continue fi local result result="$(sqlite3 "$db" "$metrics_query" 2>/dev/null)" || true local parent_assistant_messages parent_tool_call_turns terminal_stops child_task_sessions parent_assistant_messages="$(printf '%s' "$result" | awk '{print $1}')" parent_tool_call_turns="$(printf '%s' "$result" | awk '{print $2}')" terminal_stops="$(printf '%s' "$result" | awk '{print $3}')" child_task_sessions="$(printf '%s' "$result" | awk '{print $4}')" # Return once we have at least 1 stop (parent session finished) if [ -n "$terminal_stops" ] && [ "${terminal_stops:-0}" -ge 1 ] 2>/dev/null; then printf '%s %s %s %s' "$parent_assistant_messages" "$parent_tool_call_turns" "$terminal_stops" "$child_task_sessions" return 0 fi sleep 0.5 done # Return whatever we have on timeout local result result="$(sqlite3 "$db" "$metrics_query" 2>/dev/null)" || true printf '%s' "${result:-0 0 0 0}" } # Wait until a session is no longer in the server's active status map. # Args: server_url pass session_id timeout_s swsp_wait_session_idle() { local url="$1" pass="$2" ses_id="$3" timeout_s="${4:-120}" local deadline deadline=$(( $(date +%s) + timeout_s )) while [ "$(date +%s)" -lt "$deadline" ]; do local status_json status_json="$(curl -sf -u "opencode:${pass}" "${url}/session/status" 2>/dev/null)" || true if [ -z "$status_json" ] || ! printf '%s' "$status_json" | grep -q "$ses_id" 2>/dev/null; then return 0 fi sleep 0.5 done swsp_log "WARNING: session $ses_id did not go idle within ${timeout_s}s; proceeding with current DB state" return 0 } # Count plugin_inits from the omo log since a byte offset, filtered to sandbox dir. # Args: log_offset sandbox_dir swsp_count_plugin_inits() { local offset="$1" local sandbox_dir="$2" local log_path="${TMPDIR:-/tmp}/oh-my-opencode.log" if [ ! -f "$log_path" ]; then printf '0' return 0 fi swsp_tail_log_since_offset "$offset" "$log_path" \ | grep "ENTRY - plugin loading" \ | awk -v sandbox_dir="$sandbox_dir" 'index($0, sandbox_dir) { count += 1 } END { print count + 0 }' } # Detect WAKE_DISPATCHED_DURING_PARENT_TURN: # true iff the omo log (since offset) contains a [prompt-async-gate] promptAsync dispatching # line with source containing "parent-wake", AND that line's timestamp is within the # parent-hold window (between branch=parent-hold line and next non-wake completion). # Args: log_offset fake_llm_log sandbox_dir swsp_detect_wake_during_parent() { local offset="$1" local fake_log="$2" local sandbox_dir="$3" local omo_log="${TMPDIR:-/tmp}/oh-my-opencode.log" # Find parent-hold timestamp from fake-llm.log local hold_ts hold_ts="$(grep "branch=parent-hold" "$fake_log" 2>/dev/null | head -1 | grep -o '\[.*\]' | tr -d '[]')" || true if [ -z "$hold_ts" ]; then # parent-hold never fired — cannot determine printf 'false' return 0 fi # Check for gate dispatch log line with parent-wake source since offset local dispatch_line dispatch_line="$(swsp_tail_log_since_offset "$offset" "$omo_log" \ | grep "promptAsync dispatching" \ | grep -i "parent-wake\|background-agent-parent-wake" \ | head -1)" || true if [ -z "$dispatch_line" ]; then printf 'false' return 0 fi # Extract timestamp from dispatch line (ISO 8601 in brackets or as prefix) local dispatch_ts dispatch_ts="$(printf '%s' "$dispatch_line" | grep -o '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]:[0-9][0-9]' | head -1)" || true if [ -z "$dispatch_ts" ]; then # Can't compare timestamps — fall back to presence check printf 'true' return 0 fi # Simple lexicographic timestamp compare (ISO 8601 sorts correctly) # The hold_ts is the start of the hold window; if dispatch happened after it, signal is true if [ "$dispatch_ts" \> "$hold_ts" ] || [ "$dispatch_ts" = "$hold_ts" ]; then printf 'true' else printf 'false' fi } swsp_has_session_live_dispatch() { local route_prov="$1" local session_id="$2" [ -n "$session_id" ] || return 1 printf '%s\n' "$route_prov" \ | grep -F "dispatch via live listener" \ | grep -F "\"sessionID\":\"${session_id}\"" >/dev/null 2>&1 } swsp_is_nonnegative_int() { case "${1:-}" in ''|*[!0-9]*) return 1 ;; *) return 0 ;; esac } swsp_fixed_topology_observed() { local parent_assistant_messages="$1" local parent_tool_call_turns="$2" local terminal_stops="$3" local child_task_sessions="$4" local parent_tool_call_branches="$5" local parent_hold_branches="$6" local child_branches="$7" local wake_branches="$8" local default_branches="$9" local has_live_dispatch="${10}" swsp_is_nonnegative_int "$parent_assistant_messages" || return 1 swsp_is_nonnegative_int "$parent_tool_call_turns" || return 1 swsp_is_nonnegative_int "$terminal_stops" || return 1 swsp_is_nonnegative_int "$child_task_sessions" || return 1 swsp_is_nonnegative_int "$parent_tool_call_branches" || return 1 swsp_is_nonnegative_int "$parent_hold_branches" || return 1 swsp_is_nonnegative_int "$child_branches" || return 1 swsp_is_nonnegative_int "$wake_branches" || return 1 swsp_is_nonnegative_int "$default_branches" || return 1 [ "$has_live_dispatch" = "true" ] || return 1 if [ "${terminal_stops:-0}" -ne 1 ] \ || [ "${child_task_sessions:-0}" -ne 1 ] \ || [ "${parent_tool_call_turns:-0}" -ne 2 ] \ || [ "${parent_assistant_messages:-0}" -ne 3 ] \ || [ "${parent_tool_call_branches:-0}" -ne 1 ] \ || [ "${parent_hold_branches:-0}" -ne 1 ] \ || [ "${child_branches:-0}" -ne 1 ] \ || [ "${default_branches:-0}" -lt 1 ] \ || [ "${wake_branches:-0}" -ne 0 ] 2>/dev/null; then return 1 fi return 0 } swsp_collect_route_provenance() { local offset="$1" local log_path="$2" local project_dir="$3" local session_id="$4" local output_file="$5" local all_output_file="$6" local timeout_s="${7:-0}" local deadline deadline=$(( $(date +%s) + timeout_s )) while :; do local route_prov_all="" route_prov="" if [ -f "$log_path" ]; then route_prov_all="$(swsp_tail_log_since_offset "$offset" "$log_path" \ | grep -E "live-server-route" || true)" route_prov="$(printf '%s\n' "$route_prov_all" \ | awk -v dir="$project_dir" -v sid="\"sessionID\":\"${session_id}\"" 'index($0, dir) || index($0, sid)' || true)" fi printf '%s' "$route_prov" >"$all_output_file" printf '%s' "$route_prov" >"$output_file" if swsp_has_session_live_dispatch "$route_prov" "$session_id"; then return 0 fi if [ "$timeout_s" -le 0 ] || [ "$(date +%s)" -ge "$deadline" ]; then return 1 fi sleep 0.25 done } # Verify branch-count guard: all required branches fired. # Returns 0 if OK, 1 if any required branch missing (also sets RESULT=HARNESS_ERROR). swsp_check_branch_counts() { local fake_log="$1" local mode="${2:-}" local ptc pc cc wc ptc="$(grep -c "branch=parent-tool-call" "$fake_log" 2>/dev/null | tr -d '[:space:]')"; ptc="${ptc:-0}" pc="$(grep -c "branch=parent-hold" "$fake_log" 2>/dev/null | tr -d '[:space:]')"; pc="${pc:-0}" cc="$(grep -c "branch=child" "$fake_log" 2>/dev/null | tr -d '[:space:]')"; cc="${cc:-0}" wc="$(grep -c "branch=wake" "$fake_log" 2>/dev/null | tr -d '[:space:]')"; wc="${wc:-0}" ptc="${ptc%%[!0-9]*}"; pc="${pc%%[!0-9]*}"; cc="${cc%%[!0-9]*}"; wc="${wc%%[!0-9]*}" ptc="${ptc:-0}"; pc="${pc:-0}"; cc="${cc:-0}"; wc="${wc:-0}" swsp_info "branch counts: parent-tool-call=$ptc parent-hold=$pc child=$cc wake=$wc" if [ "$ptc" -lt 1 ] || [ "$pc" -lt 1 ] || [ "$cc" -lt 1 ]; then printf 'RESULT=HARNESS_ERROR branch_counts parent-tool-call=%s parent-hold=%s child=%s wake=%s\n' \ "$ptc" "$pc" "$cc" "$wc" return 1 fi return 0 } # ---- self-test --------------------------------------------------------------- swsp_self_test() { swsp_info "running self-test..." local fails=0 # Deps oqa_require opencode sqlite3 curl jq bun || { swsp_log "FAIL: missing dependencies"; fails=$((fails+1)); } # Start fake-LLM local st_log st_log="$(mktemp -t swsp-st-llm.XXXXXX)" OQA_TMPDIRS+=("$st_log") if ! swsp_start_fake_llm "$st_log"; then swsp_log "FAIL: fake-LLM did not start (port=${FAKE_OPENAI_PORT:-dynamic})" fails=$((fails+1)) else swsp_info "fake-LLM started on port $FAKE_SERVER_PORT" # Health check if curl -sf "http://127.0.0.1:${FAKE_SERVER_PORT}/health" >/dev/null 2>&1; then swsp_info "PASS: fake-LLM /health 200" else swsp_log "FAIL: fake-LLM /health did not return 200" fails=$((fails+1)) fi fi local missing_expect_out missing_expect_err missing_evidence_out missing_evidence_err missing_expect_out="$(mktemp -t swsp-missing-expect-out.XXXXXX)" missing_expect_err="$(mktemp -t swsp-missing-expect-err.XXXXXX)" missing_evidence_out="$(mktemp -t swsp-missing-evidence-out.XXXXXX)" missing_evidence_err="$(mktemp -t swsp-missing-evidence-err.XXXXXX)" OQA_TMPDIRS+=("$missing_expect_out" "$missing_expect_err" "$missing_evidence_out" "$missing_evidence_err") if bash "${BASH_SOURCE[0]}" --expect >"$missing_expect_out" 2>"$missing_expect_err"; then swsp_log "FAIL: missing --expect operand unexpectedly succeeded" fails=$((fails+1)) elif grep -q "error: --expect requires reproduced or fixed" "$missing_expect_err"; then swsp_info "PASS: missing --expect operand fails with usage error" else swsp_log "FAIL: missing --expect operand did not emit usage error" fails=$((fails+1)) fi if bash "${BASH_SOURCE[0]}" --evidence-dir --self-test >"$missing_evidence_out" 2>"$missing_evidence_err"; then swsp_log "FAIL: missing --evidence-dir operand unexpectedly succeeded" fails=$((fails+1)) elif grep -q "error: --evidence-dir requires a directory" "$missing_evidence_err"; then swsp_info "PASS: missing --evidence-dir operand fails with usage error" else swsp_log "FAIL: missing --evidence-dir operand did not emit usage error" fails=$((fails+1)) fi # Sandbox + opencode serve if ! oqa_start_server; then swsp_log "FAIL: opencode serve did not start" swsp_stop_fake_llm fails=$((fails+1)) else swsp_info "PASS: opencode serve started at $OQA_SERVER_URL" # /global/health check local health_code health_code="$(curl -so /dev/null -w "%{http_code}" -u "opencode:${OQA_SERVER_PASS}" \ "${OQA_SERVER_URL}/global/health" 2>/dev/null)" || true if [ "$health_code" = "200" ]; then swsp_info "PASS: /global/health 200" else swsp_log "FAIL: /global/health returned $health_code" fails=$((fails+1)) fi # OMO_SANDBOX_OMO_CONFIG env contract assertion: merge keeps base overrides local omo_cfg_path="$XDG_CONFIG_HOME/opencode/oh-my-openagent.json" OMO_SANDBOX_OMO_CONFIG='{"_probe":true}' swsp_write_omo_config "$XDG_CONFIG_HOME" local probe_val explore_model probe_val="$(jq -r '._probe' "$omo_cfg_path" 2>/dev/null)" explore_model="$(jq -r '.agents.explore.model' "$omo_cfg_path" 2>/dev/null)" if [ "$probe_val" = "true" ] && [ "$explore_model" = "openai/gpt-fake" ]; then swsp_info "PASS: OMO_SANDBOX_OMO_CONFIG merge assertion (env key + base overrides both present)" else swsp_log "FAIL: omo config merge wrong: _probe='$probe_val' explore_model='$explore_model'" fails=$((fails+1)) fi fi local route_fixture route_fixture='[2026-06-19T00:00:00.000Z] [live-server-route] dispatch via live listener {"sessionID":"ses_other","source":"background-agent-parent-wake"} [2026-06-19T00:00:01.000Z] [live-server-route] dispatch via live listener {"sessionID":"ses_probe","source":"background-agent-parent-wake"}' if swsp_has_session_live_dispatch "$route_fixture" "ses_probe" \ && ! swsp_has_session_live_dispatch "$route_fixture" "ses_missing"; then swsp_info "PASS: live dispatch detection is scoped to the probe session" else swsp_log "FAIL: live dispatch detection accepted an unrelated session" fails=$((fails+1)) fi local branch_log branch_log="$(mktemp -t swsp-branch-log.XXXXXX)" OQA_TMPDIRS+=("$branch_log") { printf '[2026-06-19T00:00:00.000Z] branch=parent-tool-call\n' printf '[2026-06-19T00:00:01.000Z] branch=parent-hold\n' printf '[2026-06-19T00:00:02.000Z] branch=child\n' } >"$branch_log" if swsp_check_branch_counts "$branch_log" reproduced >/dev/null 2>&1; then swsp_info "PASS: reproduced branch guard accepts mechanism-only evidence" else swsp_log "FAIL: reproduced branch guard still requires wake branch" fails=$((fails+1)) fi if swsp_fixed_topology_observed 3 2 1 1 1 1 1 0 1 true; then swsp_info "PASS: fixed topology accepts scoped live dispatch plus deterministic DB/provider evidence" else swsp_log "FAIL: fixed topology rejected scoped live dispatch plus deterministic DB/provider evidence" fails=$((fails+1)) fi if swsp_fixed_topology_observed 3 2 1 1 1 1 1 0 1 false; then swsp_log "FAIL: fixed topology accepted missing scoped live dispatch" fails=$((fails+1)) else swsp_info "PASS: fixed topology rejects missing scoped live dispatch" fi if swsp_fixed_topology_observed bad 2 1 1 1 1 1 0 1 true; then swsp_log "FAIL: fixed topology accepted malformed numeric evidence" fails=$((fails+1)) else swsp_info "PASS: fixed topology rejects malformed numeric evidence" fi if swsp_fixed_topology_observed 3 2 2 1 1 1 1 0 1 true; then swsp_log "FAIL: fixed topology accepted duplicate terminal stop" fails=$((fails+1)) else swsp_info "PASS: fixed topology rejects duplicate terminal stop" fi local stale_log stale_scoped stale_all stale_log="$(mktemp -t swsp-stale-log.XXXXXX)" stale_scoped="$(mktemp -t swsp-stale-scoped.XXXXXX)" stale_all="$(mktemp -t swsp-stale-all.XXXXXX)" OQA_TMPDIRS+=("$stale_log" "$stale_scoped" "$stale_all") printf '[2026-06-19T00:00:00.000Z] [live-server-route] dispatch via live listener {"sessionID":"ses_unrelated","source":"background-agent-parent-wake"}\n' >"$stale_log" swsp_collect_route_provenance 999999 "$stale_log" "/probe" "ses_probe" "$stale_scoped" "$stale_all" 0 || true if [ ! -s "$stale_scoped" ] && [ ! -s "$stale_all" ]; then swsp_info "PASS: stale log offset does not persist unrelated route provenance" else swsp_log "FAIL: stale log offset persisted unrelated route provenance" fails=$((fails+1)) fi local scoped_log scoped_out scoped_all scoped_log="$(mktemp -t swsp-scoped-log.XXXXXX)" scoped_out="$(mktemp -t swsp-scoped-out.XXXXXX)" scoped_all="$(mktemp -t swsp-scoped-all.XXXXXX)" OQA_TMPDIRS+=("$scoped_log" "$scoped_out" "$scoped_all") { printf '[2026-06-19T00:00:00.000Z] [live-server-route] dispatch via live listener {"sessionID":"ses_other","source":"background-agent-parent-wake"}\n' printf '[2026-06-19T00:00:01.000Z] [live-server-route] dispatch via live listener {"sessionID":"ses_probe","source":"background-agent-parent-wake"}\n' } >"$scoped_log" swsp_collect_route_provenance 0 "$scoped_log" "/probe" "ses_probe" "$scoped_out" "$scoped_all" 0 || true if grep -q "ses_probe" "$scoped_all" && ! grep -q "ses_other" "$scoped_all"; then swsp_info "PASS: route provenance artifact excludes unrelated sessions" else swsp_log "FAIL: route provenance artifact included unrelated sessions" fails=$((fails+1)) fi local route_wait_log route_wait_scoped route_wait_all route_wait_log="$(mktemp -t swsp-route-wait-log.XXXXXX)" route_wait_scoped="$(mktemp -t swsp-route-wait-scoped.XXXXXX)" route_wait_all="$(mktemp -t swsp-route-wait-all.XXXXXX)" OQA_TMPDIRS+=("$route_wait_log" "$route_wait_scoped" "$route_wait_all") printf '[2026-06-19T00:00:00.000Z] [live-server-route] registered {"directory":"/probe","hasServerUrl":true}\n' >"$route_wait_log" ( sleep 0.5 printf '[2026-06-19T00:00:01.000Z] [live-server-route] dispatch via live listener {"sessionID":"ses_wait","source":"background-agent-parent-wake"}\n' >>"$route_wait_log" ) & local route_wait_pid=$! if swsp_collect_route_provenance 0 "$route_wait_log" "/probe" "ses_wait" "$route_wait_scoped" "$route_wait_all" 3; then swsp_info "PASS: route provenance waits for delayed session dispatch" else swsp_log "FAIL: route provenance did not wait for delayed session dispatch" fails=$((fails+1)) fi wait "$route_wait_pid" 2>/dev/null || true local st_fake_pid="$FAKE_SERVER_PID" swsp_stop_fake_llm if [ -n "$st_fake_pid" ] && ! kill -0 "$st_fake_pid" 2>/dev/null; then swsp_info "PASS: fake-openai server process stopped" else swsp_log "FAIL: fake-openai server process still running" fails=$((fails+1)) fi if [ "$fails" -eq 0 ]; then printf 'SELF-TEST OK\n' return 0 fi printf 'SELF-TEST FAILED (%d failure(s))\n' "$fails" >&2 return 1 } # ---- main probe run ---------------------------------------------------------- swsp_run_probe() { local evidence_dir="${EVIDENCE_DIR:-$(mktemp -d -t swsp-evidence.XXXXXX)}" mkdir -p "$evidence_dir" OQA_TMPDIRS+=("$evidence_dir") 2>/dev/null || true # only auto-clean if we created it # Override: if caller gave --evidence-dir, don't delete it if [ -n "$EVIDENCE_DIR" ]; then # Remove from cleanup list (last element we added) unset 'OQA_TMPDIRS[${#OQA_TMPDIRS[@]}-1]' 2>/dev/null || true fi swsp_info "evidence dir: $evidence_dir" local fake_llm_log="$evidence_dir/fake-llm.log" local harness_log="$evidence_dir/harness.log" local serve_stdout="$evidence_dir/opencode-serve.stdout" local serve_stderr="$evidence_dir/opencode-serve.stderr" # Step 1: Record real-DB session count (read-only) local real_db_path real_db_count_before real_db_path="$(opencode db path 2>/dev/null | head -1 || echo "")" if [ -n "$real_db_path" ] && [ -f "$real_db_path" ]; then real_db_count_before="$(sqlite3 "$real_db_path" 'SELECT count(*) FROM session' 2>/dev/null || echo "0")" else real_db_count_before="0" real_db_path="(not found)" fi swsp_info "real DB session count before: $real_db_count_before" printf 'real_db=%s before=%s\n' "$real_db_path" "$real_db_count_before" >"$evidence_dir/isolation-receipt.txt" # Capture omo log byte offset local omo_log="${TMPDIR:-/tmp}/oh-my-opencode.log" local omo_log_offset if [ -f "$omo_log" ]; then omo_log_offset="$(wc -c <"$omo_log" 2>/dev/null | tr -d ' ')" || omo_log_offset=0 else omo_log_offset=0 fi # Step 2: Start fake-openai server swsp_info "starting fake-openai server..." if ! swsp_start_fake_llm "$fake_llm_log"; then swsp_log "HARNESS_ERROR: fake-openai server failed to start" printf 'RESULT=HARNESS_ERROR fake_llm_start_failed\n' | tee -a "$harness_log" return 1 fi swsp_info "fake-openai on port $FAKE_SERVER_PORT" printf 'fake_llm_port=%s\n' "$FAKE_SERVER_PORT" >>"$evidence_dir/isolation-receipt.txt" # Step 3: Create isolated sandbox and write opencode.jsonc # oqa_mk_isolated_xdg sets XDG_CONFIG_HOME, XDG_DATA_HOME, etc. oqa_mk_isolated_xdg swsp_info "sandbox: $OQA_XDG_ROOT" swsp_write_omo_config "$XDG_CONFIG_HOME" swsp_write_opencode_config "$XDG_CONFIG_HOME" "$FAKE_SERVER_PORT" local sandbox_db="$XDG_DATA_HOME/opencode/opencode.db" # Step 4: Start opencode serve (using oqa_start_server internals but with our config) # oqa_start_server includes another oqa_mk_isolated_xdg call which would reset XDG vars. # Instead, start server directly using the already-set XDG vars. swsp_info "starting opencode serve..." local port pass port="$(oqa_free_port)" pass="oqa-${RANDOM}${RANDOM}" OPENCODE_SERVER_PASSWORD="$pass" opencode serve --port "$port" --hostname 127.0.0.1 \ >"$serve_stdout" 2>"$serve_stderr" & OQA_SERVER_PID=$! disown "$OQA_SERVER_PID" 2>/dev/null || true export OQA_SERVER_PORT="$port" export OQA_SERVER_PASS="$pass" export OQA_SERVER_URL="http://127.0.0.1:$port" if ! oqa_wait_http "$OQA_SERVER_URL/global/health" "opencode:$pass" 30; then swsp_log "HARNESS_ERROR: opencode serve failed to start" cat "$serve_stderr" >&2 2>/dev/null || true printf 'RESULT=HARNESS_ERROR opencode_serve_start_failed\n' | tee -a "$harness_log" swsp_stop_fake_llm return 1 fi swsp_info "opencode serve ready at $OQA_SERVER_URL" # Encode the working directory for use in URL local enc_dir enc_dir="$(python3 -c 'import urllib.parse,sys;print(urllib.parse.quote(sys.argv[1],safe=""))' "$OQA_PROJ" 2>/dev/null \ || printf '%s' "$OQA_PROJ" | sed 's|/|%2F|g')" # Step 5: Create a session swsp_info "creating session..." local ses_response ses_id ses_response="$(curl -sS -u "opencode:${pass}" \ -X POST "${OQA_SERVER_URL}/session?directory=${enc_dir}" \ -H 'content-type: application/json' \ -d '{"title":"wake split probe"}' 2>/dev/null)" || ses_response="" ses_id="$(printf '%s' "$ses_response" | jq -r '.id // .sessionID // empty' 2>/dev/null)" || ses_id="" if [ -z "$ses_id" ]; then swsp_log "HARNESS_ERROR: could not create session (response: $ses_response)" printf 'RESULT=HARNESS_ERROR session_create_failed\n' | tee -a "$harness_log" swsp_stop_fake_llm return 1 fi swsp_info "session: $ses_id" printf 'session_id=%s\n' "$ses_id" >>"$evidence_dir/isolation-receipt.txt" # Step 6: Send the split probe prompt swsp_info "sending split probe prompt..." local prompt_response prompt_response="$(curl -sS -u "opencode:${pass}" \ -X POST "${OQA_SERVER_URL}/session/${ses_id}/prompt_async?directory=${enc_dir}" \ -H 'content-type: application/json' \ -d '{"parts":[{"type":"text","text":"Run the split probe: call call_omo_agent exactly once as instructed, then run the bash hold command."}]}' \ 2>/dev/null)" || prompt_response="" swsp_info "prompt_async response: $prompt_response" swsp_info "polling DB for wake-split metrics (up to 120s)..." local metrics metrics="$(swsp_poll_db_metrics "$sandbox_db" '%Run the split probe:%' 120)" local parent_assistant_messages parent_tool_call_turns terminal_stops child_task_sessions parent_assistant_messages="$(printf '%s' "$metrics" | awk '{print $1}')" parent_tool_call_turns="$(printf '%s' "$metrics" | awk '{print $2}')" terminal_stops="$(printf '%s' "$metrics" | awk '{print $3}')" child_task_sessions="$(printf '%s' "$metrics" | awk '{print $4}')" parent_assistant_messages="${parent_assistant_messages:-0}" parent_tool_call_turns="${parent_tool_call_turns:-0}" terminal_stops="${terminal_stops:-0}" child_task_sessions="${child_task_sessions:-0}" swsp_info "DB metrics: parent_assistant_messages=$parent_assistant_messages parent_tool_call_turns=$parent_tool_call_turns terminal_stops=$terminal_stops child_task_sessions=$child_task_sessions" # Wait for parent session to go idle swsp_info "waiting for parent session to go idle..." swsp_wait_session_idle "$OQA_SERVER_URL" "$pass" "$ses_id" 60 # Re-read metrics after idle metrics="$(swsp_poll_db_metrics "$sandbox_db" '%Run the split probe:%' 10)" parent_assistant_messages="$(printf '%s' "$metrics" | awk '{print $1}')" parent_tool_call_turns="$(printf '%s' "$metrics" | awk '{print $2}')" terminal_stops="$(printf '%s' "$metrics" | awk '{print $3}')" child_task_sessions="$(printf '%s' "$metrics" | awk '{print $4}')" parent_assistant_messages="${parent_assistant_messages:-0}" parent_tool_call_turns="${parent_tool_call_turns:-0}" terminal_stops="${terminal_stops:-0}" child_task_sessions="${child_task_sessions:-0}" swsp_info "final DB metrics: parent_assistant_messages=$parent_assistant_messages parent_tool_call_turns=$parent_tool_call_turns terminal_stops=$terminal_stops child_task_sessions=$child_task_sessions" printf 'parent_assistant_messages=%s parent_tool_call_turns=%s terminal_stops=%s child_task_sessions=%s\n' \ "$parent_assistant_messages" "$parent_tool_call_turns" "$terminal_stops" "$child_task_sessions" \ >"$evidence_dir/marker-metrics.txt" # Step 8: Plugin-init count local plugin_inits plugin_inits="$(swsp_count_plugin_inits "$omo_log_offset" "$OQA_PROJ")" plugin_inits="${plugin_inits:-0}" swsp_info "plugin_inits: $plugin_inits" printf '%s\n' "$plugin_inits" >"$evidence_dir/plugin-init-count.txt" # Step 9: Route provenance local route_prov="" swsp_collect_route_provenance \ "$omo_log_offset" \ "$omo_log" \ "$OQA_PROJ" \ "$ses_id" \ "$evidence_dir/route-provenance.log" \ "$evidence_dir/route-provenance-all.log" \ 10 || true route_prov="$(cat "$evidence_dir/route-provenance.log" 2>/dev/null || true)" swsp_info "route-provenance lines: $(printf '%s' "$route_prov" | wc -l | tr -d ' ')" # WAKE_DISPATCHED_DURING_PARENT_TURN mechanism signal local wake_during_parent wake_during_parent="$(swsp_detect_wake_during_parent "$omo_log_offset" "$fake_llm_log" "$OQA_PROJ")" swsp_info "WAKE_DISPATCHED_DURING_PARENT_TURN=$wake_during_parent" local real_db_count_after="" if [ -n "$real_db_path" ] && [ "$real_db_path" != "(not found)" ] && [ -f "$real_db_path" ]; then real_db_count_after="$(sqlite3 "$real_db_path" 'SELECT count(*) FROM session' 2>/dev/null || echo "0")" else real_db_count_after="$real_db_count_before" fi printf 'after=%s unchanged=%s\n' \ "$real_db_count_after" \ "$([ "$real_db_count_after" = "$real_db_count_before" ] && echo yes || echo NO)" \ >>"$evidence_dir/isolation-receipt.txt" swsp_info "isolation: real DB before=$real_db_count_before after=$real_db_count_after" sqlite3 "$sandbox_db" ".backup '$evidence_dir/sandbox-opencode.db'" 2>/dev/null || true # Step 10: Branch-count guard if ! swsp_check_branch_counts "$fake_llm_log" "$EXPECT_MODE" >&2; then # Branch counts not met — HARNESS_ERROR local ptc pc cc wc ptc="$(grep -c "branch=parent-tool-call" "$fake_llm_log" 2>/dev/null || true)"; ptc="${ptc:-0}" pc="$(grep -c "branch=parent-hold" "$fake_llm_log" 2>/dev/null || true)"; pc="${pc:-0}" cc="$(grep -c "branch=child" "$fake_llm_log" 2>/dev/null || true)"; cc="${cc:-0}" wc="$(grep -c "branch=wake" "$fake_llm_log" 2>/dev/null || true)"; wc="${wc:-0}" local verdict_line verdict_line="RESULT=HARNESS_ERROR parent_assistant_messages=${parent_assistant_messages} parent_tool_call_turns=${parent_tool_call_turns} terminal_stops=${terminal_stops} child_task_sessions=${child_task_sessions} plugin_inits=${plugin_inits} WAKE_DISPATCHED_DURING_PARENT_TURN=${wake_during_parent} branch_counts=parent-tool-call:${ptc},parent-hold:${pc},child:${cc},wake:${wc}" printf '%s\n' "$verdict_line" | tee -a "$harness_log" swsp_stop_fake_llm return 1 fi # Step 11: Determine verdict local result="INCONCLUSIVE" local exit_code=1 local ptc pc cc wc ptc="$(grep -c "branch=parent-tool-call" "$fake_llm_log" 2>/dev/null || true)"; ptc="${ptc:-0}" pc="$(grep -c "branch=parent-hold" "$fake_llm_log" 2>/dev/null || true)"; pc="${pc:-0}" cc="$(grep -c "branch=child" "$fake_llm_log" 2>/dev/null || true)"; cc="${cc:-0}" wc="$(grep -c "branch=wake" "$fake_llm_log" 2>/dev/null || true)"; wc="${wc:-0}" local dc dc="$(grep -c "branch=default" "$fake_llm_log" 2>/dev/null || true)"; dc="${dc:-0}" if [ "${terminal_stops:-0}" -gt 1 ] || [ "${child_task_sessions:-0}" -gt 1 ] 2>/dev/null; then result="REPRODUCED" fi # Arm 2: Mechanism signal (wake dispatched during parent turn + in-process path) # in-process path: no live-server-route dispatch line for this wake local has_live_dispatch=false if swsp_has_session_live_dispatch "$route_prov" "$ses_id"; then has_live_dispatch=true fi if [ "$wake_during_parent" = "true" ] && [ "$has_live_dispatch" = "false" ]; then result="REPRODUCED" fi if [ "$result" = "INCONCLUSIVE" ] \ && swsp_fixed_topology_observed \ "$parent_assistant_messages" \ "$parent_tool_call_turns" \ "$terminal_stops" \ "$child_task_sessions" \ "$ptc" \ "$pc" \ "$cc" \ "$wc" \ "$dc" \ "$has_live_dispatch"; then result="FIXED" fi local verdict_line verdict_line="RESULT=${result} parent_assistant_messages=${parent_assistant_messages} parent_tool_call_turns=${parent_tool_call_turns} terminal_stops=${terminal_stops} child_task_sessions=${child_task_sessions} plugin_inits=${plugin_inits} WAKE_DISPATCHED_DURING_PARENT_TURN=${wake_during_parent} route_live_dispatch=${has_live_dispatch} branch_counts=parent-tool-call:${ptc},parent-hold:${pc},child:${cc},wake:${wc},default:${dc}" printf '%s\n' "$verdict_line" | tee -a "$harness_log" # Determine exit code based on expected mode if [ -n "$EXPECT_MODE" ]; then if [ "$EXPECT_MODE" = "reproduced" ] && [ "$result" = "REPRODUCED" ]; then exit_code=0 elif [ "$EXPECT_MODE" = "fixed" ] && [ "$result" = "FIXED" ]; then exit_code=0 else exit_code=1 fi else exit_code=0 # no expectation: just report fi # Cleanup receipt local stopped_fake_pid="$FAKE_SERVER_PID" swsp_stop_fake_llm printf 'fake_llm=stopped opencode_serve=stopping\n' >"$evidence_dir/cleanup-receipt.txt" if [ -n "$stopped_fake_pid" ] && kill -0 "$stopped_fake_pid" 2>/dev/null; then swsp_log "WARNING: fake-openai server process $stopped_fake_pid still running after cleanup" printf 'fake_llm_pid_alive=yes pid=%s\n' "$stopped_fake_pid" >>"$evidence_dir/cleanup-receipt.txt" else printf 'fake_llm_pid_alive=no pid=%s\n' "$stopped_fake_pid" >>"$evidence_dir/cleanup-receipt.txt" fi return "$exit_code" } # ---- dispatch ---------------------------------------------------------------- if [ "$SELF_TEST" -eq 1 ]; then swsp_self_test exit $? fi swsp_run_probe exit $? -
server-smoke.sh 2.1 KB
#!/usr/bin/env bash # server-smoke.sh - boot an ISOLATED opencode HTTP server and verify the core # API surface end to end. Uses an isolated XDG sandbox + a random password, so # it never touches the real ~/.local/share/opencode DB, and tears the server # down on exit. # # Checks: # 1. GET /global/health -> {"healthy":true,"version":...} # 2. GET /doc -> OpenAPI spec with >=100 paths # 3. GET /session (no credentials) -> HTTP 401 (auth is enforced) # # Usage: # server-smoke.sh # run the smoke test # server-smoke.sh --self-test # same thing (alias for the QA sweep) SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" . "$SCRIPT_DIR/lib/common.sh" oqa_server_smoke() { oqa_require opencode curl jq || return 1 if ! oqa_start_server; then oqa_log "FAIL: server did not become ready"; return 1 fi local url="$OQA_SERVER_URL" auth="opencode:$OQA_SERVER_PASS" fails=0 local healthy version healthy="$(curl -s -u "$auth" "$url/global/health" | jq -r '.healthy // false')" version="$(curl -s -u "$auth" "$url/global/health" | jq -r '.version // "?"')" if [ "$healthy" = "true" ]; then oqa_pass "GET /global/health healthy=true version=$version ($url)" else oqa_log "FAIL: /global/health healthy=$healthy"; fails=$((fails+1)) fi local npaths npaths="$(curl -s -u "$auth" "$url/doc" | jq '.paths | length' 2>/dev/null)" if [ "${npaths:-0}" -ge 100 ]; then oqa_pass "GET /doc lists $npaths documented paths (>=100)" else oqa_log "FAIL: /doc path count=$npaths"; fails=$((fails+1)) fi local code code="$(curl -s -o /dev/null -w '%{http_code}' "$url/session?directory=$OQA_PROJ")" if [ "$code" = "401" ]; then oqa_pass "unauthenticated GET /session rejected with HTTP 401" else oqa_log "FAIL: unauthenticated /session returned $code (expected 401)"; fails=$((fails+1)) fi if [ "$fails" -eq 0 ]; then oqa_pass "server-smoke" return 0 fi oqa_log "server-smoke had $fails failure(s)"; return 1 } case "${1:-}" in -h|--help) sed -n '2,18p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//' exit 0 ;; *) oqa_server_smoke; exit $? ;; esac -
sse-hook-probe.sh 4.4 KB
#!/usr/bin/env bash # sse-hook-probe.sh - QA opencode's event stream (the plumbing behind hooks). # # opencode publishes lifecycle events over Server-Sent Events at GET /event # (per-instance) and GET /global/event. Plugins observe the same events via the # `event` hook, so confirming an event on the wire is how you prove a hook # would have fired. # # Two modes: # (default / --self-test) Spawn an ISOLATED server and assert the stream # opens with a `server.connected` event. No real DB # is touched. This proves the SSE plumbing works. # --attach <url> Watch an ALREADY-RUNNING server's /event stream for # a specific event type (default: server.connected). # Use this against your real server to verify a hook # or action. Pair it with a prompt in another shell: # curl -X POST -u opencode:$PASS \ # -H 'Content-Type: application/json' \ # -d '{"parts":[{"type":"text","text":"hi"}]}' \ # "<url>/session/<ses_id>/prompt_async?directory=<dir>" # then watch for e.g. message.part.updated. # # --attach options: # --password <p> server password (user defaults to "opencode") # --user <u> server username (default: opencode) # --directory <d> instance directory (default: $PWD) # --event <type> event type to wait for (default: server.connected) # --timeout <s> seconds to wait (default: 15) # # Usage: # sse-hook-probe.sh --self-test # sse-hook-probe.sh --attach http://127.0.0.1:4096 --password secret \ # --directory "$PWD" --event message.part.updated --timeout 30 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" . "$SCRIPT_DIR/lib/common.sh" # Watch an SSE stream for an event type. Args: url auth directory event timeout # Returns 0 if seen, 1 otherwise. Always kills its curl watcher. oqa_sse_watch() { local url="$1" auth="$2" dir="$3" want="$4" timeout="${5:-15}" local out cpid found="" deadline out="$(mktemp -t oqa-sse.XXXXXX)"; OQA_TMPDIRS+=("$out") if [ -n "$auth" ]; then curl -sN -u "$auth" "$url/event?directory=$dir" >"$out" 2>/dev/null & else curl -sN "$url/event?directory=$dir" >"$out" 2>/dev/null & fi cpid=$!; OQA_CURL_PIDS+=("$cpid") disown "$cpid" 2>/dev/null || true deadline=$(( $(date +%s) + timeout )) while [ "$(date +%s)" -lt "$deadline" ]; do if grep -q "\"$want\"" "$out" 2>/dev/null; then found=1; break; fi kill -0 "$cpid" 2>/dev/null || break sleep 0.2 done kill "$cpid" 2>/dev/null || true sleep 0.1 kill -0 "$cpid" 2>/dev/null && kill -9 "$cpid" 2>/dev/null || true if [ -n "$found" ]; then printf 'first matching event: ' grep -m1 "\"$want\"" "$out" | sed 's/^data: //' | jq -c '{type: .type}' 2>/dev/null || true return 0 fi oqa_log "stream head (no '$want' within ${timeout}s):"; head -5 "$out" >&2 return 1 } oqa_self_test() { oqa_require opencode curl jq || return 1 if ! oqa_start_server; then oqa_log "FAIL: server did not start"; return 1; fi if oqa_sse_watch "$OQA_SERVER_URL" "opencode:$OQA_SERVER_PASS" "$OQA_PROJ" "server.connected" 15; then oqa_pass "SSE /event opened and delivered server.connected" return 0 fi oqa_log "FAIL: did not observe server.connected"; return 1 } oqa_attach_mode() { local url="" user="opencode" pass="" dir="$PWD" event="server.connected" timeout=15 shift # drop --attach url="$1"; shift || true while [ $# -gt 0 ]; do case "$1" in --password) pass="$2"; shift 2 ;; --user) user="$2"; shift 2 ;; --directory) dir="$2"; shift 2 ;; --event) event="$2"; shift 2 ;; --timeout) timeout="$2"; shift 2 ;; *) oqa_log "unknown option: $1"; shift ;; esac done [ -n "$url" ] || { oqa_log "error: --attach requires a URL"; return 2; } local auth=""; [ -n "$pass" ] && auth="$user:$pass" oqa_log "watching $url/event?directory=$dir for '$event' (<=${timeout}s)" if oqa_sse_watch "$url" "$auth" "$dir" "$event" "$timeout"; then oqa_pass "observed '$event' on $url" return 0 fi oqa_log "FAIL: '$event' not observed"; return 1 } case "${1:-}" in --attach) oqa_attach_mode "$@"; exit $? ;; -h|--help) sed -n '2,34p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//' exit 0 ;; *) oqa_self_test; exit $? ;; esac -
tui-smoke.sh 3.6 KB
#!/usr/bin/env bash # tui-smoke.sh - launch the opencode TUI under tmux in an ISOLATED sandbox, # confirm it renders, prove send-keys reaches the composer, then tear down. # # This is a feasibility/smoke check, NOT a functional assertion harness. The # TUI is a 60fps full-screen app; reading its frame is fine for "did it boot # and accept a keystroke", but brittle for asserting conversation output. For # real behavior assertions use `opencode run` (Case A) or the server API / # SSE probe (Case B). See references/tui-tmux.md. # # Safety: runs opencode under an isolated XDG sandbox so no session is written # to the real ~/.local/share/opencode DB; the tmux session is always killed. # # Usage: # tui-smoke.sh # run the smoke test # tui-smoke.sh --self-test # same SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" . "$SCRIPT_DIR/lib/common.sh" oqa_tui_smoke() { oqa_require opencode tmux jq sqlite3 || return 1 local before after realdb ver sess cap found="" i fails=0 # Capture the REAL DB path + count BEFORE isolation. We must read it with # sqlite3 directly: once oqa_mk_isolated_xdg exports XDG_DATA_HOME, `opencode # db` would resolve the empty sandbox DB instead of the real one. realdb="$(oqa_db_path)" before="$(sqlite3 "$realdb" "SELECT count(*) FROM session" 2>/dev/null)" ver="$(opencode --version 2>/dev/null | head -1 | tr -d '[:space:]')" oqa_mk_isolated_xdg sess="oqa_tui_${$}_${RANDOM}" OQA_TMUX_SESSIONS+=("$sess") tmux new-session -d -s "$sess" -x 200 -y 50 # launch the TUI inside the pane, carrying the isolated sandbox env tmux send-keys -t "$sess" \ "HOME='$HOME' OPENCODE_TEST_HOME='$OPENCODE_TEST_HOME' XDG_DATA_HOME='$XDG_DATA_HOME' XDG_CONFIG_HOME='$XDG_CONFIG_HOME' XDG_CACHE_HOME='$XDG_CACHE_HOME' XDG_STATE_HOME='$XDG_STATE_HOME' OPENCODE_DISABLE_AUTOUPDATE=1 OPENCODE_DISABLE_MODELS_FETCH=1 opencode '$OQA_PROJ'" Enter # poll for a render marker (version string, composer placeholder, or footer) for ((i=0; i<50; i++)); do cap="$(tmux capture-pane -t "$sess" -p 2>/dev/null)" if printf '%s' "$cap" | grep -Eq "${ver}|Ask anything|ctrl\+p|agents"; then found=1; break; fi sleep 0.5 done if [ -n "$found" ]; then oqa_pass "TUI rendered under tmux (marker found; version ${ver:-?})" else oqa_log "FAIL: TUI did not render a known marker in 25s; pane was:"; printf '%s\n' "$cap" | head -8 >&2 fails=$((fails+1)) fi # prove send-keys reaches the composer: type a sentinel, expect it on screen if [ -n "$found" ]; then tmux send-keys -t "$sess" "oqaXYZ" sleep 1 cap="$(tmux capture-pane -t "$sess" -p 2>/dev/null)" if printf '%s' "$cap" | grep -q "oqaXYZ"; then oqa_pass "send-keys reached the TUI composer (sentinel echoed)" else oqa_log "WARN: sentinel not visible (TUI may have remapped input); render still proven" fi fi # teardown + verify tmux kill-session -t "$sess" 2>/dev/null || true sleep 0.5 if tmux has-session -t "$sess" 2>/dev/null; then oqa_log "FAIL: tmux session survived teardown"; fails=$((fails+1)) else oqa_pass "tmux session torn down (has-session false)" fi after="$(sqlite3 "$realdb" "SELECT count(*) FROM session" 2>/dev/null)" if [ "$before" = "$after" ]; then oqa_pass "real DB untouched (session count $before unchanged)" else oqa_log "FAIL: real DB session count changed $before -> $after"; fails=$((fails+1)) fi [ "$fails" -eq 0 ] && { oqa_pass "tui-smoke"; return 0; } oqa_log "tui-smoke had $fails failure(s)"; return 1 } case "${1:-}" in -h|--help) sed -n '2,20p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//' exit 0 ;; *) oqa_tui_smoke; exit $? ;; esac
-
-
SKILL.md 11.3 KB
--- name: opencode-qa description: "QA opencode itself, per case: verify the CLI/terminal (opencode run, db, serve, export), prove a specific plugin hook/action/event fired via the SSE event stream, smoke-test the TUI under tmux, and investigate sessions in opencode's SQLite DB by id, title/name, or message text. Ships tested helper scripts (each with a --self-test) plus per-domain references. Use whenever someone wants to QA, smoke-test, verify, or debug opencode's CLI, HTTP server, plugin hooks/events, or TUI, or to find/inspect opencode sessions in the database. Triggers: opencode qa, qa opencode, test opencode, verify opencode hook, opencode session db, find opencode session by id/name/text, opencode tui test, opencode server health, opencode event stream." --- # opencode QA QA the opencode coding agent itself. This skill maps each QA need to a tested helper script and a deep reference. Every script ships a `--self-test` that asserts its scenario against the live machine, so the scripts are both the QA tools and their own regression checks. Verified against opencode v1.17.7 (bun 1.3.12, macOS). Confirm the installed version with `opencode --version`; the surface is stable but always sanity check a flag with `opencode <cmd> --help`. ## Golden rules (read before running anything) - READS of the live DB are safe and intended. Investigating sessions (Case D) only reads `~/.local/share/opencode/opencode.db`. - Anything that SPAWNS opencode (serve, run, the TUI) must use an isolated XDG sandbox so QA never writes junk sessions into the real DB. The bundled scripts already do this; if you run opencode by hand for QA, set `XDG_DATA_HOME` / `XDG_CONFIG_HOME` / `XDG_STATE_HOME` / `XDG_CACHE_HOME` to temp dirs first. - Global text search over the `part` table is a multi-GB scan. Always scope it (`--session`, `--recent`, or `--since`). The text script refuses an unbounded scan on purpose. - The opencode source repo (`packages/opencode`) tests itself with `bun test` and CANNOT run tests from the repo root. See `references/testing-harness.md`. ## Setup Scripts live next to this file under `scripts/`. Invoke them from this skill directory (or with their absolute path): ```bash cd <this-skill-dir> # .agents/skills/opencode-qa bash scripts/lib/common.sh --self-check # confirm the harness + deps ``` **Docker is the default QA surface.** Run QA inside a disposable container that has the latest opencode and a copy of your config, with the host untouched: `script/agent/qa-docker.sh` (see [references/docker-qa.md](references/docker-qa.md)). The local scripts below are the fallback for when Docker is unavailable or on Windows. `common.sh` provides the shared harness (DB path, SQL escaping, isolated XDG sandbox, free port, server start/stop, and an EXIT-trap cleanup). It requires `opencode`, `sqlite3`, `curl`, `jq`, and `tmux` on PATH. ## Router: pick your case | You want to... | Case | Script | Reference | |---|---|---|---| | Run opencode non-interactively / check a CLI command | A | `opencode run --format json` (inline) | `references/cli-commands.md` | | Find a session by its id | D | `scripts/db-session-by-id.sh <ses_id>` | `references/db-investigation.md` | | Find sessions by title/name | D | `scripts/db-session-by-name.sh "<text>"` | `references/db-investigation.md` | | Find sessions by message text | D | `scripts/db-session-by-text.sh --recent N "<text>"` | `references/db-investigation.md` | | Export a whole session as JSON | D | `scripts/export-roundtrip.sh <ses_id>` | `references/db-investigation.md` | | Check the HTTP server / an endpoint | B | `scripts/server-smoke.sh` | `references/server-api.md` | | Prove a hook / action / event fired | B | `scripts/sse-hook-probe.sh` | `references/events-hooks.md` | | Prove serve-topology wake runner-split (reproduced/fixed) | B | `scripts/serve-wake-split-probe.sh --expect reproduced\|fixed --evidence-dir DIR` (self-test: `--self-test`; fake LLM: `scripts/lib/fake-openai-server.mjs`) | `references/events-hooks.md` | | Smoke-test the TUI | C | `scripts/tui-smoke.sh` | `references/tui-tmux.md` | | Write/run a test in the opencode source | - | (bun test) | `references/testing-harness.md` | | Drive opencode from a Bun/TS script | - | (SDK) | `references/sdk.md` | ## Case A: CLI / terminal works The canonical scriptable, non-interactive entry is `opencode run`. JSON mode emits one event per line so you can assert on it. ```bash # stream structured events (types: text, tool_use, step_start, step_finish, reasoning, error) opencode run "list files in src" --format json # run a slash command opencode run --command commit # resume the last session opencode run -c "continue" # target an already-running server instead of booting one opencode run "explain auth" --attach http://127.0.0.1:4096 -p "$OPENCODE_SERVER_PASSWORD" ``` Other QA-useful commands: `opencode db path`, `opencode debug paths`, `opencode session list --format json`, `opencode models --verbose`. Full flag detail in `references/cli-commands.md`. ## Case B: a specific hook, action, or event opencode publishes lifecycle events over Server-Sent Events at `GET /event`. Plugins observe the same events via the `event` hook, so seeing an event on the wire proves a hook would fire. ```bash # prove the SSE plumbing works (isolated server, asserts server.connected) bash scripts/sse-hook-probe.sh --self-test # watch a REAL server for a specific event while you trigger an action bash scripts/sse-hook-probe.sh --attach http://127.0.0.1:4096 \ --password "$OPENCODE_SERVER_PASSWORD" --directory "$PWD" \ --event message.part.updated --timeout 30 ``` Trigger an action over HTTP (fire-and-forget so the stream is not blocked): ```bash curl -X POST -u opencode:$OPENCODE_SERVER_PASSWORD -H 'Content-Type: application/json' \ -d '{"parts":[{"type":"text","text":"say hi"}]}' \ "http://127.0.0.1:4096/session/<ses_id>/prompt_async?directory=$PWD" ``` A real prompt needs a configured provider, so run the watch-and-trigger pattern against your real server, not the isolated sandbox. Event-type catalog, the 21 plugin hook points, and how to load a local plugin: `references/events-hooks.md`. Server start, auth, and routes: `references/server-api.md`. ## Case C: the TUI ```bash bash scripts/tui-smoke.sh --self-test ``` This launches the TUI under tmux in an isolated sandbox, confirms it renders (`capture-pane`), confirms `send-keys` reaches the composer, tears the tmux session down, and verifies the real DB session count is unchanged. When TUI visual QA evidence is needed for a PR, follow `docs/reference/web-terminal-visual-qa.md`: render the TUI through the real xterm.js web terminal and screenshot it - NEVER the `tmux capture-pane` frame, which degrades color and CJK width. From the repository root: ```bash node script/qa/web-terminal-visual-qa.mjs --title "OpenCode TUI QA" \ --command "opencode" --input "{Enter}" \ --evidence-dir .omo/evidence/<slug>/opencode-web-terminal ``` This runs a real pty, renders it in xterm.js under Chrome, and writes `terminal.txt`, `terminal-ansi.txt`, `terminal.png` (the true-color artifact), and `metadata.json` with a cleanup receipt (`--from-file <capture.ansi>` replays a saved raw stream). The isolated `scripts/tui-smoke.sh` remains the canonical OpenCode TUI boot smoke (tmux), separate from this visual evidence. Honest verdict: tmux is fine for SMOKE (did it boot, render, accept a key) but fragile for asserting conversation output (the TUI is a 60fps full-screen app). For real behavior assertions use Case A (`opencode run`), Case B (server API + SSE), or the TUI control HTTP API (`POST /tui/append-prompt`, `POST /tui/submit-prompt`, `POST /tui/execute-command`). Details and the manual tmux recipe: `references/tui-tmux.md`. ## Case D: investigate sessions in the DB Read-only against the live SQLite DB. The `session` table is small (title and id lookups are instant); message text lives in the multi-GB `part` table, so text search must be scoped. ```bash # by id bash scripts/db-session-by-id.sh ses_3a4ee6335ffedFB8f76BPU1Eb3 # by title / name (newest first; second arg = limit) bash scripts/db-session-by-name.sh "auth refactor" 20 # by message text - scope with --session, --recent N, or --since "<window>" bash scripts/db-session-by-text.sh --session ses_3a4e... "ULTRAWORK" bash scripts/db-session-by-text.sh --recent 50 "permission denied" bash scripts/db-session-by-text.sh --since "7 days" --limit 50 "TODO" # export an entire session as clean JSON bash scripts/export-roundtrip.sh ses_3a4e... > session.json ``` Ad hoc queries: `opencode db "<SQL>" --format json`. Schema, tested query shapes with timings, the legacy `message`/`part` vs V2 `session_message` distinction, and the 25 GB caveat: `references/db-investigation.md`. ## Scripts index Run any script with `--self-test` to verify it against the live machine, or `-h` for usage. DB-read scripts are read-only; serve/sse/tui scripts use an isolated sandbox and clean up on exit. | Script | Case | Self-test asserts | |---|---|---| | `scripts/lib/common.sh --self-check` | - | deps present, DB path resolves, SQL escaping, free port, sandbox auto-removed | | `scripts/db-session-by-id.sh` | D | id round-trips for a real session | | `scripts/db-session-by-name.sh` | D | a derived title needle returns >=1 row | | `scripts/db-session-by-text.sh` | D | scoped search hits; unbounded scan refused; bounded search <30s | | `scripts/export-roundtrip.sh` | D | export stdout is valid JSON and `.info.id` round-trips | | `scripts/server-smoke.sh` | B | `/global/health` healthy, `/doc` >=100 paths, no-auth -> 401 | | `scripts/sse-hook-probe.sh` | B | `/event` opens and delivers `server.connected` | | `scripts/tui-smoke.sh` | C | TUI renders under tmux, tears down, real DB untouched | ## Risks and caveats - 25 GB part table: never run an unbounded text scan. Use `--session`, `--recent`, or `--since`. A naive `JOIN ... WHERE session.time_created >= X` scans oldest-first and can take ~50s; the scripts use an `IN`-subquery on the newest sessions (~20ms). - `opencode export` writes its banner to STDERR; pipe with `2>/dev/null` before `jq` or you will get a parse error. - The server enforces auth only when `OPENCODE_SERVER_PASSWORD` is set; otherwise it runs unsecured. Authenticated calls use `-u opencode:$PASS`. Unauthenticated calls to a secured server return HTTP 401. - Installed binary vs dev source: cite dev source paths for internals but verify flags against the installed `opencode <cmd> --help`. - Isolation: any QA that spawns opencode must use an isolated XDG sandbox so it never pollutes the real DB. Prove it by comparing `sqlite3 "$(opencode db path)" "SELECT count(*) FROM session"` before and after. - TUI output assertions are fragile; use the API for real assertions. ## References - `references/cli-commands.md` - every QA-relevant opencode subcommand and flag - `references/db-investigation.md` - DB schema, tested queries, the 25 GB caveat - `references/server-api.md` - server start, auth, route catalog, /doc - `references/events-hooks.md` - SSE endpoints, event types, plugin hooks - `references/tui-tmux.md` - tmux recipe, isolation, TUI control API - `references/testing-harness.md` - how opencode tests itself (bun test) - `references/sdk.md` - the @opencode-ai/sdk client (reference only) - `references/docker-qa.md` - run QA in a disposable Docker container (default; local is the fallback)
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.