self-review
Review the user's own manuscript, paper, thesis chapter, rebuttal, or release packet with clean-room anti-contamination controls and, when needed, an unfamiliar-reader comprehension gate. Use for internal review, readiness checks, reviewer simulation, or claim-evidence self-audit
Install
npx skills add https://github.com/yha9806/academic-writing-toolkit/tree/main/archive/skills/self-review
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install yha9806-academic-writing-toolkit@llmmart
git clone https://github.com/yha9806/academic-writing-toolkit.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole yha9806/academic-writing-toolkit collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
/self-review - Clean-Room Manuscript Self-Review
Purpose
Audit the user's own work without letting memory, prior chats, unstated project knowledge, or the model's background knowledge become evidence.
The governing rule is:
self-review truth = explicit review packet + source anchor
Use /argument-governance first when the manuscript needs a formal intent, contribution, claim, and evidence map.
Codex-Only Baseline
Complete self-review with Codex, the review manifest, allowed sources, and the bundled packet checker. Do not require Gemini, gemini-agent, a second model, or a subagent. If an external review is available, keep it in Reviewer-risk inference or advisory notes and never use it as source support.
If /argument-governance is unavailable, manually extract the same clean-room argument spine from manifest-listed sources only.
Enhanced Advisory Mode
If the manifest and the user explicitly allow an API-key-backed advisory review, Codex may run or incorporate a second-model pass after the clean-room self-review packet is valid.
Rules:
- the base clean-room review must be possible without the external call
- API keys must be read from environment variables only
- the manifest may name
api_key_env_var, but must never store the key value - only manifest-approved source subsets may be sent externally
- external findings must be placed under
Reviewer-risk inferenceor advisory notes - external findings must be re-grounded against allowed sources before becoming revision actions
- unsupported external comments stay unsupported
Core Rules
- Use only files listed in the review manifest.
- Treat prior chat memory, unstated project assumptions, model background knowledge, and unlisted notes as forbidden evidence.
- Split every finding into
Supported by packet,Not supported by packet, orReviewer-risk inference. - Every supported finding must include a source anchor.
- Do not repair missing evidence by remembering earlier conversations.
- Do not treat generated reviews, agent drafts, or reviewer simulations as final evidence.
- Do not edit the manuscript until the user approves specific revision actions.
- Do not treat an unavailable external review tool as a blocker.
Required Packet
The preferred layout is:
review_packet/
review_manifest.yaml
manuscript.md or manuscript.pdf
references.bib
evidence/
figures/
tables/
claims/
Read references/clean_room_protocol.md before reviewing. Read references/self_review_packet_schema.md before creating or validating a packet.
Workflow
1. Validate The Clean-Room Packet
Resolve the bundled helper at scripts/check_self_review_packet.py relative to this SKILL.md, then run:
python3 {skill_dir}/scripts/check_self_review_packet.py review_packet --json
If the packet is missing or invalid, report the issue before reviewing.
2. Build The Source-Bounded Reading List
Read only manifest-listed files. If a needed file is not listed, ask whether to add it to the manifest or mark the issue as unsupported.
3. Extract The Argument Spine
From the packet only, extract:
- stated intent
- named gap
- contributions
- main claims
- evidence anchors
- limitations
- reviewer-risk areas
4. Run Self-Review Checks
Check:
- gap-contribution alignment
- claim hierarchy
- claim-evidence fit
- evidence balance
- unsupported or overextended claims
- missing limitations
- reviewer attacks with weak defenses
- internal consistency and submission blockers
4.5 Run The Unfamiliar-Reader Gate When Required
For an important version, submission-readiness claim, or revision contract that
requires a comprehension check, read
references/reader_comprehension_gate.md. Prepare a packet containing only the
title, abstract, Figure 1, and main Results summary table, all of which must be
manifest-listed. Record the unfamiliar human reader's answers to the five
questions and compare them with the author-approved intent and argument
baseline.
Only an actual unfamiliar human response can produce passed. A model
simulation, author explanation, or response from a collaborator with prior
project knowledge remains advisory_only. If no eligible response exists,
record not_run; do not claim that the human gate passed.
5. Write A Clean-Room Report
The report must separate:
Supported by packetNot supported by packetReviewer-risk inference
Do not merge these categories.
Output Pattern
## Clean-Room Self-Review
### Packet Boundary
- Manifest:
- Allowed sources:
- Forbidden sources:
### Supported By Packet
| Finding | Source anchor | Severity | Action |
### Not Supported By Packet
| Claim or need | Missing source | Risk | Action |
### Reviewer-Risk Inference
| Risk | Basis in packet | Why it matters | Action |
### Unfamiliar-Reader Gate
- Packet:
- Reader independence:
- Decision: passed / failed / not_run / advisory_only
- Misunderstood or missing functions:
- Required next action:
### Revision Actions
| Priority | Action | Requires new evidence? |
Stop Conditions
Stop and report a blocker if:
- no review manifest is provided
- the user asks to rely on previous chat or memory for evidence
- a central claim needs a source not listed in the manifest
- the packet validator reports missing allowed files
- the user asks to mark unsupported claims as supported
- a required unfamiliar-reader gate lacks an eligible human response but the manuscript is being described as having passed it
Files (academic-writing-toolkit)
-
references
-
clean_room_protocol.md 2 KB
# Clean-Room Self-Review Protocol Use this protocol whenever reviewing the user's own work. ## Source Boundary The review may use only: - files listed under `allowed_sources` - text pasted in the current request and explicitly identified as review material - deterministic checker outputs generated from allowed sources Codex can complete the review from these sources alone. External model output is never required. ## Enhanced Advisory Review An API-key-backed external model pass is optional. Use it only when: - the user explicitly enables it - the manifest permits external advisory review - the source subset is listed - the API key is stored in an environment variable - the output is labelled advisory External output can suggest reviewer risks, but it cannot support claims. The review must not use: - prior chat memory - unstated project assumptions - model background knowledge as evidence - unpublished notes not listed in the manifest - untracked local files not listed in the manifest - generated reviewer suggestions as final evidence - external model output as source support - API key values stored in files ## Finding Categories ### Supported by packet Use this only when the finding is directly anchored to a listed source. Required fields: - finding - source anchor - claim ID or section - severity - action ### Not supported by packet Use this when a claim, number, interpretation, or revision need cannot be anchored to the packet. Required fields: - claim or need - missing source - risk - action ### Reviewer-risk inference Use this for plausible reviewer objections inferred from the packet, not for claims of fact. Required fields: - risk - basis in packet - why it matters - action ## Anti-Contamination Tests Before finalising the report, ask: 1. Did I use any fact that is not in the packet? 2. Did I turn prior conversation context into evidence? 3. Did I treat my own general knowledge as source support? 4. Did I explain every supported finding with a source anchor? 5. Did I mark missing support as a gap rather than silently filling it? -
reader_comprehension_gate.md 2.1 KB
# Unfamiliar-Reader Comprehension Gate Use this gate for an important manuscript version after its research spine is author-approved. It tests whether the paper's purpose and evidence boundary are recoverable without exposing the reader to the full manuscript. ## Reader Packet Provide only: - title - abstract - Figure 1 with its caption - the main Results summary table with its caption All items must be listed in the self-review manifest. Do not add explanatory notes, the author-intent card, prior chat, or a verbal briefing. ## Reader Questions Ask an unfamiliar human reader to answer, preferably within three minutes: 1. What concrete problem does the paper address? 2. Who could use or benefit from the result? 3. What did the authors actually do? 4. What is the main result? 5. What has not yet been validated? Record answers verbatim, elapsed time when available, and whether the reader had prior project knowledge. ## Decision Compare the answers with the author-approved intent and argument baseline. Use: - `passed`: the unfamiliar reader recovers all five functions without author explanation and without material contradiction; - `failed`: one or more functions are missing, materially wrong, or require author explanation; - `not_run`: no eligible human response is available; - `advisory_only`: a model, collaborator with prior knowledge, or other non-independent reader supplied the response. A model simulation can identify likely ambiguity but cannot pass this human gate. Numerical, citation, reference, or PDF checks do not substitute for it. If the result is `failed`, classify the failure as missing application purpose, unclear research object or method task, hidden main result, missing evidence boundary, or cross-section identity drift. Route structural failures to `/thesis-control` or `/manuscript-reframe`; do not repeatedly paraphrase the same abstract without an approved edit contract. ## Record Template ```text Version: Argument baseline: Reader independence: Elapsed time: Q1 answer: Q2 answer: Q3 answer: Q4 answer: Q5 answer: Decision: passed / failed / not_run / advisory_only Mismatch summary: Required next action: ``` -
self_review_packet_schema.md 1.8 KB
# Self-Review Packet Schema Create `review_packet/review_manifest.yaml` with this shape: ```yaml review_mode: self_review_clean_room allowed_sources: - manuscript.md - references.bib - evidence/claim_register.csv - evidence/evidence_matrix.csv - evidence/claim_hierarchy.csv - evidence/contribution_chain.csv forbidden_sources: - prior_chat_memory - unstated_project_assumptions - model_background_knowledge_as_evidence - unpublished_notes_not_listed_in_manifest review_outputs: - self_review_report.md advisory_review: enabled: false provider: "" model: "" api_key_env_var: "" allowed_to_send_sources: false allowed_source_subset: - manuscript.md output_path: advisory/advisory_review.md ``` ## Required Keys - `review_mode` - `allowed_sources` - `forbidden_sources` ## Required Forbidden Sources The manifest should explicitly forbid: - `prior_chat_memory` - `unstated_project_assumptions` - `model_background_knowledge_as_evidence` - `unpublished_notes_not_listed_in_manifest` ## Source Rules - Allowed sources must be relative paths inside the packet. - Absolute local paths should not appear in the manifest. - Directories may be listed with a trailing slash. - If a file is needed but missing from `allowed_sources`, add it to the manifest before using it or mark the related finding as unsupported. ## Advisory Review Rules - `advisory_review` is optional and defaults to disabled. - Store API key names only, such as `OPENAI_API_KEY` or `GEMINI_API_KEY`. - Never store API key values in the manifest. - `allowed_to_send_sources` must be true before any listed source is sent to an external model. - `allowed_source_subset` must be a subset of `allowed_sources`. - Advisory output is not evidence; it must be re-grounded against allowed sources.
-
-
scripts
-
check_self_review_packet.py 7.1 KB
#!/usr/bin/env python3 """Validate clean-room self-review packets.""" from __future__ import annotations import argparse import json import re import sys from pathlib import Path, PureWindowsPath from typing import Dict, List REQUIRED_FORBIDDEN = { "prior_chat_memory", "unstated_project_assumptions", "model_background_knowledge_as_evidence", "unpublished_notes_not_listed_in_manifest", } def issue(kind: str, severity: str, location: str, message: str) -> Dict[str, str]: return { "kind": kind, "severity": severity, "location": location, "message": message, } def normalise_token(value: str) -> str: return re.sub(r"[^a-z0-9]+", "_", value.lower()).strip("_") def parse_simple_yaml(path: Path) -> tuple[Dict[str, object], List[Dict[str, str]]]: issues: List[Dict[str, str]] = [] data: Dict[str, object] = {} current_key = "" text = path.read_text(encoding="utf-8-sig") for line_no, raw in enumerate(text.splitlines(), start=1): stripped = raw.strip() if not stripped or stripped.startswith("#"): continue if re.match(r"(?i)api[_-]?key\s*:", stripped) and "api_key_env_var" not in stripped: issues.append(issue("secret-in-manifest", "critical", f"{path}:{line_no}", "store API key values in environment variables, not the manifest")) if re.search(r"(?i)(sk-[A-Za-z0-9_-]{12,}|AIza[0-9A-Za-z_-]{20,}|api[_-]?key\s*:\s*['\"]?[A-Za-z0-9_-]{16,})", stripped): issues.append(issue("possible-secret-in-manifest", "critical", f"{path}:{line_no}", "manifest appears to contain a credential-like value")) if stripped.startswith("- "): if not current_key: issues.append(issue("yaml-list-without-key", "high", f"{path}:{line_no}", "list item has no parent key")) continue value = stripped[2:].strip().strip("'\"") data.setdefault(current_key, []) if isinstance(data[current_key], list): data[current_key].append(value) else: issues.append(issue("yaml-type-conflict", "high", f"{path}:{line_no}", f"{current_key} cannot be both scalar and list")) continue if ":" not in stripped: issues.append(issue("yaml-parse-warning", "medium", f"{path}:{line_no}", "expected key: value or key:")) continue key, value = stripped.split(":", 1) current_key = key.strip() value = value.strip() if value: data[current_key] = value.strip("'\"") else: data[current_key] = [] return data, issues def is_abs_path(value: str) -> bool: path = Path(value) win = PureWindowsPath(value) return path.is_absolute() or win.is_absolute() def list_value(data: Dict[str, object], key: str) -> List[str]: value = data.get(key, []) if isinstance(value, list): return [str(item).strip() for item in value if str(item).strip()] if isinstance(value, str) and value.strip(): return [value.strip()] return [] def validate_packet(packet_dir: Path) -> List[Dict[str, str]]: issues: List[Dict[str, str]] = [] manifest = packet_dir / "review_manifest.yaml" if not manifest.is_file(): alt = packet_dir / "review_manifest.yml" if alt.is_file(): manifest = alt else: return [issue("missing-manifest", "high", str(packet_dir), "review_manifest.yaml is required")] try: data, parse_issues = parse_simple_yaml(manifest) issues.extend(parse_issues) except UnicodeDecodeError: return [issue("manifest-decode-error", "high", str(manifest), "manifest is not valid UTF-8")] mode = str(data.get("review_mode", "")).strip() if mode != "self_review_clean_room": issues.append(issue("invalid-review-mode", "high", "review_manifest.yaml:review_mode", "review_mode must be self_review_clean_room")) allowed = list_value(data, "allowed_sources") forbidden = list_value(data, "forbidden_sources") if not allowed: issues.append(issue("missing-allowed-sources", "high", "review_manifest.yaml:allowed_sources", "at least one allowed source is required")) if not forbidden: issues.append(issue("missing-forbidden-sources", "high", "review_manifest.yaml:forbidden_sources", "forbidden sources must be explicit")) forbidden_tokens = {normalise_token(item) for item in forbidden} missing_forbidden = sorted(REQUIRED_FORBIDDEN - forbidden_tokens) if missing_forbidden: issues.append(issue("missing-required-forbidden-source", "high", "review_manifest.yaml:forbidden_sources", "missing: " + ", ".join(missing_forbidden))) for source in allowed: token = normalise_token(source) if token in REQUIRED_FORBIDDEN: issues.append(issue("forbidden-source-allowed", "critical", f"review_manifest.yaml:allowed_sources:{source}", "forbidden source appears in allowed_sources")) if is_abs_path(source): issues.append(issue("absolute-source-path", "medium", f"review_manifest.yaml:allowed_sources:{source}", "allowed sources should be relative packet paths")) continue candidate = packet_dir / source if source.endswith("/"): if not candidate.is_dir(): issues.append(issue("allowed-directory-missing", "high", f"review_manifest.yaml:allowed_sources:{source}", "listed directory does not exist")) elif not candidate.is_file() and not candidate.is_dir(): issues.append(issue("allowed-source-missing", "high", f"review_manifest.yaml:allowed_sources:{source}", "listed source does not exist")) report = packet_dir / "self_review_report.md" if report.is_file(): text = report.read_text(encoding="utf-8", errors="replace") for heading in ["Supported By Packet", "Not Supported By Packet", "Reviewer-Risk Inference"]: if heading not in text and heading.replace("By", "by") not in text: issues.append(issue("report-missing-section", "medium", f"self_review_report.md:{heading}", "clean-room report should keep findings separated")) return issues def main() -> int: parser = argparse.ArgumentParser(description="Validate a clean-room self-review packet.") parser.add_argument("packet_dir", help="Directory containing review_manifest.yaml") parser.add_argument("--json", action="store_true", dest="emit_json") args = parser.parse_args() packet_dir = Path(args.packet_dir) if not packet_dir.is_dir(): sys.stderr.write("error: packet_dir is not a directory\n") return 2 issues = validate_packet(packet_dir) payload = { "schema_version": 1, "packet_dir": str(packet_dir), "issue_count": len(issues), "issues": issues, } if args.emit_json: print(json.dumps(payload, indent=2)) else: for item in issues: print("{severity}: {location}: {kind}: {message}".format(**item)) if not issues: print("self-review packet is clean-room valid") return 1 if issues else 0 if __name__ == "__main__": sys.exit(main())
-
-
SKILL.md 5.9 KB
--- name: self-review description: Review the user's own manuscript, paper, thesis chapter, rebuttal, or release packet with clean-room anti-contamination controls and, when needed, an unfamiliar-reader comprehension gate. Use for internal review, readiness checks, reviewer simulation, or claim-evidence self-audit where prior chat memory and unstated context must not become evidence. allowed-tools: Read, Glob, Grep, Bash, Edit, Write --- # /self-review - Clean-Room Manuscript Self-Review ## Purpose Audit the user's own work without letting memory, prior chats, unstated project knowledge, or the model's background knowledge become evidence. The governing rule is: ```text self-review truth = explicit review packet + source anchor ``` Use `/argument-governance` first when the manuscript needs a formal intent, contribution, claim, and evidence map. ## Codex-Only Baseline Complete self-review with Codex, the review manifest, allowed sources, and the bundled packet checker. Do not require Gemini, gemini-agent, a second model, or a subagent. If an external review is available, keep it in `Reviewer-risk inference` or advisory notes and never use it as source support. If `/argument-governance` is unavailable, manually extract the same clean-room argument spine from manifest-listed sources only. ## Enhanced Advisory Mode If the manifest and the user explicitly allow an API-key-backed advisory review, Codex may run or incorporate a second-model pass after the clean-room self-review packet is valid. Rules: - the base clean-room review must be possible without the external call - API keys must be read from environment variables only - the manifest may name `api_key_env_var`, but must never store the key value - only manifest-approved source subsets may be sent externally - external findings must be placed under `Reviewer-risk inference` or advisory notes - external findings must be re-grounded against allowed sources before becoming revision actions - unsupported external comments stay unsupported ## Core Rules 1. Use only files listed in the review manifest. 2. Treat prior chat memory, unstated project assumptions, model background knowledge, and unlisted notes as forbidden evidence. 3. Split every finding into `Supported by packet`, `Not supported by packet`, or `Reviewer-risk inference`. 4. Every supported finding must include a source anchor. 5. Do not repair missing evidence by remembering earlier conversations. 6. Do not treat generated reviews, agent drafts, or reviewer simulations as final evidence. 7. Do not edit the manuscript until the user approves specific revision actions. 8. Do not treat an unavailable external review tool as a blocker. ## Required Packet The preferred layout is: ```text review_packet/ review_manifest.yaml manuscript.md or manuscript.pdf references.bib evidence/ figures/ tables/ claims/ ``` Read `references/clean_room_protocol.md` before reviewing. Read `references/self_review_packet_schema.md` before creating or validating a packet. ## Workflow ### 1. Validate The Clean-Room Packet Resolve the bundled helper at `scripts/check_self_review_packet.py` relative to this `SKILL.md`, then run: ```bash python3 {skill_dir}/scripts/check_self_review_packet.py review_packet --json ``` If the packet is missing or invalid, report the issue before reviewing. ### 2. Build The Source-Bounded Reading List Read only manifest-listed files. If a needed file is not listed, ask whether to add it to the manifest or mark the issue as unsupported. ### 3. Extract The Argument Spine From the packet only, extract: - stated intent - named gap - contributions - main claims - evidence anchors - limitations - reviewer-risk areas ### 4. Run Self-Review Checks Check: - gap-contribution alignment - claim hierarchy - claim-evidence fit - evidence balance - unsupported or overextended claims - missing limitations - reviewer attacks with weak defenses - internal consistency and submission blockers ### 4.5 Run The Unfamiliar-Reader Gate When Required For an important version, submission-readiness claim, or revision contract that requires a comprehension check, read `references/reader_comprehension_gate.md`. Prepare a packet containing only the title, abstract, Figure 1, and main Results summary table, all of which must be manifest-listed. Record the unfamiliar human reader's answers to the five questions and compare them with the author-approved intent and argument baseline. Only an actual unfamiliar human response can produce `passed`. A model simulation, author explanation, or response from a collaborator with prior project knowledge remains `advisory_only`. If no eligible response exists, record `not_run`; do not claim that the human gate passed. ### 5. Write A Clean-Room Report The report must separate: - `Supported by packet` - `Not supported by packet` - `Reviewer-risk inference` Do not merge these categories. ## Output Pattern ```text ## Clean-Room Self-Review ### Packet Boundary - Manifest: - Allowed sources: - Forbidden sources: ### Supported By Packet | Finding | Source anchor | Severity | Action | ### Not Supported By Packet | Claim or need | Missing source | Risk | Action | ### Reviewer-Risk Inference | Risk | Basis in packet | Why it matters | Action | ### Unfamiliar-Reader Gate - Packet: - Reader independence: - Decision: passed / failed / not_run / advisory_only - Misunderstood or missing functions: - Required next action: ### Revision Actions | Priority | Action | Requires new evidence? | ``` ## Stop Conditions Stop and report a blocker if: - no review manifest is provided - the user asks to rely on previous chat or memory for evidence - a central claim needs a source not listed in the manifest - the packet validator reports missing allowed files - the user asks to mark unsupported claims as supported - a required unfamiliar-reader gate lacks an eligible human response but the manuscript is being described as having passed it
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.