# Response-Letter Voice Reference (R2R / Rebuttal)

A reference for writing response-to-reviewers (R2R) letters and editor cover letters that
read as human-written scientific argument rather than machine-generated change-logs. Use
alongside the **Response-Letter Voice & AI-Tell Avoidance** section of the revise SKILL.md.

All examples below are **synthetic** (a fictional deep-learning lung-nodule CT study) and are
illustrative only. They contain no real study, author, or institution.

Sources:
- matsuikentaro1/humanizer_academic principles (general AI-writing patterns)
- Register calibrated from the conventions of published point-by-point response letters

---

## The core problem: the editing-mechanism register

Machine-drafted response letters narrate *how the text was edited* instead of the *science*.
The result reads like a diff log: which line changed, how many phrases were swapped, what a
verification pass returned. A careful reviewer reads this as auto-generated and as evidence
the authors are clearing a checklist rather than engaging with the critique.

The fix is one habit: **state what changed and why in plain prose, quote the new sentence,
name the section — and say nothing about the mechanism by which you found or made the edit.**

This targets the *editing* mechanism only. Narrating a **new analysis you ran** is the science
the reviewer asked for and is welcome — e.g., "we performed a sensitivity analysis restricted to
one observation per patient, and the primary findings held." The tell is describing the
text-editing operation ("we softened six phrases", "v2 adds a sentence"), not describing the
analysis or its result.

---

## Two acceptable formats (the structure is not the problem)

Real published response letters use any of several formats, and all read as human:

- **A. Free prose** — "We agree. We have added to the Methods: '...'."
- **B. Structured** — a `Response:` paragraph followed by a `Changes made:` (or `Action taken in
  the revised manuscript:`) block, often with `Original: '...'` → `Revised: '...'` before/after
  pairs and the section and revised-manuscript page/line.
- **C. Two-column table** — reviewer comment on the left, author response on the right.

Other normal conventions seen across real letters: restating the reviewer's comment before
responding, and numbering multiple changes within one response as (1), (2), (3).

The AI-tell is **not** the structure, the tables, or the before/after pairs — careful authors
write all of these. The tell is the **editing-mechanism narration** layered on top: version
prefixes ("v2 adds..."), internal draft line numbers, grep/verification language, internal FIX
codes, `§` markers, and bare "No further manuscript change" stubs. Keep whichever format you
like; strip the mechanism.

---

## Before / after gallery

### 1. Version-prefixed change narration + internal line numbers

> Reviewer: The causal language overstates an observational design.

**AI-tell (bad):**
"v2 Methods §Design (line 43) adds one sentence: 'The user/non-user contrast is reported
descriptively and is not interpreted as evidence of efficacy.' This is a short visible
clarification in the manuscript body rather than only in the Limitations."

**Natural (good):**
"We agree. The design is observational, so we have added a sentence to the Methods (Design
subsection) making the descriptive intent explicit: 'The user/non-user contrast is reported
descriptively and is not interpreted as evidence of efficacy.' The corresponding causal
phrasing in the Results has been removed."

Why: the natural version keeps the quoted new sentence and a section-name pointer but drops
the version prefix, the `§` marker, the internal line number, and the meta-commentary about
*where* the clarification sits.

### 2. Tooling / verification leak

> Reviewer: Several results are described as "numerical trends," which is speculative.

**AI-tell (bad):**
"A grep-and-soften pass replaced six phrases. Final grep verification across the manuscript
and the circulated supplementary tables returned zero occurrences of 'numerical trend',
'consistent direction', or 'possible relationship'."

**Natural (good):**
"We have rephrased these passages so that non-significant point estimates are no longer
presented as trends. For example, 'users showed a numerically larger reduction' now reads
'the point estimate favoured the user arm, but the 95% confidence interval crossed zero and
is uninformative for inference.'"

Why: the reviewer cares about the corrected wording and the principle behind it, not that the
authors ran a search to confirm the strings were gone.

### 3. Redundant "no change" stubs

> Reviewer: Retention was low.

**AI-tell (bad):**
"This was already reported in v1. No further manuscript change was applied."

**Natural (good):**
"We have kept the retention figure but moved it to the front of the Conclusions so it leads
the feasibility assessment rather than appearing as an aside: the final retention of 58%
fell short of the 80% target and is the binding constraint on the present pilot."

Why: even when little changes, reply with substance. A bare "no change" stub repeated across
comments is a hallmark of checklist-driven drafting.

### 4. Uniform openers

**AI-tell (bad):** every response begins "We thank the reviewer for this important suggestion."

**Natural (good):** vary by stance —
- "We agree, and have revised accordingly."
- "This is a fair concern about generalisability."
- "We see how the original wording invited this reading; we have corrected it."
- "We respectfully maintain the original analysis, for the reasons below."

### 5. Admitting an error

**AI-tell (bad):**
"v3 corrects the acquisition-time value at line 211 per the reviewer's observation."

**Natural (good):**
"The reviewer is correct. During revision we found an error in the reported acquisition time;
the correct value is 15 s per phase. We have corrected this in the Methods and apologise for
the oversight."

### 6. Tempering an over-claim (the most common revision task)

> Reviewer: Several results are described as "significantly outperformed," but the confidence
> intervals overlap; this overstates the evidence.

**AI-tell (bad):**
"v2 grep-and-soften pass replaced 'significantly outperformed' at lines 14, 88, and 203. Final
grep verification across the circulated manuscript returned zero occurrences of
'significantly', 'consistent direction', or 'numerical superiority' (§Results, §Discussion).
FIX-2 vocabulary cascade applied."

**Natural (good):**
"We fully agree. Given the overlapping 95% confidence intervals, 'significantly outperformed'
overstated the evidence. We have tempered this language in the Abstract, Results, and
Discussion to describe the difference as a modest, consistent improvement rather than a
statistically significant one. For example, in the Results:
*Original:* 'the 2.5D model significantly outperformed the 2D model.'
*Revised:* 'the 2.5D model achieved a modestly higher C-index (0.71 vs 0.68); the confidence
intervals overlap, so this is a consistent but not statistically significant difference.'"

Why: the natural version makes the same correction the reviewer asked for, shows the exact
before/after wording, and names the affected sections — without the grep narration, the
internal line numbers, the `§` markers, or the internal FIX label.

---

## Succinctness & non-defensiveness (R2+)

Items 1-6 strip the *editing-mechanism* tell. These strip the *defensive over-elaboration* tell,
which dominates later (R2+) rounds: pre-emptive lobbying, manufactured paragraphs for satisfied
reviewers, and disclosure piled into a front section. The fix is always to say less and let the
point-by-point carry the work. (Examples remain synthetic — a fictional imaging deep-learning study.)

### 7. Pre-emptive hand-holding / cross-reviewer lobbying

**AI-tell (bad):**
"We respectfully note that Reviewers 1 and 3 found this analysis appropriate, and we left the
primary endpoint unchanged on that basis. We hope Reviewer 2 will agree that the consensus of
the panel supports our approach."

**Natural (good):**
"We retained the primary endpoint as pre-registered. Our reasoning is [one-sentence scientific
justification]; we have added it to the Methods: '[new sentence].'"

Why: answer the reviewer in front of you on the merits. Citing what other reviewers thought is
lobbying, not science, and reads as defensive.

### 8. Defensive meta-comment about an unchanged passage

**AI-tell (bad):**
"We confirm that this statement is unchanged and has not been softened or weakened in any way,
and we emphasise that it already fully conveys the intended claim."

**Natural (good):**
"This is already stated in the Discussion: '[existing sentence].'"

Why: point to where the matter is handled and stop. Insisting that nothing was weakened invites
the suspicion that something was.

### 9. Over-elaborated response to a satisfied reviewer

> Reviewer: The revised manuscript is much improved and I have no further concerns.

**AI-tell (bad):**
"We are deeply grateful to the reviewer for this generous assessment. It has been a privilege to
benefit from such careful and constructive guidance throughout, and we are delighted that the
substantial revisions across the Methods, Results, and Discussion have fully addressed the
concerns raised in the previous round."

**Natural (good):**
"Thank you."

Why: a satisfied reviewer needs one sentence. A paragraph of gratitude is padding and reads as
machine-generated filler.

### 10. Methodology disclosure as a separate front section

**AI-tell (bad):**
A standalone "Statistical Note" at the top of the letter: "Before responding to individual
comments, we wish to disclose that the subgroup analyses were not adjusted for multiplicity..."

**Natural (good):** inside the response to the comment that raised it —
"The reviewer is right to ask about multiplicity. These subgroup analyses were exploratory and
are not adjusted for multiple comparisons; we now state this in the Methods and label them as
exploratory in Table 3."

Why: keep the disclosure (never hide a deviation) but put it where the reviewer raised it. A
front-loaded "note" separates the admission from the question and reads as pre-emptive defence.

### 11. Bundling a multi-point paragraph into one block reply

> Reviewer: The introduction is too long, the cohort definition is unclear, and Figure 2 is
> hard to read.

**AI-tell (bad):**
"We thank the reviewer for these helpful comments. We have shortened the introduction, clarified
the cohort, and improved Figure 2 accordingly."

**Natural (good):** split into three —
"**Comment 1.** *'The introduction is too long.'* We have cut the introduction by roughly a
third, removing [what]. …
**Comment 2.** *'The cohort definition is unclear.'* We have added to the Methods: '[new
sentence].' …
**Comment 3.** *'Figure 2 is hard to read.'* We have remade Figure 2 with [change]. …"

Why: each sub-point gets a quoted comment and a specific response. A single block reply hides
which point you actually addressed and which you skipped. Succinct means short answers, not
fewer comments.

---

## Three response skeletons

Pick the skeleton that matches the comment's substance; do not force every reply into one shape.

### A. Full agreement (most common)

> [quote the reviewer comment]

"We agree. We have [substantive change] in the [section]. The revised text reads:
'[new sentence verbatim].'"

Keep it to 2-5 sentences. Quote the new text; name the section; stop.

### B. Partial agreement with a bounded clarification

> [quote the reviewer comment]

"The reviewer raises a valid point about [aspect]. We have [the part you accept] in the
[section]. We have not [the part you decline] because [specific scientific reason]; instead,
we [the bounded alternative], which addresses the underlying concern without [the cost of the
full request]."

### C. Polite, evidence-backed rebuttal

> [quote the reviewer comment]

"We understand the concern that [restate the reviewer's point fairly]. We respectfully
maintain [your position] because [reasoning grounded in the data or design]. [If literature
supports you: 'This is consistent with (Author, Year), who showed ...'] To make this clearer
to readers, we have added a sentence to the [section]: '[new sentence].'"

Never open a rebuttal combatively, and never write "the reviewer is wrong." Frame disagreement
as a shared interest in getting the interpretation right.

### Graceful defer (when you decline an addition but want to leave the door open)

"We considered [the requested addition], but believe [the change already made] adequately
addresses the underlying concern without [the cost]. If the reviewer prefers, we are happy to
add [the requested item] in a subsequent revision."

This is a common, courteous move: it shows you took the request seriously, gives a reason, and
defers rather than flatly refusing.

### Response-letter head (R2+)

On R1 the editor gets a separate cover letter; on **R2+ there is no separate cover letter** — its
content becomes the head of the response letter. Keep the head to a short greeting, a
one-paragraph "in brief" summary of the principal changes, an optional one-sentence
companion/verification note, the single line stating that all quotations are from the revised
manuscript, and the signature. Then go straight to the point-by-point.

"Dear Dr. [Editor], thank you for the opportunity to revise our manuscript once more. In brief,
this revision [1-2 sentence change summary]. All quotations below are from the revised
manuscript; a point-by-point response follows. Sincerely, [First Author], on behalf of all
authors."

Reuse this head verbatim in any portal "cover letter" field rather than writing a second document.

---

## Meta-phrase to natural-expression conversion

| Editing-mechanism phrase | Natural reviewer-facing phrasing |
|---|---|
| "v2 adds one sentence at line 88" | "we have added to the [section]: '...'" |
| "softened six over-interpretive phrases" | "we have rephrased the over-interpretive passages so that ..." |
| "grep verification returned zero occurrences" | (delete entirely — describe the corrected wording instead) |
| "No further manuscript change" | "the existing text in the [section] already addresses this; the relevant statement is ..." |
| "the FIX-1 vocabulary cascade" | (delete — name the actual wording change) |
| "(Methods §X)" / "see §Discussion" | "in the Methods" / "in the Discussion" |
| "the internal supplementary index (not in the circulated bundle)" | (delete — never reference internal scaffolding) |
| "demoted the term throughout at all 13 locations" | "we have replaced [old term] with [new term] throughout the manuscript" |

---

## Location-pointer rule

- Point to changes by **section name** ("in the Methods, Statistical analysis subsection").
- A **revised-manuscript page and/or line number is acceptable and common** ("Methods, page 7,
  lines 177-178") *when those numbers refer to the revised manuscript the reviewer is reading*
  (state once at the top: "all page and line numbers refer to the revised manuscript"). This
  is a normal human convention, not an AI-tell.
- What is banned is the **internal markdown/draft line number** ("line 43" pointing into your
  working file) — it will not match the reviewer's PDF and reads as a diff log. If you cannot
  guarantee the line number matches the reviewer's manuscript, use the section name only.
- When the change is a sentence or two, **quote the new text** in quotation marks. This is
  concrete, verifiable, and reads as the work of an author who knows their own manuscript.
- For larger changes (a new paragraph, a restructured subsection), summarise the change in
  one sentence and point to the section; do not transcribe the whole block.

---

## Pre-submission checklist (response letter + cover letter)

- [ ] `grep -c "§"` = 0
- [ ] No internal line-number references ("(line NN)", "at lines N, M, ...")
- [ ] No version-prefixed change narration ("v2 adds...", "the v3 revision demotes...")
- [ ] No tooling/verification leak ("grep", "softened N phrases", "circulated bundle", internal FIX labels)
- [ ] No bare "No further manuscript change" stubs — each reply carries substance
- [ ] Openers varied across responses
- [ ] New manuscript text quoted verbatim where a sentence-level change was made
- [ ] Em dashes below threshold (see humanize Pattern 13)
- [ ] Ran `/humanize` on both documents; triage hits reviewed, confirmed patterns 22-24 instances = 0 (`§` = 0 hard)
- [ ] (R2+) No separate cover letter; its summary is folded into the response-letter head
- [ ] Satisfied reviewers answered in 1-2 sentences; no cross-reviewer lobbying or defensive meta-comments
- [ ] Multi-point reviewer paragraphs split into discrete, individually quoted comments
- [ ] Methodology disclosures folded into the relevant response, not a separate front section

#!/usr/bin/env python3
""""Your paper is too dense" is the one comment you cannot address by adding text.

The failure this gate catches, in the shape it takes:

    previous   as the reviewers saw it  ->  "too dense / shorten / move to supplement"
    revised    THE revision answering   ->  LONGER.  Every named term goes UP, because each
               those comments               comment is answered point-by-point, and point-by-point
                                            response REWARDS adding text.
    next       the cut version          ->  below where it started.  <- what the letter asked for.

Answering "your text is too dense" comment-by-comment makes it denser, because point-by-point
culture rewards showing you addressed each comment — and length is the one comment adding text
cannot address. The shrink then slips another round away.

This gate is pure arithmetic. If the reviewer comments contain a density/length complaint AND the
revised manuscript body did not get SHORTER than the previous version, the complaint was not
addressed — it was made worse. It fires on the longer revision immediately, and stays silent on the cut one.

It reads:
  --comments   the reviewer decision letter (where the complaint lives)
  --previous   the manuscript as the reviewers saw it   (the word count they complained about)
  --revised    the manuscript you are about to send back (must be shorter, if a complaint was made)

Body word count is measured the way a reviewer counts it: Introduction through Discussion, excluding
the title page, abstract, references, tables, figure legends, and — so a citation-heavy paragraph is
not scored as verbose — bracketed/parenthetical citation markers. It is a ratio check, so the exact
rule matters less than applying it identically to both versions, which it does.

Usage:
    check_density_complaint.py --comments letter.md --previous v_prev.docx --revised v_new.docx \
        [--out qc/density.json] [--strict]

Stdlib only (.docx via python-docx when present).
"""

from __future__ import annotations

import argparse
import json
import re
import sys
import unicodedata
from pathlib import Path

# The vocabulary a reviewer reaches for when a manuscript is too long or too dense. Each is a
# complaint that adding text cannot answer.
COMPLAINT = re.compile(
    r"\b(too (long|dense|detailed|verbose|wordy)|overly (long|detailed|dense|complex)|"
    r"shorten(ed|ing)?|condens(e|ed|ing)|trim(med|ming)?|cut down|tighten(ed|ing)?|"
    r"reduce (the )?(length|word count|detail)|"
    r"(mov\w+|belongs?).{0,30}?(to (the )?supplement|to supplementary|in (the )?supplement)|"
    r"excessive(ly)? (detail|length|long)|"
    r"(somewhat |very |quite |rather |overly )?(dense|verbose|wordy)\b|"
    r"(is|are|reads?|too) (very )?(repetitive|redundant)|"
    r"difficult to follow|hard to follow|dilut(e|es|ing) the (main )?message|"
    r"streamlin(e|ed|ing)|language editing|more concise|less dense|for (legibility|readability)|"
    r"repeat(ed|s|ing)? (multiple times|throughout|itself))\b",
    re.IGNORECASE,
)

# Section headings, in reviewer-count order. Body = first Introduction..Introduction-like heading
# through the end of Discussion / before References.
BODY_START = re.compile(r"^#{0,4}\s*\**\s*(introduction|background)\b", re.IGNORECASE | re.MULTILINE)
BODY_END = re.compile(r"^#{0,4}\s*\**\s*(references|acknowledg|funding|conflict|"
                      r"data availability|supplementary|supporting information|figure legends?)\b",
                      re.IGNORECASE | re.MULTILINE)
ABSTRACT = re.compile(r"^#{0,4}\s*\**\s*abstract\b", re.IGNORECASE | re.MULTILINE)

CITATION = re.compile(r"\[[\d,\s–\-]+\]|\((?:[A-Z][A-Za-z'`-]+(?: et al\.?)?,?\s*\d{4}[a-z]?;?\s*)+\)")


def read_text(path: Path) -> str:
    if path.suffix.lower() == ".docx":
        try:
            from docx import Document  # type: ignore
        except Exception as exc:  # pragma: no cover
            raise SystemExit(f"python-docx required to read {path}: {exc}")
        doc = Document(str(path))
        parts: list[str] = []

        def walk(tbl):
            for row in tbl.rows:
                for cell in row.cells:
                    for p in cell.paragraphs:
                        parts.append(p.text)
                    for t in cell.tables:
                        walk(t)

        for p in doc.paragraphs:
            parts.append(p.text)
        for t in doc.tables:
            walk(t)
        return "\n".join(parts)
    return path.read_text(encoding="utf-8", errors="replace")


def body_word_count(text: str) -> int:
    """Introduction..before-References, abstract excluded, citation markers removed."""
    text = unicodedata.normalize("NFKC", text)
    m0 = BODY_START.search(text)
    body = text[m0.start():] if m0 else text
    m1 = BODY_END.search(body)
    if m1:
        body = body[: m1.start()]
    # if an Abstract heading is inside our slice (no Introduction found), drop it
    if not m0:
        a = ABSTRACT.search(body)
        if a:
            nxt = BODY_START.search(body, a.end())
            if nxt:
                body = body[nxt.start():]
    body = CITATION.sub(" ", body)
    body = re.sub(r"^#{1,6}\s.*$", " ", body, flags=re.MULTILINE)  # drop heading lines themselves
    body = re.sub(r"[*_`>|#]", " ", body)                          # markdown punctuation
    words = re.findall(r"[A-Za-z0-9][\w'–-]*", body)
    return len(words)


def complaints(text: str) -> list[str]:
    """The density/length complaint sentences in the decision letter."""
    out: list[str] = []
    for sent in re.split(r"(?<=[.!?])\s+|\n", text):
        if COMPLAINT.search(sent):
            out.append(re.sub(r"\s+", " ", sent).strip()[:160])
    # dedupe preserving order
    seen, uniq = set(), []
    for s in out:
        if s.lower() not in seen:
            seen.add(s.lower()); uniq.append(s)
    return uniq


def main(argv: list[str] | None = None) -> int:
    ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
    ap.add_argument("--comments", required=True, type=Path, help="reviewer decision letter (.md/.txt/.docx)")
    ap.add_argument("--previous", required=True, type=Path, help="manuscript the reviewers saw (.md/.txt/.docx)")
    ap.add_argument("--revised", required=True, type=Path, help="manuscript you are about to send back")
    ap.add_argument("--out", type=Path, help="write JSON report here")
    ap.add_argument("--strict", action="store_true", help="exit 1 if the complaint was not addressed")
    ap.add_argument("--quiet", action="store_true")
    a = ap.parse_args(argv)

    for p in (a.comments, a.previous, a.revised):
        if not p.is_file():
            print(f"error: file not found: {p}", file=sys.stderr)
            return 2

    comp = complaints(read_text(a.comments))
    prev_wc = body_word_count(read_text(a.previous))
    new_wc = body_word_count(read_text(a.revised))
    delta = new_wc - prev_wc

    fired = bool(comp) and delta >= 0
    report = {
        "detector": "check_density_complaint",
        "density_complaints": comp,
        "previous_body_words": prev_wc,
        "revised_body_words": new_wc,
        "delta_words": delta,
        "verdict": "DENSITY_COMPLAINT_UNADDRESSED" if fired else "OK",
    }
    if a.out:
        a.out.parent.mkdir(parents=True, exist_ok=True)
        a.out.write_text(json.dumps(report, indent=2), encoding="utf-8")

    if not a.quiet:
        if not comp:
            print(f"OK: no density/length complaint in the decision letter "
                  f"(body {prev_wc} -> {new_wc} words, {delta:+d}).")
        elif not fired:
            print(f"OK: a density complaint was raised and the body got shorter "
                  f"({prev_wc} -> {new_wc} words, {delta:+d}).")
        else:
            print(f"DENSITY_COMPLAINT_UNADDRESSED: reviewers said the manuscript was too dense/long, "
                  f"and the body did not get shorter ({prev_wc} -> {new_wc} words, {delta:+d}).\n")
            for c in comp[:6]:
                print(f"  reviewer: {c}")
            print(
                "\n'Too dense' is the one comment you cannot address by adding text, and point-by-point\n"
                "response rewards adding it: answering each density comment individually is how a\n"
                "revision comes back longer than the version that drew the complaint. Cut, or move\n"
                "detail to the supplement — do not defend length by adding a paragraph that explains\n"
                "it."
            )
    return 1 if (fired and a.strict) else 0


if __name__ == "__main__":
    sys.exit(main())

#!/usr/bin/env python3
"""Response-letter claim <-> revised-manuscript verification (rule-backed gate).

A response-to-reviewers letter asserts concrete edits: "we added the sentence
'...'", "we now cite Tariq et al. [15]". The single-source-of-truth is the
*revised manuscript*, not the response prose — yet a claimed edit can be absent
from the body (a real incident: an added Discussion citation was described in the
response but never inserted, and both a reviewer round and the authors missed it
until a body grep). This gate makes that class deterministic for both sides:
`/revise` (author, before sending) and `/peer-review` (reviewer, verifying the
author's claims against the revised manuscript).

It is deliberately conservative — it verifies only claims carrying a strong,
checkable anchor, so paraphrase and honest rewording do not false-positive:

  * RESPONSE_QUOTE_UNVERIFIED (major) — the letter says specific text was
    added / inserted / "now reads" and quotes it verbatim, but that quoted text
    is absent from the revised manuscript body.
  * RESPONSE_QUOTE_UNRESOLVED (minor) — the quoted text IS there in order, but
    only once foreign tokens are allowed between its words, or a word or two is
    missing. That is the signature of a dirty extraction (a bled reference
    column, PDF line numbers, a footnote marker, a hyphen split across a line),
    not of an edit that was never made. Reported so a human looks; never counted
    as drift. A contiguous substring test cannot tell these apart and calls a
    correct quote absent — the failure that once came one step from having two
    accurate verbatim quotes deleted. Matching lives in _quote_match.py.
  * RESPONSE_CITATION_UNVERIFIED (major) — the letter says a citation was added
    / "now cite(d)", but none of the cited tokens ([N] / [@key] / Author et al.)
    appear in the revised manuscript body.

Vague claims with no quote and no citation ("we clarified the Methods") are not
verifiable and are intentionally NOT flagged. Reviewer-comment blockquotes
(lines beginning with '>') are excluded so the reviewer's own quoted text is
never mistaken for an author addition.

Usage:
    check_response_claims.py --response response.md --manuscript revised.md [--strict]
    check_response_claims.py --response r.md --manuscript revised.docx --out qc/response_claims.json

Exit 0 when every anchored claim is verified (or none exist). With --strict,
exit 1 if any major verdict fires. Stdlib only; .docx read via python-docx when
available.
"""

from __future__ import annotations

import argparse
import json
import re
import sys
import unicodedata
from pathlib import Path

sys.path.insert(0, str(Path(__file__).resolve().parent))
from _quote_match import match_quality  # noqa: E402  (same-dir helper)

# A claim that asserts an addition/edit to the manuscript.
CLAIM_VERB = re.compile(
    r"\b("
    r"added the (?:sentence|statement|clause|text|following|phrase)|"
    r"added a (?:sentence|statement|clause|citation|reference|paragraph)|"
    r"we (?:have )?added|have added|now added|"
    r"inserted|included the (?:sentence|statement|text|citation|reference)|"
    r"now (?:reads|read|states|state)|"
    r"(?:revised|changed|reworded|rephrased|amended) [^.\n]{0,60}? to (?:read|state)|"
    r"now cites?|now cited|we (?:now )?cite|added (?:the )?(?:citation|reference)s?"
    r")\b",
    re.IGNORECASE,
)

# Quoted string: straight or curly, >= 12 chars (a sentence-like assertion).
QUOTE = re.compile(r"[\"“‘']([^\"“”‘’']{12,600})[\"”’']")

# Citation tokens claimed as added.
CIT_NUMERIC = re.compile(r"\[(\d{1,3}(?:\s*[,–-]\s*\d{1,3})*)\]")
CIT_BIBKEY = re.compile(r"\[@([A-Za-z0-9_:.\-]+)\]")
CIT_AUTHOR = re.compile(r"\b([A-Z][A-Za-zÀ-ſ'-]{2,})\s+et\s+al\.?")

WINDOW = 320  # chars after a claim verb to look for its object


def read_text(path: Path) -> str:
    """Return plain text from .md/.txt or .docx (recursive paragraphs + tables)."""
    if path.suffix.lower() == ".docx":
        try:
            from docx import Document  # type: ignore
            from docx.document import Document as _Doc  # noqa: F401
        except Exception as exc:  # pragma: no cover
            raise SystemExit(f"python-docx required to read {path}: {exc}")
        doc = Document(str(path))
        parts: list[str] = []

        def walk_table(tbl):
            for row in tbl.rows:
                for cell in row.cells:
                    for p in cell.paragraphs:
                        parts.append(p.text)
                    for t in cell.tables:
                        walk_table(t)

        for p in doc.paragraphs:
            parts.append(p.text)
        for t in doc.tables:
            walk_table(t)
        return "\n".join(parts)
    return path.read_text(encoding="utf-8", errors="replace")


def normalize(s: str) -> str:
    """Casefold + collapse whitespace + strip markdown emphasis for substring match."""
    s = unicodedata.normalize("NFKC", s)
    s = s.replace("’", "'").replace("‘", "'")
    s = s.replace("“", '"').replace("”", '"')
    s = re.sub(r"[*_`]", "", s)  # markdown emphasis / code ticks
    s = re.sub(r"\s+", " ", s)
    return s.casefold().strip()


def strip_response_blockquotes(text: str) -> str:
    """Drop reviewer-comment blockquote lines (>) so their quotes aren't scanned."""
    keep = [ln for ln in text.splitlines() if not ln.lstrip().startswith(">")]
    return "\n".join(keep)


def extract_claims(response: str):
    """Yield (kind, anchor, context) for anchored addition claims in Response prose."""
    prose = strip_response_blockquotes(response)
    claims = []
    for m in CLAIM_VERB.finditer(prose):
        start = m.start()
        window = prose[start : start + WINDOW]
        ctx = re.sub(r"\s+", " ", prose[max(0, start - 20) : start + 120]).strip()
        # quoted additions
        for q in QUOTE.finditer(window):
            text = q.group(1).strip()
            if len(text.split()) >= 4:
                claims.append(("quote", text, ctx))
        # citation additions
        cits = []
        for cm in CIT_NUMERIC.finditer(window):
            for n in re.split(r"[,–-]", cm.group(1)):
                if n.strip():
                    cits.append(("num", n.strip()))
        for cm in CIT_BIBKEY.finditer(window):
            cits.append(("key", cm.group(1)))
        for cm in CIT_AUTHOR.finditer(window):
            cits.append(("author", cm.group(1)))
        # only treat a verb as a citation claim if the verb itself is citation-ish
        if cits and re.search(r"cit|reference", m.group(0), re.IGNORECASE):
            claims.append(("citation", cits, ctx))
    return claims


def grade_quote(body: str, quote: str) -> dict:
    """Grade the quote's presence in the body via the extraction-tolerant matcher.

    A contiguous search alone is not safe here: the manuscript may arrive as a .docx whose
    extraction wedges a footnote marker, a line number, or a bled column of reference text
    into the middle of the very sentence being checked. Those quotes are present and correct,
    and a substring test calls them absent — the failure that once nearly had two accurate
    verbatim quotes deleted. See _quote_match.py."""
    return match_quality(quote, body)


def body_has_citation(body: str, norm_body: str, cits) -> bool:
    """True if ANY cited token appears in the body (conservative: any-match passes)."""
    for kind, tok in cits:
        if kind == "num" and re.search(r"\[\s*\d*[,\s–-]*" + re.escape(tok) + r"\b", body):
            return True
        if kind == "num" and ("[" + tok + "]") in body:
            return True
        if kind == "key" and ("@" + tok) in body:
            return True
        if kind == "author" and normalize(tok) in norm_body:
            return True
    return False


def build_report(response_path: Path, manuscript_path: Path) -> dict:
    response = read_text(response_path)
    body = read_text(manuscript_path)
    norm_body = normalize(body)
    findings = []
    for kind, anchor, ctx in extract_claims(response):
        if kind == "quote":
            g = grade_quote(body, anchor)
            if g["grade"] == "INTERLEAVED":
                findings.append(
                    {
                        "verdict": "RESPONSE_QUOTE_UNRESOLVED",
                        "severity": "minor",
                        "claimed_text": anchor,
                        "context": ctx,
                        "match": g,
                        "message": (
                            f"Every word of the quoted text appears in order, but with {g['inserted']} "
                            "foreign token(s) wedged in — consistent with a dirty extraction (a bled "
                            "reference column, line numbers, a footnote marker), not a missing edit. "
                            "Confirm by eye; do not treat as absent."
                        ),
                    }
                )
            elif g["grade"] == "PARTIAL":
                findings.append(
                    {
                        "verdict": "RESPONSE_QUOTE_UNRESOLVED",
                        "severity": "minor",
                        "claimed_text": anchor,
                        "context": ctx,
                        "match": g,
                        "message": (
                            f"{g['matched']} of {g['total']} words of the quoted text appear in order "
                            "— enough to be the same sentence damaged in extraction (a hyphen split "
                            "across a line, a dropped glyph) rather than an edit that was never made. "
                            "Confirm by eye."
                        ),
                    }
                )
            elif g["grade"] == "ABSENT":
                findings.append(
                    {
                        "verdict": "RESPONSE_QUOTE_UNVERIFIED",
                        "severity": "major",
                        "claimed_text": anchor,
                        "context": ctx,
                        "match": g,
                        "message": "Response quotes added text that is absent from the revised manuscript body.",
                    }
                )
        elif kind == "citation":
            if not body_has_citation(body, norm_body, anchor):
                findings.append(
                    {
                        "verdict": "RESPONSE_CITATION_UNVERIFIED",
                        "severity": "major",
                        "claimed_citation": [t for _, t in anchor],
                        "context": ctx,
                        "message": "Response claims a citation was added but none of the cited tokens appear in the revised manuscript body.",
                    }
                )
    n_major = sum(1 for f in findings if f["severity"] == "major")
    return {
        "response": str(response_path),
        "manuscript": str(manuscript_path),
        "findings": findings,
        "summary": {"major": n_major, "unresolved": len(findings) - n_major},
        # An UNRESOLVED quote is a "look at this", not a defect: the words are demonstrably
        # there and only the extraction is suspect. Safety therefore turns on MAJOR findings,
        # which is what --strict has always documented.
        "submission_safe": n_major == 0,
    }


def main(argv: list[str] | None = None) -> int:
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("--response", required=True, type=Path, help="response-to-reviewers letter (.md/.txt/.docx)")
    ap.add_argument("--manuscript", required=True, type=Path, help="revised manuscript (.md/.txt/.docx)")
    ap.add_argument("--out", type=Path, help="write JSON report here")
    ap.add_argument("--strict", action="store_true", help="exit 1 if any major verdict fires")
    ap.add_argument("--quiet", action="store_true")
    args = ap.parse_args(argv)

    for p in (args.response, args.manuscript):
        if not p.is_file():
            print(f"error: file not found: {p}", file=sys.stderr)
            return 2

    report = build_report(args.response, args.manuscript)
    if args.out:
        args.out.parent.mkdir(parents=True, exist_ok=True)
        args.out.write_text(json.dumps({"detector": "check_response_claims", **report}, indent=2, ensure_ascii=False), encoding="utf-8")

    if not args.quiet:
        s = report["summary"]
        if not report["findings"]:
            print("OK: every anchored response claim is verified against the revised manuscript.")
        else:
            print(f"RESPONSE_CLAIM findings — {s['major']} major, {s['unresolved']} unresolved:")
            for f in report["findings"]:
                anchor = f.get("claimed_text") or ", ".join(f.get("claimed_citation", []))
                print(f"  [{f['verdict']}] ({f['severity']}) {anchor!r}")
                print(f"      near: {f['context']}")
                if f["severity"] == "minor":
                    print(f"      {f['message']}")
            if s["major"] == 0:
                print("\nNo major drift: the unresolved item(s) are extraction-quality doubts, "
                      "not claims of an edit that was never made.")

    if args.strict and not report["submission_safe"]:
        print("\nRESPONSE_CLAIM_UNVERIFIED: a response-letter claim is not reflected in the revised manuscript.", file=sys.stderr)
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(main())

"""Quote matching that survives an extraction layer — the substrate under quote gates.

WHY THIS EXISTS (the failure it removes)

Verifying "the manuscript contains this quoted sentence" by searching a CONTIGUOUS string
is wrong whenever the haystack came out of an extractor, because extractors interleave
tokens the source never had. In one submission-day session that single assumption produced
thirteen false positives, all the same shape:

  * a two-column PDF bled reference-list text into the middle of a sentence
    ("learners form independent" | "civile." | "assessments before seeing AI output");
  * a line-numbered supplement PDF put the line number inside the sentence
    ("were" | "86" | "performed");
  * superscript markers and footnote references landed mid-clause;
  * hyphenation across a line break split one word into two ("assess-" + "ments").

Every one of those quotes was CORRECT and present. The contiguous check called them absent.
It came within one step of instructing an author to delete two accurate verbatim quotes.

THE RULE THIS ENCODES

A quote that cannot be matched contiguously is not thereby "not in the source". It is
UNRESOLVED until something stronger says otherwise. So this module grades a match instead
of answering yes/no:

  EXACT        the normalized quote is a contiguous substring — verified, no doubt.
  INTERLEAVED  every quote token appears IN ORDER, with only a bounded number of foreign
               tokens wedged between them — the text is there and the extraction is dirty.
  PARTIAL      most quote tokens appear in order but some are missing — consistent with
               extraction damage (hyphen splits, dropped glyphs); too weak to call absent.
  ABSENT       not even a partial ordered run — the text really is not there.

Only ABSENT justifies a "you claimed an edit you did not make" verdict. INTERLEAVED and
PARTIAL are reported as unresolved so a human looks, rather than as a defect.

WHY THE GAPS ARE BOUNDED (the precision that makes this safe)

An unbounded subsequence match is worthless: the tokens of almost any short sentence appear
"in order" somewhere in a long document if you allow arbitrary distance. The bound that works
is not a token budget but an INTERRUPTION COUNT, because the two cases differ in shape:

    a real extraction artifact interrupts a sentence once or twice, and each interruption can
    be long (a bled reference line is a dozen tokens);

    a spurious "match" interrupts at nearly every token, each time by a little.

So the limits are: at most MAX_GAP foreign tokens at any single join, at most
MAX_INTERRUPTIONS joins that are interrupted at all, and a total-insertion sanity cap. A
quote whose words are scattered one-by-one across a Discussion section needs an interruption
at every join and fails, while a quote split once by a column bleed passes.

Not a detector: a helper imported by the gates that need it (leading underscore keeps it out
of the detector catalog glob). Stdlib only.
"""

from __future__ import annotations

import re
import unicodedata

# At most this many foreign tokens may sit at ONE join. A bled reference line ("civile. Rev
# Med Suisse 2019;15:1122.") is around a dozen tokens; a running header a handful.
MAX_GAP = 25
# At most this many joins may be interrupted AT ALL. This is the limit that separates a dirty
# extraction (one or two interruptions) from a spurious scatter (an interruption per token).
MAX_INTERRUPTIONS = 4
# Sanity cap on total foreign tokens, so a short quote cannot absorb an entire paragraph.
MAX_TOTAL_INSERT_FRAC = 5.0
MIN_TOTAL_INSERT = 20
# A PARTIAL match must still account for this share of the quote's tokens; below it, ABSENT.
PARTIAL_COVERAGE = 0.80

_TOKEN_RE = re.compile(r"[0-9a-z]+(?:'[a-z]+)?", re.IGNORECASE)


def normalize(s: str) -> str:
    """Casefold, unify quotes/dashes, drop markdown emphasis, repair line-break hyphenation,
    and collapse whitespace. Hyphenation repair matters: an extractor that wraps "assess-
    ments" across a line otherwise destroys the token the quote is looking for."""
    s = unicodedata.normalize("NFKC", s)
    s = s.replace("’", "'").replace("‘", "'")
    s = s.replace("“", '"').replace("”", '"')
    # join a word split by a hyphen at a line break: "assess-\n  ments" -> "assessments"
    s = re.sub(r"(\w)[-‐‑]\s*\n\s*(\w)", r"\1\2", s)
    s = re.sub(r"[*_`]", "", s)
    s = re.sub(r"\s+", " ", s)
    return s.casefold().strip()


def tokens(s: str) -> list[str]:
    """Normalized word/number tokens. Punctuation is dropped, so an injected '.' or a stray
    bracket never breaks a match on its own."""
    return _TOKEN_RE.findall(normalize(s))


def _ordered_run(needle: list[str], hay: list[str], allow_missing: bool):
    """Best ordered match of `needle` inside `hay`.

    Walks every candidate start and consumes needle tokens in order, skipping at most
    MAX_GAP foreign tokens per join, at most MAX_INTERRUPTIONS interrupted joins, and a
    total-insertion sanity cap. With allow_missing, a needle token that cannot be found
    within the gap window is skipped (counted as missing) instead of failing the run.

    Returns (matched_count, inserted_count) for the best run, or (0, 0)."""
    if not needle or not hay:
        return (0, 0)
    budget = max(MIN_TOTAL_INSERT, int(len(needle) * MAX_TOTAL_INSERT_FRAC))
    max_missing = len(needle) - int(len(needle) * PARTIAL_COVERAGE)
    best = (0, 0)
    first = needle[0]
    starts = [i for i, t in enumerate(hay) if t == first]
    if allow_missing and not starts:
        # the opening token itself may be the damaged one — try any token of the quote
        wanted = set(needle)
        starts = [i for i, t in enumerate(hay) if t in wanted]
    for start in starts:
        hi = start
        matched = inserted = missing = interruptions = 0
        for tok in needle:
            found = -1
            for j in range(hi, min(hi + MAX_GAP + 1, len(hay))):
                if hay[j] == tok:
                    found = j
                    break
            if found < 0:
                if not allow_missing:
                    break
                missing += 1
                if missing > max_missing:
                    break
                continue
            gap = found - hi
            if gap:
                interruptions += 1
                if interruptions > MAX_INTERRUPTIONS:
                    break
            inserted += gap
            if inserted > budget:
                break
            matched += 1
            hi = found + 1
        if matched > best[0]:
            best = (matched, inserted)
        if matched == len(needle):
            break
    return best


def match_quality(quote: str, haystack: str) -> dict:
    """Grade how well `quote` is present in `haystack`.

    Returns {"grade": EXACT|INTERLEAVED|PARTIAL|ABSENT, "matched", "total", "inserted",
             "coverage"}. Only ABSENT means "this text is not in the document"."""
    nq, nh = normalize(quote), normalize(haystack)
    q_tok = tokens(quote)
    total = len(q_tok)
    if total == 0:
        return {"grade": "ABSENT", "matched": 0, "total": 0, "inserted": 0, "coverage": 0.0}
    if nq and nq in nh:
        return {"grade": "EXACT", "matched": total, "total": total, "inserted": 0, "coverage": 1.0}

    h_tok = tokens(haystack)
    matched, inserted = _ordered_run(q_tok, h_tok, allow_missing=False)
    if matched == total:
        return {"grade": "INTERLEAVED", "matched": matched, "total": total,
                "inserted": inserted, "coverage": 1.0}

    matched, inserted = _ordered_run(q_tok, h_tok, allow_missing=True)
    coverage = matched / total
    grade = "PARTIAL" if coverage >= PARTIAL_COVERAGE else "ABSENT"
    return {"grade": grade, "matched": matched, "total": total,
            "inserted": inserted, "coverage": round(coverage, 3)}

#!/usr/bin/env bash
# Regression test for skills/revise/scripts/check_response_claims.py — the
# response-letter <-> revised-manuscript verification gate. Confirms an anchored
# claim absent from the body fails under --strict, a present one passes, and the
# false-positive guards (vague claims, reviewer blockquotes) do not fire.
set -u

REPO_ROOT="$(cd "$(dirname "$0")/../../.." && pwd)"
V="$REPO_ROOT/skills/revise/scripts/check_response_claims.py"
TMP="$(mktemp -d)"
trap 'rm -rf "$TMP"' EXIT

pass=0
fail=0
ck() {
  local label="$1" expected="$2" actual="$3"
  if [ "$expected" = "$actual" ]; then
    printf '  PASS  %-52s exit=%s\n' "$label" "$actual"
    pass=$((pass + 1))
  else
    printf '  FAIL  %-52s expected=%s actual=%s\n' "$label" "$expected" "$actual"
    fail=$((fail + 1))
  fi
}

# --- manuscript that DOES contain the added sentence + citation ---
cat > "$TMP/body_good.md" <<'MD'
## Methods
Diabetes was defined by a fasting glucose of at least 126 mg/dL or medication use.

## Discussion
Dosing errors are a recognized hazard in this setting, as Tariq et al. [15] reported.
MD

# --- manuscript that is MISSING both ---
cat > "$TMP/body_bad.md" <<'MD'
## Methods
Baseline characteristics were summarized descriptively.

## Discussion
The findings are consistent with prior work.
MD

# --- response letter with an anchored quote claim + a citation claim ---
cat > "$TMP/response.md" <<'MD'
**Comment 1.**
> The Methods do not define diabetes.

**Response 1.** Thank you. We added the sentence "Diabetes was defined by a fasting glucose of at least 126 mg/dL or medication use." to the Methods.

**Comment 2.**
> Please acknowledge dosing-error risk.

**Response 2.** We now cite Tariq et al. [15] in the Discussion.
MD

# --- response with only a VAGUE claim (no quote, no citation) ---
cat > "$TMP/response_vague.md" <<'MD'
**Response.** We clarified the Methods and revised the Introduction for readability.
MD

# --- response whose ONLY unverifiable quote is inside a reviewer blockquote ---
cat > "$TMP/response_reviewerquote.md" <<'MD'
**Comment 1.**
> The authors claim "a mortality reduction of ninety percent" without support.

**Response 1.** We have tempered this statement and now report the observed range only.
MD

# 1) anchored claims absent from body -> exit 1 (--strict)
python3 "$V" --response "$TMP/response.md" --manuscript "$TMP/body_bad.md" --strict > /dev/null 2>&1
ck "missing added quote + citation fails (--strict)" 1 "$?"

# 2) same claims present in body -> exit 0
python3 "$V" --response "$TMP/response.md" --manuscript "$TMP/body_good.md" --strict > /dev/null 2>&1
ck "verified quote + citation passes (--strict)" 0 "$?"

# 3) vague claim (no anchor) -> not flagged, exit 0
python3 "$V" --response "$TMP/response_vague.md" --manuscript "$TMP/body_bad.md" --strict > /dev/null 2>&1
ck "vague unanchored claim not flagged" 0 "$?"

# 4) reviewer-blockquote quote (not an author addition) -> not flagged, exit 0
python3 "$V" --response "$TMP/response_reviewerquote.md" --manuscript "$TMP/body_bad.md" --strict > /dev/null 2>&1
ck "reviewer blockquote quote not flagged" 0 "$?"

# 5) drift reported but tolerated without --strict -> exit 0
python3 "$V" --response "$TMP/response.md" --manuscript "$TMP/body_bad.md" > /dev/null 2>&1
ck "drift tolerated without --strict" 0 "$?"

# 6) the flagged verdicts are the expected two
OUT="$(python3 "$V" --response "$TMP/response.md" --manuscript "$TMP/body_bad.md" 2>&1)"
echo "$OUT" | grep -q RESPONSE_QUOTE_UNVERIFIED && echo "$OUT" | grep -q RESPONSE_CITATION_UNVERIFIED
ck "both expected verdicts present" 0 "$?"

# --- extraction tolerance: a CORRECT quote must survive a dirty extraction ---------------
# Each manuscript below really does contain the claimed sentence; the variants are what an
# extractor emits, not what the author wrote. A contiguous substring test calls every one of
# them "absent" — the false-positive class that once nearly had accurate quotes deleted.
cat > "$TMP/resp_quote.md" <<'MD'
**Response 1.** We added the sentence "learners form independent assessments before seeing AI output" to the Discussion.
MD

# (a) two-column PDF: a reference line bled into the middle of the sentence
cat > "$TMP/body_bleed.md" <<'MD'
## Discussion
We note that learners form independent civile. Rev Med Suisse 2019;15:1122. assessments before seeing AI output.
MD
# (b) line-numbered supplement PDF: line numbers sit inside the sentence
cat > "$TMP/body_linenum.md" <<'MD'
## Discussion
86 We note that learners form 87 independent assessments 88 before seeing AI output.
MD
# (c) a superscript / footnote marker landed mid-clause
cat > "$TMP/body_supersc.md" <<'MD'
## Discussion
learners form independent 3 assessments before seeing AI output
MD
# (d) hyphenation across a line break split one word in two
cat > "$TMP/body_hyphen.md" <<'MD'
## Discussion
learners form independent assess-
ments before seeing AI output
MD

for variant in bleed linenum supersc hyphen; do
  python3 "$V" --response "$TMP/resp_quote.md" --manuscript "$TMP/body_$variant.md" --strict > /dev/null 2>&1
  ck "dirty extraction ($variant) is not drift (--strict)" 0 "$?"
done

# the interleaved variants must SAY so (unresolved), not pass silently
for variant in bleed linenum supersc; do
  python3 "$V" --response "$TMP/resp_quote.md" --manuscript "$TMP/body_$variant.md" 2>&1 \
    | grep -q RESPONSE_QUOTE_UNRESOLVED
  ck "dirty extraction ($variant) reports UNRESOLVED" 0 "$?"
done

# the hyphen split is repaired outright -> no finding at all
python3 "$V" --response "$TMP/resp_quote.md" --manuscript "$TMP/body_hyphen.md" 2>&1 \
  | grep -q RESPONSE_QUOTE
ck "line-break hyphenation repaired (no finding)" 1 "$?"

# PRECISION GUARD: tolerance must not excuse a genuine miss. The words below appear in
# order but scattered a paragraph apart — that is not the claimed sentence.
{
  echo "## Discussion"
  echo "Some learners were enrolled."
  for _ in $(seq 1 12); do echo "The study reported outcomes across sites and years."; done
  echo "We form working groups."
  for _ in $(seq 1 12); do echo "The study reported outcomes across sites and years."; done
  echo "An independent committee met."
  for _ in $(seq 1 12); do echo "The study reported outcomes across sites and years."; done
  echo "Their assessments were filed before seeing AI output."
} > "$TMP/body_scattered.md"
python3 "$V" --response "$TMP/resp_quote.md" --manuscript "$TMP/body_scattered.md" --strict > /dev/null 2>&1
ck "scattered words are still MAJOR drift (--strict)" 1 "$?"

echo "----"
echo "test_response_claims: $pass passed, $fail failed"
[ "$fail" -eq 0 ]