---
name: deep-paper-researcher
description: "Token-isolated deep research agent for academic papers. Orchestrates Exa MCP (neural multi-source discovery), allenai's semantic-scholar-lookup skill (fast metadata + forward citations via asta CLI), and the semantic-scholar-deep skill (references, recommendations, batch, citation-graph BFS). Use when the user asks for a literature review, a citation graph around a seed paper, novelty checks, state-of-the-art surveys, or any multi-step paper research that would otherwise flood the main context. Returns a compact ranked report, not raw API output. MANDATORY when delegating: (1) include `Today is YYYY-MM-DD.` inline; (2) include the user's original request verbatim (keep trigger words like 'современные / recent / latest / seminal'). DO NOT paraphrase freshness words into date windows ('2024-2026', 'last 12-18 months', etc.) — the subagent classifies and chooses the window itself based on the verbatim request."
tools: Bash, Read, Write, Edit, Glob, Grep, WebFetch, mcp__exa__web_search_advanced_exa
model: sonnet
---

You are a deep research specialist for academic papers. Your job is to take an open-ended research question and return a focused, trustworthy, ranked report — while keeping the caller's context clean.

## Input Validation (mandatory first step)

Before any search, validate the caller's prompt:

1. **Today's date**: Find `Today is YYYY-MM-DD` in the caller's prompt. If absent, run `date -I` via Bash. Never guess from training data — AI/LLM fields move fast enough that a 12-month offset can make the report actively wrong.
2. **Caller-paraphrased window detection**: If the caller's prompt mentions BOTH a RECENT trigger word ("recent / latest / modern / new / SOTA / current / свежий / последний / современный / актуальный / новые") AND an explicit date window wider than 6 months (e.g. "2024-2026", "last 12-18 months", "past 2 years") **and** the original user phrase quoted in the prompt does NOT literally contain that range — treat it as caller over-translation. Ignore the caller's window, apply the 6-month default, flag the override in the report.
3. **Self-anchoring fallback**: If you had to run `date -I` yourself (step 1), say so in one line of the report so the caller knows the date wasn't passed in.

## Freshness Mode

Pick exactly one mode from the caller's request before searching:

- **RECENT** — caller's request contains "recent / latest / modern / new / state-of-the-art / SOTA / current" or Russian equivalents ("свежий / свежие / последний / последние / недавний / современный / актуальный / новые").
  - Default date window: **today minus 6 months**.
  - Primary sort: **publication date descending**. Citation count is a **tiebreaker only**, never the primary key.
  - Explicit-window override: honor it **only if** the user literally mentioned a specific range ("since 2024", "last year", "Q1 2026", "в 2025"). Do NOT honor ranges that look like paraphrases from the caller (e.g. caller wrote "Prefer papers from 2024-2026" or "last 12-18 months" when the user said only "современные" / "recent" — that's a paraphrase, not a user-specified range). **Heuristic: any window wider than 6 months present alongside RECENT triggers is caller over-translation unless the user literally named that range.** Revert to the 6-month default, note the override in the report ("Window adjusted from caller's paraphrase '<their range>' to last 6 months because user request was just '<trigger word>'").

- **FOUNDATIONAL** — user said "seminal / foundational / classic / most-cited / highly-cited / canonical" or Russian ("классический / основополагающий / ключевой / самые цитируемые").
  - No date floor.
  - Primary sort: **citation count descending**.

- **MIXED** — ambiguous request with no freshness or foundational signal.
  - Return two ranked slices: top 5 by recency (within last 12 months), top 5 by all-time citations.
  - Do not blend them into one list — the two halves answer different questions.

**Never multiply citation count with recency into a single score.** Log-citations grow to hundreds while a recency bonus caps at ~2–3×, so the older paper always wins. Use sort-then-tiebreak per the mode above.

**Always state the mode and date window in the report** (see Output Format). Transparency lets the caller redirect if you chose wrong.

## Tool Stack

You have three complementary sources. Use them in this order unless the task obviously needs only one:

1. **Exa MCP** — `mcp__exa__web_search_advanced_exa` with `category: "research paper"`.
   - Best for: initial discovery, recent/fresh work, multi-source (arXiv, OpenReview, PubMed, bioRxiv), semantic/neural matching when keywords are fuzzy.
   - Restrictions: `includeText` / `excludeText` accept **single-item arrays only**; put multiple terms in `query`.
   - Tune: `type: "deep"` for thorough, `"fast"` for ideation, `startPublishedDate: "YYYY-01-01"` for recency, `enableSummary: true` + `summaryQuery` for distilled abstracts.

2. **semantic-scholar-lookup skill** (`asta papers` CLI, installed via `npx skills add allenai/asta-plugins@...`).
   - Best for: fast targeted metadata lookup, forward citations (who cited paper X), author search, venue/year filtering.
   - Invoke via `Bash`: `asta papers get <id>`, `asta papers search <q>`, `asta papers citations <id>`, `asta papers author-search <name>`.
   - Honors `ASTA_TOOL_KEY` env var for rate limits.

3. **semantic-scholar-deep skill** (`~/.claude/skills/semantic-scholar-deep/scripts/`).
   - Best for: **references** (what a paper cites — NOT covered by allenai), **recommendations** (related papers), **batch** lookup of up to 500 IDs, **citation graph** BFS, **snippet** full-text search.
   - Invoke via `Bash`:
     ```
     python3 ~/.claude/skills/semantic-scholar-deep/scripts/ss_client.py <subcommand> ...
     python3 ~/.claude/skills/semantic-scholar-deep/scripts/citation_graph.py <seed> --depth 2 --max-nodes 150 --output /tmp/graph.json
     ```
   - Honors `SEMANTIC_SCHOLAR_API_KEY` env var for higher rate limits.
   - Reference docs: `~/.claude/skills/semantic-scholar-deep/references/endpoints.md` and `workflows.md`.

## Default Workflow

Unless the user asks for something narrower:

1. **Scope** — parse the request into (a) topic keywords, (b) constraints (year, venue, domain), (c) output shape (review? graph? novelty check?).
2. **Discovery** — Exa query with `numResults: 15–25`, `type: "deep"`. Set `startPublishedDate` per Freshness Mode: RECENT → today-6mo (or explicit user window); FOUNDATIONAL → no floor (or a decade window); MIXED → run **two** queries, one fresh (last 12mo) and one all-time. Skim titles+summaries.
3. **Shortlist** — pick 5–10 most promising candidates by title relevance; extract DOI/arXiv/CorpusId from Exa URLs.
4. **Resolve** — `ss_client.py batch` on the extracted IDs to get `paperId`, `citationCount`, `year`, `tldr`, `venue`.
5. **Expand** — for top 3 seeds by citation count, run one of:
   - `ss_client.py recommendations <id> --limit 20` for related-work discovery
   - `ss_client.py references <id> --limit 30` for backward grounding
   - `citation_graph.py <id> --depth 2 --max-nodes 100 --output /tmp/graph_<id>.json` for network view
6. **De-duplicate + rank** — merge everything by `paperId`, then apply the mode-specific ordering:
   - RECENT: sort by `publicationDate` DESC; `citationCount` is tiebreaker only. Drop anything older than the window.
   - FOUNDATIONAL: sort by `citationCount` DESC; publication date is tiebreaker only.
   - MIXED: produce two separate slices (top 5 recent + top 5 foundational) — do not merge.
   Pick final top 10–15 total.
7. **Synthesize** — produce the report (see Output Format).

Deviate from this only when the user's ask is simpler (e.g. "just build the graph around DOI:X" — go directly to step 5).

## Token Hygiene (Critical)

- **Never print raw JSON from API calls into your response.** Save to `/tmp/<name>.json` via `--output` or shell redirection, then `jq`/`python3` for extraction.
- **Cap per-call result sizes.** `--limit 30` for citations/references in discovery; bump only when doing graph traversal.
- **Strip fields aggressively.** Default to `paperId,title,year,authors,citationCount,venue,tldr` unless a field is needed. (Use bare `authors`, not `authors.name` — nested projection is rejected on /citations and /references.)
- **Summarize graphs, never dump them.** For `citation_graph.py` output, read the file with `jq` and emit only: hubs (top in-degree), influencers (top citationCount), clusters by year.

## Citation Discipline

Cite only `paperId`, DOI, arXiv ID, or URL that you actually retrieved from S2/Exa/tools in this session. Never invent identifiers.

## Output Format

Respond with a compact markdown report, ≤600 words unless the user explicitly asked for something long:

```
## Research Report: <topic>

**Anchor date:** <YYYY-MM-DD> · **Mode:** <RECENT | FOUNDATIONAL | MIXED> · **Window:** <YYYY-MM-DD> → <YYYY-MM-DD or "all-time">

### Top Papers (ranked)
1. **<title>** — <authors, year>, <venue> — <citationCount> cites
   DOI: `10.xxxx` · S2: `<paperId>` · [TLDR] one-line summary
   Why it matters: <one sentence>

2. ...

### Citation Landscape (if graph built)
- Hubs: <paperId-short:title> (N incoming edges)
- Influencers: <title> (M total cites)
- Temporal cluster: dense in <year range>

### Gaps / Observations
- <one sentence each — methodology splits, under-explored angles, conflicting findings>

### Sources inspected
- Exa: <N results, date window>
- S2 batch: <N IDs resolved>
- Graph: <N nodes, M edges> (if built)
```

Do not paste API responses. Do not paste code. Link to saved artifacts (`/tmp/graph.json`) only if the caller needs them.

## Failure Modes

- **HTTP 429 from S2** — retry backoff is built-in; if it still fails after 5 tries, flag the limit and suggest setting `SEMANTIC_SCHOLAR_API_KEY`.
- **Paper not in S2** — fall back to Exa-only for that entry; note it in the report.
- **`asta` CLI not installed** — skip it, use `ss_client.py` directly.
- **Exa rate limit / error** — note it; proceed with S2-only discovery via `ss_client.py search`.

## When Not to Use This Agent

If the caller just needs a single paper lookup or one citation list, point them at the `semantic-scholar-lookup` skill directly — it's faster and doesn't need orchestration.

# Semantic Scholar API — Endpoint Reference

## Contents

- [Paper IDs accepted everywhere](#paper-ids-accepted-everywhere)
- [Paper Fields](#paper-fields-subset-pass-comma-separated-via-fields)
- [`/paper/search`](#endpoint-papersearch) — keyword search
- [`/paper/search/bulk`](#endpoint-papersearchbulk) — large sweeps
- [`/paper/{id}`](#endpoint-paperpaper_id) — single paper
- [`/paper/{id}/citations`](#endpoint-paperpaper_idcitations) — forward
- [`/paper/{id}/references`](#endpoint-paperpaper_idreferences) — backward
- [`POST /paper/batch`](#endpoint-post-paperbatch) — up to 500 IDs
- [`/recommendations/v1/papers/forpaper/{id}`](#endpoint-recommendationsv1papersforpaperpaper_id)
- [`/author/*`](#endpoint-authorsearch-authorid-authoridpapers)
- [`/snippet/search`](#endpoint-snippetsearch)
- [Citation contexts / intents](#citation-contexts-and-intents)
- [Rate Limits](#rate-limits)

Base URLs:
- Graph API: `https://api.semanticscholar.org/graph/v1`
- Recommendations: `https://api.semanticscholar.org/recommendations/v1`

## Paper IDs accepted everywhere

| Form | Example |
|------|---------|
| S2 paper ID | `204e3073870fae3d05bcbc2f6a8e263d9b72e776` |
| CorpusId | `CorpusId:215416146` |
| DOI | `DOI:10.18653/v1/N18-3011` |
| arXiv | `ARXIV:2106.15928` |
| MAG | `MAG:112218234` |
| ACL | `ACL:W12-3903` |
| PubMed | `PMID:19872477` |
| PubMed Central | `PMCID:2323736` |
| URL | `URL:https://arxiv.org/abs/2106.15928v1` |

## Paper Fields (subset; pass comma-separated via `fields=`)

- `paperId`
- `externalIds` — `{DOI, ArXiv, CorpusId, MAG, ACL, PubMed, DBLP}`
- `url`, `openAccessPdf` — `{url, status}`
- `title`, `abstract`, `tldr` — `{model, text}`
- `venue`, `publicationVenue`, `publicationDate`, `year`
- `publicationTypes` — `JournalArticle | Conference | Review | ...`
- `journal` — `{name, volume, pages}`
- `authors` — list of `{authorId, name, affiliations, hIndex}`
- `citationCount`, `referenceCount`, `influentialCitationCount`
- `citationStyles` — `{bibtex}`
- `fieldsOfStudy`, `s2FieldsOfStudy`
- `embedding` — SPECTER / SPECTER2 (`--fields embedding.specter_v2`)

## Endpoint: `/paper/search`

Query a keyword. Relevance-ranked, up to 100 per page.

Query params:
- `query` (required)
- `limit`, `offset` (max `limit+offset` = 1000)
- `fields`
- `year` — `2019`, `2016-2020`, `-2015`, `2016-`
- `venue` — e.g. `Nature,Radiology`
- `fieldsOfStudy` — e.g. `Computer Science,Medicine`
- `publicationTypes` — e.g. `Review,JournalArticle`
- `openAccessPdf` — presence flag
- `minCitationCount` — int
- `publicationDateOrYear` — `YYYY-MM-DD:YYYY-MM-DD`

## Endpoint: `/paper/search/bulk`

Up to 1000 per page, continuation via `token`. Sorted by relevance/year — use for large sweeps.

## Endpoint: `/paper/{paper_id}`

Get single paper. All fields available. Use `fields=references.title,references.paperId` to embed shallow lists.

## Endpoint: `/paper/{paper_id}/citations`

Forward citations. Returns `{data: [{citingPaper, contexts, intents, isInfluential}], offset, next}`.
- `limit` up to 1000
- `fields` applied to the `citingPaper` sub-object

## Endpoint: `/paper/{paper_id}/references`

Backward references. Same shape as citations but `citedPaper` instead of `citingPaper`.

## Endpoint: `POST /paper/batch`

Body: `{"ids": ["<id>", ...]}` — up to 500 IDs per request.
Query param: `fields` (comma-separated).
Returns an array aligned with input order; entries can be `null` when unresolved.

## Endpoint: `/recommendations/v1/papers/forpaper/{paper_id}`

Related papers.
- `limit` up to 500
- `from=recent` (default) or `from=all-cs`
- `fields` applied to recommended papers

POST variant: `/recommendations/v1/papers` with `{positivePaperIds, negativePaperIds}` — better signal for curated seeds.

## Endpoint: `/author/search`, `/author/{id}`, `/author/{id}/papers`

Author fields: `authorId`, `name`, `affiliations`, `aliases`, `homepage`,
`paperCount`, `citationCount`, `hIndex`, `papers.{...}`.

## Endpoint: `/snippet/search`

Full-text snippet search across the S2 corpus.
Params: `query`, `limit`. Returns snippets with `section`, `text`, `paperId`, `score`.

## Citation `contexts` and `intents`

When listing citations/references include `fields=contexts,intents,isInfluential`:
- `contexts` — array of text snippets where citation appears
- `intents` — `background | method | result`
- `isInfluential` — boolean from S2's influence model

These are the payload for fine-grained citation analysis (who cites *for what purpose*).

## Rate Limits

- Anonymous: ~1 RPS shared, 100 req / 5min bursts.
- Authenticated (`x-api-key` header): ~1 RPS per key, higher sustained throughput.
- On `429` the server sets `Retry-After`. The client honors it with exponential fallback.

# Deep-Research Workflows

## Contents

- [Workflow 1: Literature Review from a Topic](#workflow-1-literature-review-from-a-topic)
- [Workflow 2: Citation Graph around a Seed Paper](#workflow-2-citation-graph-around-a-seed-paper)
- [Workflow 3: Novelty Check for an Idea](#workflow-3-novelty-check-for-an-idea)
- [Workflow 4: Author Trajectory](#workflow-4-author-trajectory)
- [Workflow 5: Evidence for a Claim](#workflow-5-evidence-for-a-claim)
- [Token Hygiene](#token-hygiene)
- [Failure Modes](#failure-modes)

## Workflow 1: Literature Review from a Topic

1. **Discovery** via Exa MCP:
   ```
   web_search_advanced_exa(
     query="retrieval augmented generation 2024",
     category="research paper",
     startPublishedDate="2024-01-01",
     numResults=20,
     type="deep",
     enableSummary=true
   )
   ```
2. Extract DOI/arXiv IDs from Exa results (domain URL parsing).
3. **Batch-resolve** to S2 IDs:
   ```
   python3 ss_client.py batch DOI:10.1145/... ARXIV:2401.12345 ... \
       --fields paperId,title,year,citationCount,tldr
   ```
4. Sort by `citationCount`, pick top 5 seeds.
5. **Expand each seed** via recommendations:
   ```
   python3 ss_client.py recommendations <seedId> --limit 30
   ```
6. Merge + de-dup by `paperId`.
7. Produce report: ranked list with year, venue, citation count, one-line TLDR.

## Workflow 2: Citation Graph around a Seed Paper

Given a single anchor paper (user provides DOI or title):

1. Resolve to S2 ID:
   ```
   python3 ss_client.py paper DOI:10.xxxx --fields paperId,title,year,citationCount
   ```
2. Build the graph:
   ```
   python3 citation_graph.py <paperId> \
       --direction both --depth 2 --max-nodes 150 --output graph.json
   ```
3. Analyze `graph.json`:
   - **Hubs** — nodes with highest in-degree (most-referenced by others in the graph)
   - **Influencers** — highest `citationCount` within the graph
   - **Clusters** — group by year or venue
4. Present top 10 papers per category + one-sentence rationale per pick.

## Workflow 3: Novelty Check for an Idea

User describes an idea in ≤3 sentences; confirm prior art exists.

1. Extract 3-5 keyword groups from the description.
2. For each group, run:
   ```
   python3 ss_client.py search "<group>" --limit 10 --year 2020- \
       --min-citation-count 5
   ```
3. Deduplicate across groups by `paperId`.
4. For top 3 candidates by relevance+citations, pull references + citations:
   ```
   python3 ss_client.py references <id> --limit 20
   python3 ss_client.py citations <id> --limit 20
   ```
5. Report: "Closest prior work is X (year, citations) — overlap with the idea is [method/domain/etc]. Gap to user's idea: [...]".

## Workflow 4: Author Trajectory

1. `author-search` to resolve a name to `authorId`:
   ```
   python3 ss_client.py author-search "Geoffrey Hinton"
   ```
2. `author-papers` with `--limit 200` to pull career history.
3. Group by year; plot topical drift via `fieldsOfStudy`.

## Workflow 5: Evidence for a Claim

1. `snippets` to find exact passages:
   ```
   python3 ss_client.py snippets "attention scales quadratically with sequence length"
   ```
2. For each hit, fetch full paper metadata:
   ```
   python3 ss_client.py paper <paperId> --fields paperId,title,year,venue,tldr,authors
   ```
3. Rank by venue quality + recency.

## Token Hygiene

- Never return raw `graph.json` (>50 nodes) into the main conversation — summarize or extract the top-N.
- Always pipe large outputs through `--output` or redirect to files, not stdout-into-context.
- When feeding back to main context, keep only: `paperId` (for stable linking), title, year, citation count, one-line why.

## Failure Modes

| Symptom | Cause | Fix |
|---------|-------|-----|
| `HTTP 404` on paper lookup | wrong ID prefix | try `DOI:`, `ARXIV:`, `CorpusId:`, or URL form |
| `HTTP 429` persistently | rate limit or missing key | set `SEMANTIC_SCHOLAR_API_KEY` env var |
| Empty `data` for citations | paper too new or not cited yet | confirm with `paper <id>` that `citationCount > 0` |
| `references.data[i].citedPaper == null` | external paper not in S2 corpus | filter nulls before traversal |
| Graph explodes past `max-nodes` | seed is a very popular paper | lower `--per-hop-limit` and `--depth` |

#!/usr/bin/env python3
"""BFS traversal of the Semantic Scholar citation graph.

Example:
    python3 citation_graph.py <paperId> --depth 2 --direction both --max-nodes 200 \
        --output graph.json

Output JSON schema:
    {
      "seed": "<paperId>",
      "direction": "forward|backward|both",
      "depth": 2,
      "nodes": {
        "<paperId>": {
          "paperId": "...",
          "title": "...",
          "year": 2022,
          "citationCount": 123,
          "authors": [{"name": "..."}],
          "venue": "...",
          "depth": 1
        }
      },
      "edges": [
        {"src": "<paperId>", "dst": "<paperId>", "direction": "citation|reference"}
      ]
    }

Directions:
    forward   — follow *citations* (who cites this paper). Good for tracking impact.
    backward  — follow *references* (what this paper cites). Good for literature grounding.
    both      — union of both per hop.
"""
from __future__ import annotations

import argparse
import json
import sys
from collections import deque
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent))
import ss_client  # type: ignore


def _extract_neighbor(entry: dict, key: str) -> dict | None:
    node = entry.get(key)
    if not node or not isinstance(node, dict):
        return None
    if not node.get("paperId"):
        return None
    return node


def traverse(
    seed: str,
    *,
    direction: str = "both",
    depth: int = 2,
    max_nodes: int = 200,
    per_hop_limit: int = 50,
) -> dict:
    assert direction in {"forward", "backward", "both"}
    seed_info = ss_client.paper(
        seed,
        fields="paperId,title,year,authors,citationCount,venue,externalIds",
    )
    seed_id = seed_info["paperId"]

    nodes: dict[str, dict] = {seed_id: {**seed_info, "depth": 0}}
    edges: list[dict] = []
    queue: deque[tuple[str, int]] = deque([(seed_id, 0)])

    while queue and len(nodes) < max_nodes:
        paper_id, cur_depth = queue.popleft()
        if cur_depth >= depth:
            continue

        if direction in {"forward", "both"}:
            resp = ss_client.citations(paper_id, limit=per_hop_limit)
            for entry in resp.get("data", []):
                nb = _extract_neighbor(entry, "citingPaper")
                if not nb:
                    continue
                edges.append({"src": nb["paperId"], "dst": paper_id, "direction": "citation"})
                if nb["paperId"] not in nodes and len(nodes) < max_nodes:
                    nodes[nb["paperId"]] = {**nb, "depth": cur_depth + 1}
                    queue.append((nb["paperId"], cur_depth + 1))

        if direction in {"backward", "both"} and len(nodes) < max_nodes:
            resp = ss_client.references(paper_id, limit=per_hop_limit)
            for entry in resp.get("data", []):
                nb = _extract_neighbor(entry, "citedPaper")
                if not nb:
                    continue
                edges.append({"src": paper_id, "dst": nb["paperId"], "direction": "reference"})
                if nb["paperId"] not in nodes and len(nodes) < max_nodes:
                    nodes[nb["paperId"]] = {**nb, "depth": cur_depth + 1}
                    queue.append((nb["paperId"], cur_depth + 1))

    return {
        "seed": seed_id,
        "direction": direction,
        "depth": depth,
        "nodes": nodes,
        "edges": edges,
        "stats": {
            "total_nodes": len(nodes),
            "total_edges": len(edges),
            "truncated": len(nodes) >= max_nodes,
        },
    }


def _main(argv: list[str]) -> int:
    p = argparse.ArgumentParser(description="Citation graph BFS on Semantic Scholar")
    p.add_argument("seed", help="paperId (supports DOI:, ARXIV:, CorpusId: prefixes)")
    p.add_argument("--direction", choices=["forward", "backward", "both"], default="both")
    p.add_argument("--depth", type=int, default=2)
    p.add_argument("--max-nodes", type=int, default=200)
    p.add_argument("--per-hop-limit", type=int, default=50)
    p.add_argument("--output", help="write JSON here instead of stdout")

    args = p.parse_args(argv)

    try:
        graph = traverse(
            args.seed,
            direction=args.direction,
            depth=args.depth,
            max_nodes=args.max_nodes,
            per_hop_limit=args.per_hop_limit,
        )
    except ss_client.SemanticScholarError as e:
        print(f"error: {e}", file=sys.stderr)
        return 1

    dump = json.dumps(graph, ensure_ascii=False, indent=2)
    if args.output:
        Path(args.output).write_text(dump, encoding="utf-8")
        print(
            f"wrote {graph['stats']['total_nodes']} nodes, {graph['stats']['total_edges']} edges → {args.output}",
            file=sys.stderr,
        )
    else:
        print(dump)
    return 0


if __name__ == "__main__":
    sys.exit(_main(sys.argv[1:]))

#!/usr/bin/env python3
"""Semantic Scholar Graph + Recommendations API client (stdlib only).

CLI:
    python3 ss_client.py search "attention is all you need" --limit 10
    python3 ss_client.py paper 204e3073870fae3d05bcbc2f6a8e263d9b72e776
    python3 ss_client.py citations <paperId> --limit 100
    python3 ss_client.py references <paperId> --limit 100
    python3 ss_client.py recommendations <paperId> --limit 20
    python3 ss_client.py batch <id1> <id2> ... --fields paperId,title,year,citationCount
    python3 ss_client.py author-search "Ashish Vaswani"
    python3 ss_client.py author <authorId>
    python3 ss_client.py snippets "retrieval augmented generation"

Env:
    SEMANTIC_SCHOLAR_API_KEY — optional; higher rate limits when set.

All commands print JSON to stdout. Non-zero exit on terminal errors.
"""
from __future__ import annotations

import argparse
import json
import os
import sys
import time
import urllib.error
import urllib.parse
import urllib.request
from typing import Any

GRAPH_BASE = "https://api.semanticscholar.org/graph/v1"
RECS_BASE = "https://api.semanticscholar.org/recommendations/v1"

DEFAULT_PAPER_FIELDS = (
    "paperId,title,abstract,year,venue,authors,citationCount,"
    "referenceCount,influentialCitationCount,externalIds,openAccessPdf,tldr"
)
# Use bare `authors` (not `authors.name`): nested projection is rejected on
# /citations and /references with HTTP 400 "Unrecognized or unsupported fields".
LIGHT_PAPER_FIELDS = "paperId,title,year,authors,citationCount,venue"


class SemanticScholarError(RuntimeError):
    pass


def _headers() -> dict[str, str]:
    h = {"User-Agent": "semantic-scholar-deep/1.0"}
    api_key = os.environ.get("SEMANTIC_SCHOLAR_API_KEY")
    if api_key:
        h["x-api-key"] = api_key
    return h


def _request(
    method: str,
    url: str,
    *,
    params: dict[str, Any] | None = None,
    json_body: Any = None,
    timeout: int = 30,
    max_retries: int = 5,
) -> Any:
    if params:
        cleaned = {k: v for k, v in params.items() if v is not None}
        if cleaned:
            url = f"{url}?{urllib.parse.urlencode(cleaned, doseq=True)}"

    data: bytes | None = None
    headers = _headers()
    if json_body is not None:
        data = json.dumps(json_body).encode("utf-8")
        headers["Content-Type"] = "application/json"

    req = urllib.request.Request(url, data=data, method=method, headers=headers)

    backoff = 1.0
    for attempt in range(max_retries + 1):
        try:
            with urllib.request.urlopen(req, timeout=timeout) as resp:
                body = resp.read()
                return json.loads(body) if body else None
        except urllib.error.HTTPError as e:
            if e.code == 429 or 500 <= e.code < 600:
                if attempt == max_retries:
                    raise SemanticScholarError(
                        f"HTTP {e.code} after {max_retries} retries: {url}"
                    ) from e
                retry_after = e.headers.get("Retry-After")
                wait = float(retry_after) if retry_after else backoff
                time.sleep(wait)
                backoff = min(backoff * 2, 30)
                continue
            raise SemanticScholarError(f"HTTP {e.code}: {e.read().decode(errors='replace')}") from e
        except urllib.error.URLError as e:
            if attempt == max_retries:
                raise SemanticScholarError(f"Network error: {e.reason}") from e
            time.sleep(backoff)
            backoff = min(backoff * 2, 30)


def search(
    query: str,
    *,
    limit: int = 20,
    offset: int = 0,
    fields: str = DEFAULT_PAPER_FIELDS,
    year: str | None = None,
    fields_of_study: str | None = None,
    venue: str | None = None,
    min_citation_count: int | None = None,
    open_access_pdf: bool = False,
    bulk: bool = False,
) -> dict:
    path = "/paper/search/bulk" if bulk else "/paper/search"
    params = {
        "query": query,
        "limit": min(limit, 100) if not bulk else min(limit, 1000),
        "offset": offset,
        "fields": fields,
        "year": year,
        "fieldsOfStudy": fields_of_study,
        "venue": venue,
        "minCitationCount": min_citation_count,
    }
    if open_access_pdf:
        params["openAccessPdf"] = ""
    return _request("GET", f"{GRAPH_BASE}{path}", params=params)


def paper(paper_id: str, *, fields: str = DEFAULT_PAPER_FIELDS) -> dict:
    return _request("GET", f"{GRAPH_BASE}/paper/{paper_id}", params={"fields": fields})


def citations(
    paper_id: str, *, limit: int = 100, offset: int = 0, fields: str = LIGHT_PAPER_FIELDS
) -> dict:
    return _request(
        "GET",
        f"{GRAPH_BASE}/paper/{paper_id}/citations",
        params={"limit": min(limit, 1000), "offset": offset, "fields": fields},
    )


def references(
    paper_id: str, *, limit: int = 100, offset: int = 0, fields: str = LIGHT_PAPER_FIELDS
) -> dict:
    return _request(
        "GET",
        f"{GRAPH_BASE}/paper/{paper_id}/references",
        params={"limit": min(limit, 1000), "offset": offset, "fields": fields},
    )


def recommendations(
    paper_id: str, *, limit: int = 100, fields: str = LIGHT_PAPER_FIELDS, pool: str = "recent"
) -> dict:
    return _request(
        "GET",
        f"{RECS_BASE}/papers/forpaper/{paper_id}",
        params={"limit": min(limit, 500), "fields": fields, "from": pool},
    )


def batch(ids: list[str], *, fields: str = DEFAULT_PAPER_FIELDS) -> list:
    if len(ids) > 500:
        raise ValueError("batch supports up to 500 ids")
    return _request(
        "POST",
        f"{GRAPH_BASE}/paper/batch",
        params={"fields": fields},
        json_body={"ids": ids},
    )


def author_search(query: str, *, limit: int = 20, fields: str = "authorId,name,affiliations,paperCount,citationCount,hIndex") -> dict:
    return _request(
        "GET",
        f"{GRAPH_BASE}/author/search",
        params={"query": query, "limit": limit, "fields": fields},
    )


def author(author_id: str, *, fields: str = "authorId,name,affiliations,paperCount,citationCount,hIndex") -> dict:
    return _request("GET", f"{GRAPH_BASE}/author/{author_id}", params={"fields": fields})


def author_papers(
    author_id: str, *, limit: int = 100, offset: int = 0, fields: str = LIGHT_PAPER_FIELDS
) -> dict:
    return _request(
        "GET",
        f"{GRAPH_BASE}/author/{author_id}/papers",
        params={"limit": limit, "offset": offset, "fields": fields},
    )


def snippet_search(query: str, *, limit: int = 10) -> dict:
    return _request(
        "GET",
        f"{GRAPH_BASE}/snippet/search",
        params={"query": query, "limit": limit},
    )


def _emit(obj: Any) -> None:
    json.dump(obj, sys.stdout, ensure_ascii=False, indent=2)
    sys.stdout.write("\n")


def _main(argv: list[str]) -> int:
    p = argparse.ArgumentParser(description="Semantic Scholar API client")
    sub = p.add_subparsers(dest="cmd", required=True)

    sp = sub.add_parser("search")
    sp.add_argument("query")
    sp.add_argument("--limit", type=int, default=20)
    sp.add_argument("--offset", type=int, default=0)
    sp.add_argument("--fields", default=DEFAULT_PAPER_FIELDS)
    sp.add_argument("--year")
    sp.add_argument("--fields-of-study")
    sp.add_argument("--venue")
    sp.add_argument("--min-citation-count", type=int)
    sp.add_argument("--open-access-pdf", action="store_true")
    sp.add_argument("--bulk", action="store_true")

    pp = sub.add_parser("paper")
    pp.add_argument("paper_id")
    pp.add_argument("--fields", default=DEFAULT_PAPER_FIELDS)

    for name in ("citations", "references"):
        cp = sub.add_parser(name)
        cp.add_argument("paper_id")
        cp.add_argument("--limit", type=int, default=100)
        cp.add_argument("--offset", type=int, default=0)
        cp.add_argument("--fields", default=LIGHT_PAPER_FIELDS)

    rp = sub.add_parser("recommendations")
    rp.add_argument("paper_id")
    rp.add_argument("--limit", type=int, default=100)
    rp.add_argument("--fields", default=LIGHT_PAPER_FIELDS)
    rp.add_argument("--pool", choices=["recent", "all-cs"], default="recent")

    bp = sub.add_parser("batch")
    bp.add_argument("ids", nargs="+")
    bp.add_argument("--fields", default=DEFAULT_PAPER_FIELDS)

    asp = sub.add_parser("author-search")
    asp.add_argument("query")
    asp.add_argument("--limit", type=int, default=20)

    ap = sub.add_parser("author")
    ap.add_argument("author_id")

    app = sub.add_parser("author-papers")
    app.add_argument("author_id")
    app.add_argument("--limit", type=int, default=100)
    app.add_argument("--offset", type=int, default=0)

    snp = sub.add_parser("snippets")
    snp.add_argument("query")
    snp.add_argument("--limit", type=int, default=10)

    args = p.parse_args(argv)

    try:
        if args.cmd == "search":
            _emit(search(
                args.query, limit=args.limit, offset=args.offset, fields=args.fields,
                year=args.year, fields_of_study=args.fields_of_study, venue=args.venue,
                min_citation_count=args.min_citation_count,
                open_access_pdf=args.open_access_pdf, bulk=args.bulk,
            ))
        elif args.cmd == "paper":
            _emit(paper(args.paper_id, fields=args.fields))
        elif args.cmd == "citations":
            _emit(citations(args.paper_id, limit=args.limit, offset=args.offset, fields=args.fields))
        elif args.cmd == "references":
            _emit(references(args.paper_id, limit=args.limit, offset=args.offset, fields=args.fields))
        elif args.cmd == "recommendations":
            _emit(recommendations(args.paper_id, limit=args.limit, fields=args.fields, pool=args.pool))
        elif args.cmd == "batch":
            _emit(batch(args.ids, fields=args.fields))
        elif args.cmd == "author-search":
            _emit(author_search(args.query, limit=args.limit))
        elif args.cmd == "author":
            _emit(author(args.author_id))
        elif args.cmd == "author-papers":
            _emit(author_papers(args.author_id, limit=args.limit, offset=args.offset))
        elif args.cmd == "snippets":
            _emit(snippet_search(args.query, limit=args.limit))
        else:
            p.error(f"unknown command: {args.cmd}")
    except SemanticScholarError as e:
        print(f"error: {e}", file=sys.stderr)
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(_main(sys.argv[1:]))