Scope Recall Hermes

Hermes Agent memory plugin/provider for scope-aware recall, SQLite truth, LanceDB semantic search, and hybrid retrieval.

LLM Mart
14 views 642 listing impressions

Hermes current-turn memory provider with journal-first semantic capture, durable recall, SQLite truth storage, and optional vector companions

Give Hermes durable memory that can follow the same user across windows/chats while keeping local scratch context from bleeding into the wrong place.

Current-turn recall · Journal-first capture · Durable shared memory · Background digest · Local scratch scopes · SQLite truth · LanceDB/SQLite companion · Hybrid RRF retrieval

CI License: MIT Hermes Plugin Python Storage

scope-recall is a Hermes local memory provider built for current-turn recall and durable semantic memory. Durable user/project/ops/memory facts are shared across windows/chats for the same user + agent identity; raw general turn captures stay local to the current chat/thread/session.

This repository, scope-recall-hermes, is the Hermes implementation. The Python distribution package is hermes-scope-recall, the Python import/package spelling is scope_recall, and the Hermes plugin ID/provider name remains scope-recall for runtime compatibility. The OpenClaw sibling implementation lives at scope-recall-openclaw.

Benchmark: LoCoMo long-conversation memory

In August 2026 we ran Scope Recall's 1.9.2 development line end-to-end on the LoCoMo long-conversation benchmark: all 1,540 questions across the four non-adversarial categories, with 100% coverage and zero invalid results — a full run, not a sample.

  • Overall accuracy: 70.58% (1,087 / 1,540)
  • Single-hop factual memory: 84.78%
  • Temporal reasoning: 54.83%
  • Multi-hop reasoning: 54.61%
  • Open-domain inference: 45.83%
  • Retrieval side: Top-50 evidence any-hit recall 97.66%, all-hit recall 90.63%

The run used gemini-embedding-001 for retrieval and gpt-5.4-mini as answerer, judge, and query planner. Every per-question result and the run manifest are hash-pinned, and the final score was recomputed independently from the raw rows; the public, path-free receipt is docs/benchmarks/locomo-2026-08.md. The benchmark harness (scripts/benchmark.locomo.py) is part of the 1.9.2 development line so the evaluation stays reproducible.

The August run predates the harness's secret-free model-route receipt and stricter official-comparability checklist. It is therefore legacy local evidence, not a run for which the current official_comparable_categories_1_to_4 flag may be asserted.

The runner has no machine-specific dataset, source-tree, or auth defaults. Supply the external paths explicitly, keep the run directory outside the source checkout, and add --auth-path only for model-backed phases:

python scripts/benchmark.locomo.py \
  --dataset /path/to/locomo.json \
  --run-dir /path/outside/the/checkout/locomo-run \
  --hermes-agent-root /path/to/hermes-agent \
  --phase retrieve

The three categories near 50% remain the clearest future improvement areas: multi-hop evidence completeness, temporal evidence presentation, and open-domain synthesis. We do not claim a cross-vendor ranking, because public LoCoMo evaluations use materially different models, judges, prompts, and dataset variants.

From the project's README.

Comments (0)

Sign in to join the conversation.

No comments yet.