LLM Mart Basic

Benchmark clean and incremental Xcode builds, surface compile and configuration hotspots, and produce an approval-first optimization plan before changing project files.

Compare browser-agent reliability on a repeatable task and anti-bot suite before choosing a stack or claiming progress.

Run a repeatable evaluation suite for browser agents against static web task snapshots instead of judging them from demos or one-off tests.

Run Claude Code, Codex CLI, Gemini CLI, or OpenCode through bounded H100 post-training tasks and compare how well each agent improves a base LLM.

Score deep research agents on benchmark tasks using factual verification, report-quality scoring, and process evaluation before model or workflow changes ship.

Use EnterpriseRAG-Bench to evaluate an enterprise RAG or knowledge-agent system against a realistic synthetic company corpus with answer, recall, and comparative scoring.

Run realistic enterprise-style IT scenarios before trusting an automation agent in production operations.

Run CIS benchmark checks against cluster nodes and control planes when an agent needs a narrow Kubernetes hardening audit, not a general platform listing.

Run a real-task benchmark suite against OpenClaw agents so model or harness changes can be compared before they hit production workflows.

Run structured prompt-injection attack and defense experiments against an LLM-integrated app before production by measuring attack success and testing detection or recovery pipelines.

Run concurrent scripted conversations against a target agent to measure whether it stays on task, responds correctly, and holds up in repeatable test cases.

Better Auth is an open source authentication framework for TypeScript apps. It gives agents a concrete way to wire sign-in, sessions, passkeys, OAuth providers, and plugins into modern web stacks with real package and docs support.

A fast, configurable secrets scanner built by the creator of Gitleaks and backed by Aikido Security. Betterleaks detects leaked passwords, API keys, and tokens in git repositories, directories, and stdin with CEL-based validation and parallelized scanning.

Automates migration from ESLint and Prettier to Biome (formerly Rome) by parsing .eslintrc and .prettierrc configs, mapping rules to biome.json equivalents, and running biome check --apply for bulk reformatting.

Generates Blender Python (bpy) scripts that programmatically create Geometry Nodes modifier trees, using the node_groups API and GeometryNodeTree interface for parametric 3D asset generation.

Put an inline firewall and containment layer in front of agent network traffic, tool calls, and MCP traffic before you trust an agent with local secrets.

Add hard pre-execution guardrails to Claude Code so destructive shell commands are blocked before an agent can run them.

Use CC Safety Net when coding-agent CLIs need pre-execution hooks that block destructive commands, secret access, and unsafe file operations before tools run.

Scan staged changes, commits, or repositories for secrets before they leave the workstation or CI job, instead of relying on a later platform-side catch.

Add a runtime guard that evaluates agent actions, blocks dangerous commands or secret exposure, and audits new skills before they run.

A surprising share of AI-in-the-terminal failures aren't the AI. They're zsh, and a version of bash from 2006.

A Claude Code plugin turns standalone project configuration into a namespaced, installable extension that teams and communities can update as one unit.

None of the safety came from the model. It came from six boring habits.

Skills package instructions and references. Subagents run work in a separate context and return results. They solve different problems and can be composed deliberately.

Six hours in, one step left, everything green, and the incident that didn't happen

CLAUDE.md carries persistent project context. Skills load reusable procedures when relevant. Separating stable facts from task-specific workflows keeps both easier to maintain.

Twenty minutes recovering secrets that never existed, and the one sentence from a human that ended it

An API request routing a model's tool call through an approval gate to a remote MCP server

31 config keys, two audits, and why the first one was wrong in both directions

The official MCP Registry stores standardized server metadata rather than package code. Publishers verify a namespace, describe installation or remote access, and submit immutable versions.

Everyone looks at the Dockerfile. The file that actually leaked the key was the project file.

Remote MCP authorization uses established OAuth standards, but secure integration still requires issuer validation, least-privilege scopes, protected token handling, and server-side enforcement.

"Copy it over and switch the reference" is two steps, and the outage lives in the one nobody checks

stdio fits local processes and prototypes. Streamable HTTP fits hosted services and shared integrations. The right choice follows where the capability runs and who must reach it.

The most important rule wasn't about what I could change. It was about what I was allowed to display.

Tools perform operations, resources expose readable context, and prompts provide reusable templates. Choosing the correct primitive makes an MCP server easier to understand and govern.

Use MCP Inspector to connect to local or remote servers, inspect capabilities, call tools, read resources, test prompts, and diagnose failures before release.

Build an MCP server in TypeScript with focused tools, validated schemas, local and remote transports, Inspector tests, and production security controls.

An MCP server exposes tools, resources, or prompts through a standard protocol so an AI application can discover and use external capabilities.

Treat an AI agent skill as both an instruction package and a software dependency: inspect what it says, what it runs, what it can access, and how it updates.

Periodic multi-reviewer sweep of the whole codebase — surfaces a triaged checkpoint report.

Sanctioned lane for non-behavioral work — docs-only edits, dependency bumps, reverts. Type-scaled gates; no TDD demanded of prose.

Finish an already-merged branch — classify the leftover artifacts, return to a fast-forwarded default checkout, and delete the merged local branch. Every discard confirmed per item; containment proven, never assumed.

Show the codeArbiter command catalog — the public command list and what each routes to.

Run the full commit gate — the only sanctioned path to a git commit.

Stop everything and surface a rule conflict — persona vs. docs vs. code. Present both sides and the conflict-hierarchy level; the user resolves. No silent reconciliation.

Optional manual drift audit — report stale provenance-tracked docs, then per stale doc offer re-scout, re-baseline, or defer. Not the daily loop; commit-gate auto-heal owns routine maintenance.

Brownfield back-fill — scout an existing codebase and populate .codearbiter/, then lock it initialized.

Investigate-then-decide root-cause analysis for a defect whose cause is unknown. No code changes — exits to {{CMD:fix}}, {{CMD:adr}}, or a no-action close.

Greenfield decomposition interview — a layered interview that populates .codearbiter/ and locks it initialized.

Verify the active host install, package, command ownership, enforcement{{IF:pi}}, wrapper self-test, and active-dispatch coverage gap{{ELSE}}, and harmless live-fire probe{{END}}. Read-only.

Start a feature: brainstorm a spec, get it approved, then drive it test-first through the pipeline. The one entry to implementation.

Fix a confirmed bug: a failing regression test first, then a minimal fix, then the rest of the tdd gates.

Opt this repo into codeArbiter — scaffold the root-level .codearbiter/ state store.

Read-only 3-metric governance glance — override rate, small-lane rate, sprint low-confidence ratio — each with a trend arrow vs. the prior 20-commit window.

Author a new codeArbiter skill: prove the gap is real, get the spec approved, then write it.

Sanctioned, logged bypass of a gate or hard rule — one audit line, then proceed.

Open a pull request the only sanctioned way — clear every BLOCK-level review finding, then stage the PR. Never a direct write to the default branch.

Zero-onboarding, read-only dry-run of the reviewer fleet against the current uncommitted diff. Predicts reviewers, runs the state-free secret scan, writes nothing.

Trim transcript clutter to extend session lifetime — analyze, prune a copy, or toggle the after-each-turn service. Dry-run by default; gains land at resume/compaction, not the current turn.

Open-source super AI assistant & Agent Harness. Plans tasks, runs tools and skills, self-evolves with memory and knowledge. Multi-model, multi-channel. Lightwei…

Open-source AI browser agent for web automation: a web browsing agent and computer-use agent in plain English. Browser MCP for Claude Code and Gemini CLI.

Agent-native trading terminal built on DeepSeek Harness. Crypto, US, CN and HK in one three-column GUI, 19+ hot-swappable connectors, dry-run by default with hu…

Open-source desktop and web coding agent with a first-party host and harness, durable sessions, model connections, streaming chat, and workspace tools.

OrcaReplay — Time travel for AI agents. Record, replay, fork, and debug any agent run with any model. Built by the OrcaRouter.ai team.

A complete toolkit for connecting R and LLMs

OpenSRE — the memory-first AI SRE. Self-hosted incident investigation with episodic memory, knowledge graph, web console, Slack & Teams. opensre.in

A desktop AI coding assistant that works with your local projects. Ally helps you understand code, edit files, search a workspace, manage tasks, and complete de…

【在线免费使用】 简单快速将SQL或DBML转换为美观的ER图（支持 Agent Skill）/ The best SQL to ER Diagram converter (Support Agent Skill).

Operation-bound protection for autonomous AI agents: contain and verify operations before committing changes, understand intent and risk across long-running wor…

Local-first AI coding agent desktop: Electron + Rust host core + pi Agent Harness + user-installable plugins

Local-only Go static analysis engine with a built-in MCP server. Gives AI coding agents deterministic structural awareness: call graphs, impact analysis, symbol…

Self-hosted AI agent workspace with tool calling, MCP, multi-model routing, sandboxed execution, multi-agent workflows, and LLM-authored 3D character animation…

Automated TDD enforcement for Claude Code

High-performance platform for building websites, e-commerce, and web applications—with Native AI, JavaScript development, and a marketplace for portable sites a…

Autonomous AI-agent orchestration engine for job discovery with decision traces, tool adapters, and production-grade run control.

Token efficient Claude Code full Python rebuild. AI Coding Agent in 310K LoC Python.

OPC — One Person Company. A full team in a single Claude Code skill. Adaptive agent orchestrator with 21 built-in roles, 6 flow templates, and adversarial quali…

Open-source, secure environment with real-world tools for enterprise-grade agents.

MCP server for AI-powered GitHub project management — agent orchestration, PRD-to-issues pipeline, sprint planning, and multi-agent swarm coordination