Ai Guardian
Governed local-LLM observability: model policy, prompt scanner, capture proxy, 21 tools.
- Transport
- Not stated
- Package
- —
- Registry id
- io.github.AIops-tools/ai-guardian
No install snippet on purpose. A working MCP config is a command, its arguments and an environment block — the last two are where API keys live, so this catalogue never stores them and cannot publish them. Follow the link above for the authors' own instructions.
Disclaimer: Community-maintained open-source project. Not affiliated with, endorsed by, or sponsored by Ollama, IGEL, or any AI-security vendor. Product and trademark names belong to their owners. MIT licensed.
Governed observability + governance for on-endpoint local LLMs. It lets you
observe + audit what your local models are actually fed, and gate what leaves in
a prompt — the complement to IGEL AI Armor. AI Armor governs whether a
local model may run on the endpoint; ai-guardian records what it did and gates
what goes into the prompt (secrets, PII, source, jailbreaks) plus which model
may serve it. Self-contained: it talks to each runtime's REST API and needs
nothing beyond httpx and the MCP SDK. v0.1 provides opt-in route-through
content governance, plus a transparent capture proxy for clients that did not opt in.
Supported runtimes
One tool, several local runtimes, selected per target by a runtime field in
config.yaml (the init wizard asks). Ollama uses its native API; the other three
share one OpenAI-compatible transport (/v1/models + /v1/chat/completions).
| Runtime | runtime |
Default port | List / policy | Scan + route-through guard | Provenance |
|---|---|---|---|---|---|
| Ollama | ollama |
11434 | ✅ | ✅ | digest (content hash — strong) |
llama.cpp (llama-server) |
llamacpp |
8080 | ✅ | ✅ | props — /props model path/size → pinnable id |
| LM Studio | lmstudio |
1234 | ✅ | ✅ | id only — weaker; pins report unverifiable |
| vLLM (local single-node) | vllm |
8000 | ✅ | ✅ | id only — weaker; pins report unverifiable |
The allow/deny model policy, the deterministic prompt scanner, the route-through
guard (guarded_generate / observe_chat), provenance drift, and doctor work
across all runtimes. Model lifecycle writes (pull / remove / unload)
are Ollama-only — the OpenAI-compatible servers load a model at startup and expose
no lifecycle endpoint, so those writes are refused with a clear message.
Provenance honesty: only Ollama (content digest) and llama.cpp (a /props-derived
path/size identity) expose something to pin. LM Studio and vLLM expose only a model
id, so a pinned digest is reported unverifiable rather than a false DRIFT.
vLLM here is a LOCAL endpoint-guarding use case. GPU inference-cluster operations (autoscale, drain, Ray Serve/Jobs, model lifecycle at fleet scale) belong to a different tool in the line — GPU cluster ops → inference-aiops.
What it does
Ollama persists no queryable prompt/response history — conversational context is client-supplied on every request. So ai-guardian observes on two fronts:
- Passive inventory / state auditing — over
/api/tags,/api/ps,/api/show,/api/version: what models are installed and running, their VRAM residency, license/params/capabilities, and their provenance digests. Every model is annotated with an allow/deny policy verdict, so shadow (unsanctioned) models showallowed: false. - Opt-in route-through content governance — callers send a prompt through
ai-guardian (
guarded_generate/observe_chat). It scans the text (secrets / PII / source / jailbreak), checks the model against policy, records the interaction to its own usage log (~/.ai-guardian/usage.db), and only then calls Ollama — blocking when the risk band is too high or the model is disallowed. The raw prompt is never stored (only its length + redacted findings).
From the project's README.