GitHub collection

relai-ai/agent-eval-loop

Imported from GitHub — is this yours?

6 skills imported from this repository.

View on GitHub
Stars
41
Forks
2
License
Apache-2.0

Skills (6)

coding-evals

Build and run agent-neutral coding evals from sessions, PRs or git history for a team's coding agent, including one developed here. Measure its baseline or compare models, versions, setup changes and agents. Use eval for application-agent scenarios and harnesses, and for agent-ne

Claude Skill relai

discover

Identify requirements, risks and open questions for an AI agent or LLM application from code, docs or traces; analyze failed runs, maintain its wiki, resolve questions and review risks. Excludes ordinary bugs outside an AI agent. Use trace-review to build a review app.

Claude Skill relai

improve

Fix or improve an AI agent or LLM application, compare prompts, models or settings, reduce cost or latency, or check changes for regressions. Excludes ordinary bugs outside an AI agent. Use eval for measurement alone and loop for continuous learning from new evidence.

Claude Skill relai

loop

Continuously improve an AI agent from traces, feedback and evaluations through repeated learning and experiments. Use improve for a single fix or improvement and eval for a one-off evaluation run. Excludes scheduled prompts and polling.

Claude Skill relai

start

Orient users on RELAI's capabilities, current eval status and next steps, or initialize RELAI records when a workflow needs them. Route clear requests directly to the relevant skill and reuse existing project infrastructure.

Claude Skill relai

trace-review

Build or adapt a trace review app with persistent source-linked feedback, including review of findings, questions, scenarios or graders when needed. Use for a viewer or annotation UI; use discover for analysis without an app.

Claude Skill relai