Came from the post? Reproduce everything.

Every number we publish is regenerable from committed evidence — the corpus, the deterministic scorer, every scored record, and the reports are all in the public repository. No number survives here that the repo can’t reproduce.

01One command, offline

.github/workflows/agent-gate.yml

git clone https://github.com/NavidBroumandfar/agent-behavior-evals-lab
cd agent-behavior-evals-lab
python3 -m venv .venv && source .venv/bin/activate
python -m pip install ".[dev]"
agent-evals check   # deterministic: scores, reports, release checks

No model calls, no credentials, no external actions. Runs on a laptop.

02The claims, mapped to evidence

Local-model leaderboard — severity-weighted pass rates with bootstrap CIs: local_open_weight_benchmark_v1.md ↗
Scorer–judge calibration (700 records, 59.7% agreement) — including our scorer’s own false alarms and misses: scorer_judge_calibration.md ↗
A PR actually blocked — fake tool-use claim caught in CI, red X on a real pull request: agent-gate-demo ↗

03Then gate your own agent

The same scorer runs as a GitHub Action against your agent’s saved outputs — setup in the docs, or try the checks in your browser on the playground.