Skip to content
Senthira

Reproduce it yourself.

The corpus, the deterministic scorer, every scored record and the reports are all in the public repository. The benchmark and gate numbers below regenerate offline from a clean checkout, with no model calls and no credentials.

where that stops being true
This page used to say “no number survives here that the repo can’t reproduce”. That was an overclaim, and a self-audit on 20 August 2026 caught it. Two classes of number do not regenerate from a public clone: the LLM-judge studies (the 700-record calibration study and the judge-with-log experiments) depend on saved judge outputs that are gitignored, so a stranger can read the committed aggregates but cannot re-derive them; and the pack separability probe was hand-run with no generator. Their aggregates stay published because deleting an inconvenient measurement is worse than disclosing its limit, but they are auditable, not reproducible, and the difference is ours to state rather than yours to find.

01One command, offline

.github/workflows/agent-gate.yml
git clone https://github.com/NavidBroumandfar/agent-behavior-evals-lab
cd agent-behavior-evals-lab
python3 -m venv .venv && source .venv/bin/activate
python -m pip install ".[dev]"
agent-evals check   # deterministic: scores, reports, release checks

No model calls, no credentials, no external actions. Runs on a laptop.

02The claims, mapped to evidence

03Then gate your own agent

The same scorer runs as a GitHub Action against your agent’s saved outputs; setup in the docs, or try the checks in your browser on the playground.