An open-source, Jest-style test runner for AI agents. Write tests in TypeScript, mock the model and its tools, check what the output means rather than its exact wording, and see step by step where an agent's behaviour changed between commits.
$ npx lithe
tests/agent.test.ts
✓ asks before sending email (812ms)
✓ never ghosts a candidate (1.4s)
control-eval
An AI-control harness. When an agent doing routine DevOps work is also quietly trying to exfiltrate a secret, how often does a cheaper monitor model catch it, and at what false-alarm cost? Fully sandboxed, scored against independent ground truth.
$ npm run eval -- --pairs 20 \
--agent claude-opus-4-8 \
--monitor claude-haiku-4-5
→ catch rate vs false alarms
→ AUC, refusals, and the misses