A reproducible RAG evaluation and regression-gating workbench. It scores groundedness, context relevance, citation precision, expected-answer completeness and latency without paid APIs.
- Inspectable deterministic metrics instead of a hidden evaluator prompt
- Invalid/duplicate citation detection
- Golden-answer term coverage
- Baseline-versus-candidate comparison with a 3% regression budget
- PASS, WARN and FAIL quality gates with actionable flags
- Browser dashboard, Actuator health, tests, CI and Docker
mvn spring-boot:runOpen http://localhost:8089 or call GET /api/evaluations/demo.
POST /api/evaluations— score one RAG casePOST /api/evaluations/compare— compare a candidate against a baselineGET /api/evaluations/demo— run a built-in golden case
mvn verify
docker build -t evalforge-ai .Deterministic lexical metrics are intentionally transparent and fast. They complement—not replace—human review and model-based semantic evaluation for production systems.
MIT