KKERF View source ↗

PUBLIC PROJECT DEMO · V0.4.0

AI answers.
KERF checks
the work.

An evidence-first evaluation system for AI agents that query business data. KERF validates generated SQL, executes it through a read-only boundary, scores the result against deterministic truth, and records every failure.

Start the beta test Explore the demo Inspect live run ↗

All business records are synthetic. Live model results are clearly labeled.

sample_fixture_record.json
"case": "june_revenue",
"model_answer": "$4,960",
"sql_policy": "passed",
"result_match": true,
"score": 100,
"evidence": {
  "rows": 1,
  "source": "synthetic_fixture",
  "api_charge": 0
}
PASSProse is not proof. Results are.
20Versioned test cases
2Live profiles compared
5SQL safety controls
100%Synthetic business data
01

VERIFIED BASELINE

Real model evidence,
not a demo claim.

One paid, repeatable GitHub Actions comparison. Same 20 cases, same data, same reasoning effort.

✓ VERIFIED LIVE RUN

Luna vs. Terra · low effort

Recorded September 3, 2026 · workflow attempt 2

Open evidence ↗
PROFILE A

gpt-5.6-luna

6/ 20 passed
Avg score
62.5
Avg latency
3.95s
Est. cost
$0.0096
Total tokens
13,021
PROFILE B

gpt-5.6-terra

8/ 20 passed
Avg score
66.75
Avg latency
5.88s
Est. cost
$0.0978
Total tokens
13,189

01Terra gained 2 additional passes and 4.25 score points.

02That gain came with 1.49× latency and 10.21× estimated cost.

03Both profiles missed most cases—exactly the reliability gap KERF exposes.

02

GUIDED PLAYBACK

Follow one evaluation
from prompt to proof.

This browser-only walkthrough uses a deterministic fixture. It makes no model request and cannot spend API credits.

LOCKED No API key · no write access · no charge

READYSelect a case and run the playback.
  1. 1
    Load versioned caseWaiting
  2. 2
    Validate read-only SQLWaiting
  3. 3
    Execute against synthetic dataWaiting
  4. 4
    Compare with deterministic truthWaiting
EXPECTED RESULT

Run the walkthrough to reveal the verified answer.

03

TEST COVERAGE

20 questions that
models must get right.

04

SYSTEM DESIGN

A reliability pipeline,
not a prompt wrapper.

Every stage produces inspectable evidence and fails closed when the model breaks contract.

01Versioned caseBusiness question + deterministic truth
→
02Structured model outputAnswer + SQL + explanation
→
03Read-only boundaryAllowlist + limits + SQLite authorizer
→
04Result scoringRows checked against expected query
→
05Evidence recordPass/fail + latency + tokens + cost
BACKEND

FastAPI + Python

Typed API routes coordinate evaluation runs, history, comparisons, and downloadable reports.

SAFETY

Read-only SQL

Lexical rules, immutable database mode, query-only pragmas, an authorizer allowlist, and hard limits.

EVIDENCE

CI + versioned data

GitHub Actions runs the live comparison while committed summaries keep proof durable after artifacts expire.

OPEN-SOURCE / PRE-LAUNCH

Built to find the miss
before users do.

Review the implementation, test suite, SQL safety boundary, and live evaluation evidence on GitHub.

View the repository ↗ Read the technical brief ↗