AI QUALITY ENGINEERING

QA engineering for AI. Every stage, one platform.

For teams shipping LLM apps and agents: trace production, turn real failures into tests, grade with rules, judges and your reviewers, and block bad releases before users see them.

Get startedTry it — no sign-up
Send your first traceREST API · or the SDKs and OpenTelemetry
curl -X POST "$GRADEFUSE_BASE_URL/api/v1/traces" \
  -H "Authorization: Bearer $GRADEFUSE_API_KEY" \
  -H "Content-Type: application/json" \
  -d @trace.json
paper-explainer · release historyexample
  1. v16 · shipped
    All 412 cases at or above baseline. Merged.
    PASS
  2. v15 · blocked
    Faithfulness 0.91 → 0.74 and citations 96% → 81%. Merge blocked by the gate.
    FUSE BLOWN
  3. v14 · baseline
    Reference run for pull request comparisons.
    PASS
Judge agreement with reviewers: 0.81 κillustrative example
WORKS WITHOpenTelemetryVercel AI SDKLangChainOpenAI SDKAnthropic SDKTypeScript SDKPython SDKGitHub Actions
THE QUALITY LOOP · ONE FAILURE, START TO FINISH

Every grade traces back to the request that caused it.

Follow one real failure — a summary that invents benchmarks — through all six stages.

01 / 06Live

Capture the failure

A user asks a paper-explainer app to summarize a paper. The summary names three benchmarks the paper never mentions. Gradefuse records the whole request: retrieval, prompt v15, model call and output.

Tracing through the API, the TypeScript and Python SDKs, or OpenTelemetry, with cost and latency on every span.

gradefuse / paper-explainer / traces/tr_8f3a
Trace tr_8f3a · 4 spans · 2.41s
agent.runroot2.41s
└ retrieve_sectionstool410ms
└ llm.summarize (prompt v15)llm · 1,203 tok1.62s
└ format_outputfunc90ms
Illustrative example
THE QA YOU ALREADY KNOW

Your team knows how to ship tested software. Now do it for AI.

SOFTWARE QAIN GRADEFUSEWHO USES ITSTATUS
Unit testsDeterministic checks — schema, math, links, citationsEngineerLive
Test planVersioned rubrics and golden datasetsPM · domain expertLive
Code reviewReviewer queue that calibrates the LLM judgesDomain expertLive
CI pipelineRegression gate on every pull requestEngineerLive
Production monitoringOnline evals and LLM spend trackingEngineer · PMShipping soon
Bug trackerConfirmed failures become new test casesEveryoneShipping soon
WHY GRADEFUSE

Three things you won't get elsewhere in one place.

Grades you can defend

Every LLM judge shows how often it agrees with your reviewers — before and after they correct it.

Before calibration
0.58
After 120 reviewer labels
0.81
Cohen's κ · illustrative example

Costs less to run

Rules settle what they can for free. You pay for an LLM judge only when rules can't decide.

Rules 84%LLM judge 14%Reviewer 2%
412-case run · illustrative example

Packs for your domain

Install ready-made checks, rubrics and golden cases for your kind of AI feature, then tune them to your data.

  • pack/ecommerce-deals-quality
  • pack/paper-explainer-quality
  • pack/rag-quality
  • pack/customer-support-quality
  • pack/code-generation-quality

Give your AI a QA team.

Gradefuse is in early access. Sign up, or try a comparison without an account.

Get startedTry the demo