The answer key and the checker

Misata made the data, so it knows the right answer to every question about it. The answer key writes that down, and the checker grades anything that reads the data against it.

What is in the key#

For each metric:

Field	What it is
id, title	The metric: mrr, logo_churn, billed_revenue, win_rate, or a number you stated
sql	The query that reads it from the tables (DuckDB SQL; sql_dialect says so)
values	Its value in every month of the window
stated	The value you asked for, where you stated one
tolerance	How close counts as equal (a cent for money, a row for a count)
misreadings	The metric's usual misreadings: what each computes instead, its SQL, and its values on these same rows

The file is answer_key.json ("format": "misata.answer_key/1", "sql_dialect": "duckdb"), with its window. Beside it, on the Story tab, in the demo pack and through the API:

• Identities: equations the data keeps, read in SQL (billed revenue equals MRR each month; an order's total equals its lines).
• Questions: plain-language questions drawn from the key, each with its answer, its SQL and the wrong answers its usual misreadings give.

Which metrics are in the key depends on the business. The built-in shapes carry their domain's standard set (for SaaS: MRR, ARR, ARPA, net new MRR, net and gross revenue retention, logo churn, billed revenue, win rate, deal size and sales cycle). On your own tables, the key holds every number you stated, plus the standard metrics a model proposes for your schema, kept only where their SQL runs on the rows.

Why the misreadings matter#

Most wrong dashboards are not wrong at random. They make one of a handful of known mistakes, and the key measures each of them on the actual rows:

Metric	A usual misreading	What goes wrong
Logo churn	Over every account in the table, or over the month's end	Divides by accounts that were never at risk that month
MRR	Counting subscriptions after they ended, or leaving out upgrades and downgrades	Overstates MRR, or misses expansion and contraction
ARR	The ARR each contract was signed at	Ignores every change since signing
Net revenue retention	Counting new customers' MRR too	Reports growth, not retention
Win rate	Over every deal opened in the month	Counts deals still open as losses
Average deal size	Over the deals lost too	Averages in deals that never closed

The checker also knows mistakes that are not a metric's definition: billed revenue read by issue date with voided invoices counted, for example, is found as a pair.

Because each misreading's value is known, a wrong number can be named for the mistake that produced it.

The checker#

On every dataset's Story tab, through the platform API (POST /v1/datasets/{id}/check) and through the MCP server (check_answers).

Check a query. Paste the SQL a dashboard or an AI wrote. The checker runs it on the dataset and compares the result to every metric in the key:

• A right query is named as the metric it computes.
• A wrong query is told what it computes instead ("this is churn over every account, not over accounts at risk"), with the one change that makes it right, proved by running the corrected query.
• Two mistakes at once (voids counted and months from the issue date) are found as a pair.
• A single number on a tile is read against the spans a tile usually shows: the latest month, the month before, the window, year and quarter to date, and the last 3, 6 and 12 months.
• Postgres to_char and MySQL date_format are read as DuckDB's strftime where they mean the same, and said.
• A query rounded for display is recognised as rounded.

Check numbers. Paste the numbers a dashboard shows (a tile, or a series by month) and name the metric. Each is read against the key, and a wrong one is matched to the misreading that gives it.

Take the key with you#

• Story tab: download answer_key.json and a long CSV (one row per metric and month).
• Export: the demo pack (Postgres or ClickHouse) carries the tables, a one-step loader, the key and the questions.
• API: GET /v1/datasets/{id}/answer-key and GET /v1/datasets/{id}/questions?n=20.
• Certificate: every stated number is read again in its own SQL, shown under each held number, and the public certificate link carries the story as monthly aggregates.

Grading an AI analyst or a dashboard generator#

1. Build a dataset with the story you want to test against (or reuse one: the same seed makes the same data).
2. Load it where your product reads data (Postgres, ClickHouse, a file), or let your product query it in place through the API.
3. Take questions from the key (/questions), or write your own.
4. Ask your product each question. Collect the SQL it writes, the numbers it shows, or both.
5. Send each to the checker. Every answer comes back right, or wrong with the mistake named.
6. Change the story (a new spike, another window, a different seed) and run it again. Because every dataset has its own key, you are never grading against a dataset your model has seen.

See What you can do with Misata for this and other uses, step by step.