Misata Logo
Misata Studio

AI & agent workflows

An agent is only as correct
as the world you evaluate it against.

Most synthetic data tools generate plausible values, one column at a time. That is enough for a screenshot. It is not enough for an agent, because an agent does not read a screenshot, it follows a foreign key, sums a column, and acts on what it finds. If the data underneath it does not actually hold together, the agent fails the eval for a reason that has nothing to do with whether it was right.

What an agent actually needs from the data

Three things, and a plausible-rows generator gives you at most one of them.

Relationships that actually resolve

Every foreign key an agent follows has to point somewhere real. Customers exist, orders belong to a real customer, refunds belong to a real order. One broken join and the agent's next step is undefined, not wrong, undefined.

A declared, known ground truth

To grade an agent's output you have to know what the correct answer was before it ran. Declare that 2.1% of transactions are fraudulent, or that 214 shipments breach the SLA, and you know exactly which rows a correct agent should surface.

A reproducible starting state

The same seed produces the same world, byte for byte. Run the eval again next week, or against a different model, and the only thing that changed is the agent, not the ground it is standing on.

What teams reach for instead, and what it costs

Production data, sampled or masked

Real customer records are the wrong thing to hand an autonomous agent regardless of masking, and most teams cannot clear the review to do it before the eval needs to run.

Faker or a column-at-a-time generator

Fills each field with no idea what the neighbouring column, or the neighbouring table, says. An agent that joins orders to customers finds rows pointing at customers that do not exist, and the eval fails on a broken join, not on the agent's judgment.

A hand-built fixture file

Works for one scenario. The moment the eval needs a second scenario, a harder distribution, a different fraud rate, someone is editing JSON by hand and hoping the relationships still hold.

Asking a general-purpose LLM for rows

Produces plausible values with no guarantee of referential integrity past a few tables, and no way to reproduce the exact same starting state on the next run.

Model Context Protocol

Or let the agent design the world itself

Misata ships an MCP server, so an agent in Claude Desktop, Cursor, Windsurf, Zed, or Continue can generate its own evaluation environment instead of you writing Python for it. The division of labour is deliberate: an agent is good at deciding what tables and columns a veterinary clinic or a logistics company actually needs. Misata is good at guaranteeing the math underneath that decision, referential integrity, exact aggregates, declared distributions, byte-identical reruns.

pip install "misata[mcp]"

# then, in Claude / Cursor / Windsurf:
"Generate a fintech fraud dataset with 10k customers
 and a 2% fraud rate."
generate_from_schema

Primary tool. The agent designs the schema dict; Misata returns CSVs plus an integrity proof: per-relationship orphan counts, exact roll-ups, the seed used.

generate_dataset

Story-based generation from one sentence, Misata's own parser designs the schema.

preview_story / inspect_schema

See the domain, scale, and table layout Misata would produce, or the full schema as structured JSON, before generating a row.

validate_yaml

Two-layer validation, structural and semantic, of a misata.yaml the agent or a human wrote.

Set up the MCP serverOr start from a free dataset

Further reading

Choosing an MCP server for synthetic data generation

An agent can design a schema but cannot guarantee the math. What Misata’s MCP server actually returns, and why the integrity proof is the part that matters.

Half the answer keys in text-to-SQL benchmarks are wrong

What happens to agent evaluation when the ground truth itself is wrong, and a construction where the answer key cannot disagree with the data.

Questions

What is synthetic data for AI agents?
It is a generated, relational dataset an agent reads and writes during a task or an evaluation run, built so the tables, foreign keys, and declared outcomes hold together the way a real system's would, without any real customer or production record involved.
Why can't an agent just use Faker-generated or column-independent data?
Because an agent acts on relationships, not columns. If it looks up a customer's orders, applies a refund, or checks whether a balance reconciles, every one of those steps depends on foreign keys actually resolving and totals actually summing. A generator that fills each column independently produces data that looks fine printed as a table and falls apart the moment an agent follows a join.
How do I get a reproducible starting state for an agent evaluation?
Generate from a fixed seed. Misata's generation is deterministic: the same schema, declarations, and seed produce byte-identical output every run, so an agent's behavior can be compared across runs against the same starting world instead of a new one each time.
Can an AI agent generate its own test data?
Yes, through Misata's MCP server (pip install "misata[mcp]"). An agent in Claude Desktop, Cursor, Windsurf, Zed, or Continue can call generate_from_schema with a schema it designs itself, or generate_dataset from one sentence, and get back CSVs plus an integrity proof: orphan counts, exact roll-ups, and the seed used, without a human writing Python.
How do I verify what an agent actually did, correctly?
Declare the ground truth before the agent runs. If you generate a world where exactly 214 orders have a shipment delay past the SLA, you know precisely which rows the agent should flag. Compare its output against the declaration, not against a plausible-sounding guess at what the data probably contains.
Does this replace a real agent evaluation harness?
No. Misata generates and verifies the data underneath the harness, the rows, the relationships, the declared outcomes. What the harness does with an agent's actions, scoring, retries, comparison across model versions, is a separate concern and usually a separate tool.

Misata is an open-source engine for generating relational data that satisfies declared outcomes exactly, MIT licensed, readable end to end. If you are building or evaluating agents against data and hitting a gap this page does not cover, tell us what it is. We would like to know.