Data provenance

The questions procurement and security teams ask about synthetic data, each answered with the mechanism that makes the answer true, not just a policy. misata provenance <dir> prints the same statement plus per-file hashes for any generated folder.

Is any real data involved?#

Not when you generate from a schema. Misata reads no rows to do it: the values come from declarations (a schema, distributions, curves, identities), format templates and public vocabulary pools. There is no training step and no corpus of scraped personal data behind the values.

The exception is explicit and opt-in: misata mimic fits per-column quantiles and category frequencies to a table you provide, on your machine. It keeps no rows, but the categories it saw and the extremes of each column can reappear in its output, so that output is not anonymous. A folder mimic writes carries fitted.json; misata provenance reports it and misata publish refuses it without --allow-fitted.

Does generation touch the network?#

No. Generation is fully offline. The optional LLM integration operates only at schema-design time (turning a sentence into a schema you then review), and even that is opt-in with your own key. The generated bytes are produced by a deterministic solver with no network access, no telemetry, and no phoning home. There is no analytics or usage tracking anywhere in the library.

Can you reproduce a dataset exactly?#

Yes, to the byte. Same library version + same schema + same seed + same generation mode = same bytes, on any machine, any OS. This is a tested contract, not an aspiration: the suite asserts byte-identity across round-trips, and in anchored mode a schema edit changes only the columns it touches. A dataset is therefore fully specified by (version, schema file, seed), which is three lines in an audit trail.

Where do the realistic vocabularies come from?#

Three sources, all license-clean:

• Curated built-in pools (city geodata, curriculum topics, product vocabulary, support issues) maintained in the repository under MIT.
• Reference data packs in misata/data_packs/, each listed in SOURCES.toml with its source, licence, retrieval date and transform. Today that is one file: the population of each city Misata's own lists name, from GeoNames (CC BY 4.0), used only to weight which city a row lands in. It holds 336 numbers, not records about anyone.
• Wikidata-derived capsules, fetched at build time (never at generation time), carrying per-value provenance: the source QID and the CC0 license are recorded inside the capsule file itself. A capsule is a plain JSON file you can read, diff, and pin by hash.

Nothing is scraped from the open web at run time, and every capsule value passed a validator that rejects placeholders, prose, and junk.

What does the LLM see, when one is used at all?#

Only your schema intent: the sentence you type and the table/column names it implies. It never sees generated rows, never sees real data (there is none), and never participates in generation. Its output is a schema document you can read and edit before any data exists. Bring your own key; nothing is proxied through Misata infrastructure.

What guarantees does the output carry?#

Declared properties are exact and machine-checkable: aggregate curves land to the cent, ledgers balance, group shares hold, waterfalls and stock ledgers reconcile, foreign keys have zero orphans. misata audit re-checks the invariant catalog on the output, and an evalpack additionally re-executes every shipped question with DuckDB, an engine that shares no code with the generator. Where the engine cannot satisfy a declaration it refuses loudly; the failure modes are documented, not hidden.

Licensing#

The library is MIT. Generated data is yours, with no restrictions from us: it contains no third-party records to encumber it. Wikidata-derived vocabulary values are CC0. City population weights contain data from GeoNames (https://www.geonames.org), licensed under CC BY 4.0; the weights shape the distribution of generated rows and are not copied into them.

Domain priors (misata/domain_priors.py) are hand-set distribution families and parameters, not data: the order-value shape was checked against the Olist dataset (CC BY-NC-SA 4.0), which is used only as an evaluation set and is never bundled, and the realism benchmark labels Olist as tuned-on rather than blind. No runtime dependency is under a non-open-source licence: the Gaussian copula behind CopulaGenerator is Misata's own NumPy code (it used to wrap SDV, which is under the Business Source License). SDV appears only as an optional baseline in benchmarks/.

Fitting to real data is not anonymisation#

misata.mimic() and CopulaGenerator.fit() learn from real rows. Their output reuses category values and the sample's quantiles, so it is not anonymous and carries no privacy guarantee; see Mimic. Everything generated from a schema or story, with no real rows given, contains no third-party records.