Realism report: find the tells of generated data

misata audit catches contradictions: orders shipped before they were placed, ages that disagree with birth dates. Data can pass every one of those checks and still look fake to anyone who has worked with a production table, because its shapes are wrong. realism_report looks for those shapes. It needs no real data to compare against, and it works on data from any generator: a Faker script, SDV, Mockaroo, or Misata.

import misata

tables = misata.generate("An ecommerce store with 2000 customers and 10000 orders")
report = misata.realism_report(tables)
print(report.summary())
report.passed   # False if any check failed outright
report.score    # 0.0 - 1.0: pass = 1, warn = 0.5, fail = 0

misata realism ./seed_data/                 # exit 1 on any failure
misata realism ./seed_data/ --strict        # exit 1 on warnings too
misata realism ./seed_data/ --skip too_clean --json   # the report is data.realism

The checks#

Each check runs only when the table is big enough for the statistic to mean something, so a five-row fixture gets a short report. Every finding carries its evidence (report.to_dict()): the Gini, the chi-square p-value, the hour histogram, the examples that matched.

The free-text checks#

"Free text" means prose columns: names such as description, review_text, subject, notes, bio, or any text column whose values have a median of five words or more. Emails, addresses, names, companies, codes and log or error columns are left out; real system logs are legitimately templated.

The thresholds come from a local calibration on seven real English corpora (product and hotel reviews, news, tweets, forum posts and titles), 2,000 rows each. Every real corpus compressed under 3x, had over 70% distinct word trigrams, under 16% repeated skeletons, under 0.2% near-copies, a length CV over 0.29 and a top-10 opener share under 0.28. Template text sits 5x to 50x outside those bands, and each threshold sits well inside that gap.

A skeleton is the text with numbers masked to #, emails to @E and every word that is not a stopword to W. "The kettle arrived on time" and "The lamp arrived on time" share one; people almost never write the same skeleton twice at sentence length, while slot-filling templates do on every row.

These checks are strict. Misata's own text passes the repetition and context checks for every kind, and text_diversity for every kind except chief complaints (gzip about 4.2x at 2,000 rows): seven-word triage fragments, which real triage text does not beat either; see LIMITATIONS. Skip text_diversity when the vocabulary does not matter to you, for example in UI fixtures.

What a clean report means#

Each threshold is a heuristic chosen to flag shapes real data almost never has. A clean report means none of the known tells is present. It does not mean the data is indistinguishable from production; for that you need real data to compare against (misata.fidelity_report).

Some tells are legitimate in context. A randomized A/B assignment is balanced (columns named variant, group, arm, cohort and similar are exempt), a nightly batch job does run at midnight, and a pristine demo table may be exactly what you want. Skip what does not apply:

misata.realism_report(tables, skip=["too_clean", "daily_rhythm"])