Solve for a pipeline's output

An outcome curve fixes what Misata generates: the total of orders.amount in each month, exactly. The number people look at is usually computed later, by a pipeline Misata does not own: net revenue after a fee and refunds, in a dbt model. misata solve closes that gap. It generates, runs your pipeline, compares each metric with its target, adjusts the curves you name as levers, and repeats until every period is within tolerance.

The result is a dataset that, run through your real pipeline, produces the numbers you asked for, plus a solved schema that regenerates it exactly.

 solve.yaml: targets per period, one lever (an outcome curve) per target
      │
      ▼
 ┌─▶ schema with the lever's curve points ──▶ feasibility gate (exit 3 if not)
 │        │
 │        ▼
 │   generate (same seed every iteration)
 │        │
 │        ▼
 │   your pipeline: DuckDB SQL · dbt on DuckDB · a Python function
 │        │
 │        ▼
 │   metric per period vs target ── every period within tolerance? ──▶ converged
 │        │ no                                                          │
 │        ▼                                                             ▼
 └── step each lever: coupled least squares,          write: tables, misata.solved.yaml,
     else secant, else damped ratio                   schema.solved.json, solve.json,
     (never past zero, never more than 20x)           an attestation (SQL: each solved period)

A first solve#

The schema needs an outcome curve with exact values on each lever:

# misata.yaml
name: shop
seed: 11
tables:
  orders:
    rows: 3000
    columns:
      order_id: {type: int, unique: true}
      amount: {type: float, min: 5, max: 500}
      ordered_at: {type: datetime, start: "2025-01-01", end: "2025-03-31"}
outcome_curves:
  - table: orders
    column: amount
    time_column: ordered_at
    value_mode: absolute
    curve_points:
      - {month: 1, target_value: 50000}
      - {month: 2, target_value: 52000}
      - {month: 3, target_value: 55000}

The solve config names the schema, the pipeline and the targets:

# solve.yaml
schema: misata.yaml
tolerance: 0.5%
pipeline:
  sql:
    net_revenue: |
      SELECT date_trunc('month', ordered_at), SUM(amount) * 0.97
      FROM orders GROUP BY 1
targets:
  - metric: net_revenue
    lever: orders.amount
    values: {2025-01: 70000, 2025-02: 65000, 2025-03: 72000}

pip install "misata[duckdb]"
misata solve -c solve.yaml -o solved --format parquet

solve: converged after 4 iteration(s) in 0.3s (tolerance 0.5%)
  net_revenue: worst period off by 0.1205% (iteration 4)
wrote solved (PASSED)

solved/ holds the tables, schema.solved.json, misata.solved.yaml (your schema with only the lever's curve points replaced), solve.json with every iteration, and an attestation. Generating from either solved schema reproduces the solved tables exactly.

Pipelines#

Each metric returns two columns, the period and the value, one row per period. A period matches a curve period when it falls inside it, so a date_trunc('month', ...) timestamp, a 2025-03 label and the 15th of March all name March on a monthly curve.

SQL. DuckDB queries over the generated tables, one per metric. They run in a sandbox: the tables are readable, and files, extensions, ATTACH and configuration changes are refused.

dbt. A dbt project run on a temporary DuckDB warehouse:

pipeline:
  dbt:
    project_dir: analytics          # relative to solve.yaml
    select: +fct_revenue            # optional
    source_schema: main             # where your sources read the raw tables
    seeds: true                     # optional: dbt seed first (true, a name, or a list)
    metrics:
      net_revenue: SELECT month, net_revenue FROM fct_revenue

The generated tables are loaded into source_schema, dbt seed loads the seeds you name (a package's lookup tables, a fee schedule), dbt run builds the selected models with a generated profile (your project's profile name, adapter duckdb), and the metric queries read the built models. Nothing is written into the project folder. Needs pip install dbt-duckdb, and dbt deps first if the project uses packages.

Python. From the library, any function of the tables:

import misata

def net_revenue(tables):
    orders = tables["orders"]
    month = orders["ordered_at"].dt.to_period("M")
    return {"net_revenue": orders.groupby(month)["amount"].sum() * 0.97}

result = misata.solve(
    schema,
    net_revenue,
    [misata.Target("net_revenue", "orders.amount", {"2025-01": 70000, "2025-02": 65000})],
    tolerance="0.5%",
)
print(result.summary())
result.write("solved")
result.attest("solved")

Direction#

A lever is direct when more of it raises the metric. Refunds lower net revenue, so a refund curve moving net revenue is inverse:

targets:
  - metric: net_revenue
    lever: refunds.amount
    direction: inverse
    values: {2025-01: 45000, 2025-02: 49000}

Declaring the wrong direction makes the gap widen, and the run stops as diverging and says so.

How it steps#

Each lever takes the first of these steps that applies:

1. Coupled least squares. Pipelines often move rows across a period boundary (a time-zone shift, a fiscal calendar), so feeding more into March also raises April. Once a lever has one more observation than it has periods, the solver fits how every period's output depends on every period's input and solves for the inputs that hit all targets at once. The fit is used only when it is well conditioned and does not reach far past the values it was fitted on.
2. Secant. For a period whose last two observations slope the way its direction says, the step goes to where that line meets the target. Pipelines that add or subtract the lever (net = gross - refunds) are lines, and this solves them in two or three iterations.
3. Damped ratio. Otherwise, the value is multiplied by (target / measured) ** 0.85.

No step grows a value more than twentyfold, crosses zero, or exceeds 1e12. Every iteration uses the schema's seed and generates from a complete schema, so the feasibility gate checks every candidate before it runs.

When it stops#

A config Misata cannot use (an unknown field, a lever with no curve, a period outside the curve) exits 2 before anything runs.

Generation is random within each period. A pipeline that reads rows across period boundaries therefore sees some noise between iterations, about a percent for a few thousand rows a month. A tolerance below that noise is reached by luck rather than by steering. Raise the tolerance, or add rows.

The certificate#

With a SQL pipeline, every solved period becomes an attestation invariant with comparator: within_tolerance. verify.py reruns your metric query against the written files and checks it is within the tolerance of the target, in exact decimals. A solved dataset can therefore prove, without Misata, that your query over it gives the numbers you declared.

The solver decides convergence with the same comparison the verifier uses, so a run that reports converged writes a certificate that passes.

dbt and Python pipelines need more than the written files to rerun, so their solved periods are recorded in solve.json, with the pipeline's identity (a digest of the dbt project's files, or of the function's source), rather than as invariants.

Proven on pipelines Misata did not write#

Two external engines, run unmodified, each turning generated rows into a number Misata never computes itself (measured 26 October 2026, M1 laptop):

Tax is not linear in wages: brackets, the EITC phase-out and the Child Tax Credit make doubling wages more than double the tax, which is why the secant step does the work here. The Tax-Calculator proof runs in the test suite when taxcalc is installed (tests/test_pipeline_proofs.py).

See it end to end#

Known-answer tests for dbt models declares the monthly totals a model must produce when the model only sums them. When the model transforms the rows first (fees, refunds, a fiscal calendar), give the same targets to misata solve with a dbt pipeline.

From an agent#

The MCP tool solve_outcome takes a schema dict, SQL queries and targets, and returns the solved curve points per lever. It writes nothing, and it runs SQL pipelines only: each query in the sandbox above, limited to 30 seconds and 2 GB.