Build Spark pipelines you can trust before they run.
Draw the DAG on a visual canvas or write YAML — both compile to the same typed Scala engine. Every step is validated, previewed and measured before a single Spark core is spent.
Canvas and YAML, one engine
Every node on the canvas is a typed step — JDBC and Kafka ingestion, JSON/Avro parsing, SQL transforms, variables and UDFs. Prefer code? The same pipeline is plain YAML, reviewable in a pull request.
Predictive dry-run
Sample-execute each step against the live source via Trino before running Spark. You see row estimates, output schema and a data preview per node — schema mistakes die on the canvas, not in production.
Canvas time-machine
Scrub through any past run directly on the canvas: nodes light up in execution order with live record and byte counters, at 1×, 2× or 4× playback. Debugging a run becomes watching it.
Run data-diff
Compare two runs by what changed in the data — rows, bytes, duration and per-step deltas — not just what changed in the code. Regressions surface as numbers, not surprises.
Agentic copilot
On a failed run the copilot classifies the root cause, proposes the fix and computes the downstream blast radius through column-level lineage. You can also edit the canvas in plain language.
Real metrics per step
Rows, bytes, shuffle, memory peaks and wall-clock per step, pulled from the live Spark API during the run. No estimates, no sampling — the actual numbers.