Pipelines you can see, run and promote
A flow in DataLens is a canvas, not a YAML file. Sources, transforms, joins, branches, quality gates, AI steps and script nodes are laid out, connected, and run from the same screen you designed them on.
What goes on the canvas
Sources
Any connected source, or a dataset already in the catalogue.
Transforms
The same operations as the preparation stage, applied as pipeline steps rather than one-off actions.
Joins
Designed on their own canvas, with key matching and match quality shown before the join is wired in.
Quality gates
A rule that stops the run when the data does not meet it, so bad data fails loudly instead of arriving quietly.
AI steps
A model call as a pipeline node, bound to a versioned prompt and a model provider connection.
Script nodes
For the step that is genuinely easier to write than to draw. The escape hatch is part of the design, not an admission of defeat.
Control flow
Branches and conditions, so a flow can take a different path for a different kind of input.
Variables
Values passed through the run rather than hard-coded into every node that needs them.
Runs, and what you can see about them
A run reports what it did per stage: rows in, rows out, what each gate decided, and where it stopped if it stopped. A failure names the node and the reason rather than surfacing a stack trace from three layers down.
AI steps are accounted for separately, because they are the ones with a bill attached. Cost, tokens, latency and cache hits per step are recorded, along with the estimate the platform made before the run so you can see where estimates and reality diverge.
Releases and environments
A release is a bundle
Models, mappings, prompts, golden datasets and flows sealed together, because promoting one without the others is how environments drift.
Promotion
Move a sealed release through environments as a unit, with the comparison between what is live and what is proposed shown first.
History
What was deployed, when, and by whom — including the bindings each environment supplied.
Golden datasets
A known input with a known output, kept with the release so a change that breaks it is caught before promotion rather than after.
Orchestration questions
Is this a replacement for Airflow or dbt?
For the work that happens between a source and a governed output, it covers the same ground without a separate repository, scheduler and deployment story. Teams with a large existing DAG estate usually adopt it for new pipelines first.
Can I run a flow on a schedule?
Yes. Flows run on demand and on a schedule, and a scheduled run reports the same per-stage detail as a manual one.
What happens when a run fails halfway?
The run stops at the failing node and reports which one and why. Because transforms are applied to working copies, the source datasets are not left in a partial state.
See it on your own data
DataLens is in private beta. Bring a file, a database or an API and work through the whole lifecycle in one sitting.
Request beta access