Machine learning

Machine learning on data you already govern

The hard part of putting a model into production is rarely the model. It is knowing which version was approved for which environment, what data it was trained on, what left the building when it was called, and whether anyone was allowed to call it. DataLens records those four things as part of the platform the data is already in.

What is different about doing it here

The training data has a lineage already

A model trained here is trained on a dataset that was profiled, cleaned, modelled and governed in the same platform. Where the rows came from and what was applied to them is recorded before the model ever sees them.

Approval is per environment and it expires

A model version approved in development is not thereby approved in production, and an approval that has passed its expiry is treated as absent rather than as a lingering yes.

Absent means no

If the trust record for a model version cannot be found, or cannot be read, the platform refuses the run. A gate that opens when its own store is unavailable is not a gate.

A refused run is still a run

A blocked invocation is written down with the reason, not silently dropped. "The model was never called" and "there is no record of it" are different answers to an audit.

Acting on models is a separate grant

Being an editor of a workspace does not make you a data scientist in it. The ML capability is granted per person per workspace, and removing it is a stamped revocation rather than a deleted row.

Features carry their sensitivity

A feature knows the sensitivity of the columns it was derived from, so the question of what may be sent to an external model has an answer before anyone has to remember one.

Why this sits inside the data platform

Most ML governance is retrofitted: a registry in one tool, the training data in another, the approval in a spreadsheet, the lineage in nobody. Each of those is correct on the day it is written and drifts afterwards, because keeping them agreeing is a job nobody is measured on.

The alternative is not a better registry. It is a registry that sits beside the datasets, so the question "what was this version trained on" is answered by a foreign key rather than by a convention about naming things. That is the only claim DataLens makes here, and it is the reason the ML tables were added to the same database as the catalogue rather than to a service of their own.

The same reasoning produced the evaluation decision. Scoring predicted values against known-correct values is one piece of arithmetic, so an ML evaluation is recorded with the same shape as an AI prompt evaluation instead of in a parallel set of tables that would disagree with it within a release.

Where this is today

The machine learning layer is in build. What is described on this page is the data and governance model that is implemented in the platform: the tables that hold it, the rules that enforce it, and the refusals that happen when a rule is not satisfied.

The Data Science tab is in the application. It reads: the model registry and its versions, the approval held for each version in each environment, the integrations and whether each one may currently be called, the record of every run including the refused ones, the projects, and the egress rules. It does not invoke a model - running belongs to the execution provider, and the record of a run is audit evidence that a browser must not be able to write.

Questions about the ML layer

Does DataLens train models for me?

No. DataLens prepares and governs the data, records the model and its versions, decides whether a version may run in a given environment, and keeps the record of what happened. Training and serving happen where you already do them - a REST endpoint, Azure ML, SageMaker, Vertex AI, Databricks, Fabric, Snowflake, MLflow or a Hugging Face endpoint.

Can I use a model that was not built in DataLens?

That is the expected case. A model version is registered with its framework, licence and execution provider, and DataLens holds the connection and the input and output contracts that describe how to call it.

What stops a model being called on data it should not see?

Three independent checks, and all three must pass: the caller holds the data scientist capability in that workspace, the model version is approved for that environment and the approval has not expired, and the egress rules for the fields being sent allow the call. Any one of them failing refuses the run.

Is my data sent to a model provider?

Only if you configure an integration that calls one, and only the fields the egress rules allow. Rules act per field and can allow, mask, transform, require approval, or block outright.

Is the ML layer available now?

The data and governance model is implemented, and the Data Science tab reads it: registry, per-environment approval, integrations, runs, projects and egress rules. Invocation is not run from DataLens - the model runs where it already runs, and DataLens records and governs it. DataLens itself is in private beta.

Talk to us about the ML layer

DataLens is in private beta. If model governance is the reason you are here, say so when you request access - the ML layer is being built against real use rather than a roadmap.

Request beta access