Scores and runs, both kept
Two records make a model reviewable: what it scored when somebody measured it, and what it has actually been doing since. DataLens keeps both, and keeps them where the equivalent records for AI steps already live.
What a run records
Which version, through which integration
The run points at the binding that was used, so the provider and the connection are part of the record rather than inferred from a timestamp.
Status
Pending, submitted, running, succeeded, failed, cancelled or blocked. Seven states, and a run is always in exactly one.
A reason when it did not work
A failed or blocked run must carry an error code. The platform refuses to record a failure that does not say why.
The external job reference
An asynchronous run that has left the platform records the handle it was given, so it can be followed up rather than assumed.
Refusals
A run blocked by the trust gate, the capability check or the egress rules is written down as a blocked run with its reason.
Lineage links
What the run touched, joined into the same lineage the rest of the platform records.
Why a refused call is still written down
The easy implementation returns an error to the caller and writes nothing. Six months later the question arrives - has anyone tried to run this model against production since we blocked it - and there is no way to answer it.
A refusal is information about how the platform is being used, and it is the information a control is judged by. So a blocked run is a row with a status and a reason, and the absence of rows means the absence of attempts.
One evaluation model, two kinds of thing being evaluated
Scoring predicted values against known-correct values is one piece of arithmetic whether the prediction came from a model or from a prompt. Building a second set of evaluation tables for ML would have produced two implementations of that arithmetic that disagreed within a release, so an ML evaluation is recorded in the same place as an AI evaluation, marked as an ML inference evaluation.
The price of that reuse is one rule, and it is enforced in the stored query, in the in-memory path and by name in the promotion check: an ML evaluation can never satisfy a gate that was asking about a prompt. A model version scoring well says nothing about whether a prompt behaves, and a system that let the first stand in for the second would be approving text it had never measured.
Questions about evaluation and runs
What is a model evaluated against?
A golden dataset - a set of rows with known-correct values, versioned and approved - which is the same mechanism the platform already uses to evaluate AI steps.
Can a good evaluation approve a model for production?
An evaluation is evidence for the approval decision, not the decision. Approval is recorded separately, per environment, with an expiry.
Does an ML evaluation count towards promoting a prompt?
No, and that is enforced in three independent places rather than left to a convention.
How long are runs kept?
Runs are records in your workspace and are subject to the same retention decisions as the rest of the data in it.
Where this is today
The machine learning layer is in build. What is described on this page is the data and governance model that is implemented in the platform: the tables that hold it, the rules that enforce it, and the refusals that happen when a rule is not satisfied.
The Data Science tab is in the application. It reads: the model registry and its versions, the approval held for each version in each environment, the integrations and whether each one may currently be called, the record of every run including the refused ones, the projects, and the egress rules. It does not invoke a model - running belongs to the execution provider, and the record of a run is audit evidence that a browser must not be able to write.
The rest of the ML layer
Model registry
Every model and model version recorded once, with its task, framework, licence and where it runs.
Model trust and approval
A model version is approved per environment, with an expiry, or it does not run. Absent approval is a no.
Feature store
Versioned features and feature sets, each carrying the sensitivity of the columns it was built from.
Training data and splits
Dataset snapshots and reproducible train, validation and test splits that can be rerun and compared.
ML projects and access
A project gathers datasets, features, models and integrations, and only granted data scientists can act.
Data egress controls
What leaves the platform when a model is called elsewhere, decided per field before the call is made.
Talk to us about the ML layer
DataLens is in private beta. If model governance is the reason you are here, say so when you request access - the ML layer is being built against real use rather than a roadmap.
Request beta access