Training data you can produce again
The question that sinks a model review is not how it scored. It is which rows it scored on, and whether anyone can get them back. A split that was made once in a notebook is not an answer to that question.
Roles a dataset can take
A dataset in the catalogue is given a role in ML rather than copied into a second place.
Training
The rows a version learned from.
Validation
Held back for tuning, and recorded as such so it cannot quietly become training data.
Test
Held back for the score that gets reported.
Scoring
Rows a model is run over in production rather than learned from.
Inference input and output
What went into a call and what came back, both roles in their own right so a result set is traceable to its input.
Feature set
A dataset that exists to carry the features a model consumes.
How a split is defined
Random
The plain case, with the seed recorded so it is the same split next time.
Stratified
Proportions preserved across the columns you name, each column recorded as a row rather than a delimited list.
Time based
Split on a point in time, for the many problems where a random split leaks the future into the past.
Custom filter
An explicit predicate, when the split is a business rule rather than a statistical one.
Parts
The proportions each part takes. A definition whose parts do not add up is refused rather than silently normalised.
Runs and outputs
Executing a split definition produces a run, and the run records what each part became. The definition is the intent; the run is the fact.
Snapshots, and why the split is not enough on its own
A split definition applied to a dataset that has since gained ten thousand rows produces a different split, correctly and unhelpfully. Reproducibility needs the data to be pinned as well as the rule.
A dataset snapshot is that pin: the state of the dataset at the moment the split was taken, so rerunning the definition against the snapshot gives back the same rows. Rerunning it against the live dataset gives the current answer. Both are useful, and the difference between them is exactly what a model review needs to be able to see.
Questions about training data
Is the data copied when I snapshot it?
A snapshot records the state a dataset was in so the split can be reproduced. It is a record about the dataset, not a second catalogue of duplicates to keep in step.
Can one dataset serve more than one role?
Yes. Roles are recorded per dataset, and the same dataset can be the scoring input for one model and a training source for another.
Why record the seed?
Because "random" without a seed means the split cannot be produced twice, and a score that cannot be reproduced is an anecdote.
Where this is today
The machine learning layer is in build. What is described on this page is the data and governance model that is implemented in the platform: the tables that hold it, the rules that enforce it, and the refusals that happen when a rule is not satisfied.
The Data Science tab is in the application. It reads: the model registry and its versions, the approval held for each version in each environment, the integrations and whether each one may currently be called, the record of every run including the refused ones, the projects, and the egress rules. It does not invoke a model - running belongs to the execution provider, and the record of a run is audit evidence that a browser must not be able to write.
The rest of the ML layer
Model registry
Every model and model version recorded once, with its task, framework, licence and where it runs.
Model trust and approval
A model version is approved per environment, with an expiry, or it does not run. Absent approval is a no.
Feature store
Versioned features and feature sets, each carrying the sensitivity of the columns it was built from.
ML projects and access
A project gathers datasets, features, models and integrations, and only granted data scientists can act.
Evaluation and runs
Model scores recorded against the same evaluation model as AI steps, and every invocation kept as a run.
Data egress controls
What leaves the platform when a model is called elsewhere, decided per field before the call is made.
Talk to us about the ML layer
DataLens is in private beta. If model governance is the reason you are here, say so when you request access - the ML layer is being built against real use rather than a roadmap.
Request beta access