How-to

How to connect a data lake to DataLens

Read from storage you already own and control, rather than copying a lake into somebody else's platform to look at it.

Request beta access Open the app

What you will do

You will connect an object store or lakehouse engine, point at the bucket, container or catalog holding your data, and read it as a dataset.

The Bring Your Own Lake option means the data stays in storage you control. DataLens reads from it with credentials you issue and can revoke.

When this is useful

  • Your data already lives in S3, ADLS or GCS and copying it elsewhere is not acceptable.
  • You run Databricks, Dremio or Trino and want to read through the engine rather than around it.
  • Policy requires that data stays in storage your organisation owns.
  • You want lake data beside operational and CRM data for a single analysis.

Before you start

  • Storage or engine credentials — S3 needs an Access Key ID and Secret Access Key with a Bucket Name and Region. ADLS Gen2 needs a Storage Account Name, Container and SAS Token. GCS needs a Bucket Name and Service Account JSON. Databricks needs a Server Hostname, HTTP Path and Personal Access Token.
  • The path to your data — A key prefix, object prefix or directory for object stores; a catalog and schema for the query engines.
  • The file format — Object store connectors ask what format the files are in.

Steps

  1. Open Data → Connections → Data Lake

    Select Data, then Connections, then the Data Lake tab.

    A Provider list, and a choice between Managed R2 and BYOL (Bring Your Own Lake).

  2. Choose managed storage or your own

    Managed R2 uses storage DataLens provides. BYOL (Bring Your Own Lake) points at storage your organisation owns and controls.

    If policy requires the data stays in your account, BYOL is the option that satisfies it.

  3. Pick your provider

    Choose from AWS S3, Azure Data Lake Gen2, Google Cloud Storage, Databricks / Delta Lake, Dremio, Starburst / Trino, or MinIO (S3-compatible).

    Apache Kafka and Apache Iceberg appear in the list marked as not yet available and cannot be selected.

    A credential form matching the provider you chose.

  4. Enter the credentials and location

    Fill in the fields for your provider — bucket and keys for S3, storage account and SAS token for ADLS Gen2, service account JSON for GCS, hostname and access token for Databricks.

    Set the prefix, path or catalog so the connector looks in the right place rather than at the root of a large bucket.

    Give the connection a name in Connection Name.

  5. Test the connection

    Test before going further. A failure here is a credential or permission problem, and separating it from a read problem saves a great deal of time.

    A success message, or a specific error from the provider.

  6. List the files and preview

    List what is at the path you configured, then preview a file to see its columns and sample rows before loading it.

    For the query engines, point at the table using the catalog.schema.table form the connector shows.

    A file or table listing with a preview action.

  7. Load it as a dataset

    Load the file or table. It becomes a dataset under Data → Datasets and behaves like every other dataset in the platform.

What happens next

Lake data is often already partly modelled. Take it into Model to see how it relates to your other sources, or straight into Analyse if you only need to interrogate it.

Where the lake is your target rather than your source, look at Deliver — Publish and Data Products are the surfaces for pushing governed output back out.

Example

A team keeps three years of event data as files in S3. Rather than restoring a slice into a database to analyse it, they connect the bucket with a read-only key scoped to one prefix, preview a file to confirm the schema, and load a month as a dataset to profile before deciding what to model.

Tips

  • Scope credentials to the prefix you need, not the whole account. The connector only needs to read the path you point it at.
  • Set the prefix or path rather than leaving it at the root. Listing a large bucket from the top is slow and rarely what you meant.
  • Test before listing. It separates a credentials problem from a path problem.
  • BYOL is the option to choose when data residency matters — the data stays in your storage account.

Limitations

  • Apache Kafka and Apache Iceberg are shown in the provider list but are not yet available for selection.
  • Object store connectors read the file formats offered in the File Format field for that provider. A format not on that list is not read.
  • A load is a point-in-time read. It does not track new files arriving in the bucket.
  • Credentials must permit listing as well as reading, or the connector can authenticate but find nothing.

Related how-to guides

Connect a database

Connect PostgreSQL, MySQL, SQL Server, Redshift, BigQuery or Snowflake, browse the tables and load what you need.

Build a data productPro

Turn scattered datasets into a defined, governed product with a named owner and a documented schema.

Publish and export data

Export as CSV, TSV, JSON, JSONL or Excel, publish an API endpoint, or generate a PDF or Word report.

Profile a dataset and find PII

See what is really in each column, and find the personal data before it ends up somewhere it should not.

Related questions

More of these on the DataLens FAQ page.

Which data lakes and lakehouse engines does DataLens support?

AWS S3, Azure Data Lake Gen2, Google Cloud Storage, Databricks / Delta Lake, Dremio, Starburst / Trino and MinIO. Apache Kafka and Apache Iceberg are listed in the connector but are not yet available.

Can I keep my data in my own storage account?

Yes. Choose BYOL (Bring Your Own Lake) rather than Managed R2 and point the connector at storage your organisation owns, using credentials you issue and can revoke.

Try this in DataLens

DataLens is in private beta. Request access and work through this guide on your own data.

Request beta access