Skip to main content

Starburst Enterprise - Data Product integration

The Starburst Enterprise connector extracts metadata from your Starburst Enterprise instance across two categories of assets: SQL assets (catalogs, schemas, tables) and data product assets (domains, data products, datasets). The connector links these together for unified discovery and governance.

What metadata is extracted​

SQL assets​

SQL assets are extracted from catalogs via SQL:

  • Catalogs (mapped to Database in Atlan)
  • Schemas
  • Tables
  • Views
  • Materialized views
  • Columns

This extraction queries INFORMATION_SCHEMA tables and the system.metadata catalog to discover and extract metadata from all selected catalogs.

Data product assets​

Data product assets are extracted from the Starburst Enterprise data products API:

  • Data domains: Organizational groupings for data governance
  • Data products: Curated data offerings that map 1:1 to schemas
  • Datasets: Views and materialized views within a data product (mapped to StarburstDataset in Atlan)
  • Dataset columns: Columns within datasets (mapped to StarburstDatasetColumn in Atlan)
  • Source ratings: Average rating and rating count from Starburst Enterprise, stored as Custom Metadata (Starburst Data Product Ratings) on each data product with color-coded badges

Cross-asset linking​

After extraction, the connector establishes relationships between SQL assets and data product assets:

Data product assetSQL assetRelationship
DatasetView or materialized viewDataset backed by the SQL view definition
Dataset columnColumnDataset column backed by the SQL column

These links enable you to navigate between the business-oriented data product view and the technical SQL view of the same data in Atlan.

Asset hierarchy​

Domains and data products hierarchy​

Connection
└── Data Domain
    └── Data Product
        └── StarburstDataset
            └── StarburstDatasetColumn

Catalog hierarchy​

Connection
└── Database (Catalog)
    └── Schema
        ├── Table
        │   └── Column
        ├── View
        │   └── Column
        └── Materialized View
            └── Column

The connector links datasets and dataset columns to their underlying SQL assets in Atlan. This enables you to navigate from a dataset to the cataloged view or column in the backing catalog.

  • Dataset → View / Materialized View: Each dataset within a data product maps to a view or materialized view in the backing schema. The dataset's asset page links to the underlying SQL object.
  • Dataset Column → Column: Each dataset column links to the corresponding column in the SQL view or materialized view.

The dotted links in the diagram represent cross-stream navigation—clicking these links in Atlan takes you from the data product hierarchy into the SQL catalog hierarchy.

Data product editability​

The connector provides a toggle to control whether data product entities are editable in Atlan:

  • Editable (default): Domains, data products, datasets, and dataset columns can be modified in the Atlan UI.
  • Read-only: Domains, data products, datasets, and dataset columns are published as read-only, preserving Starburst Enterprise as the authoritative source.

SQL entities (databases, schemas, tables, views, columns) are unaffected by this setting.

See also​