Skip to main content

Manage Databricks tags

You must have a Unity Catalog-enabled workspace to import Databricks tags in Atlan.

Atlan enables you to import your Databricks tags, update your Databricks assets with the imported tags, and push the tag updates back to Databricks:

  • Import tags - crawl Databricks tags from Databricks to Atlan
  • Reverse sync - sync Databricks tag updates from Atlan to Databricks

Once you've imported your Databricks tags to Atlan:

  • Your Databricks assets in Atlan are automatically enriched with their Databricks tags.
  • Imported Databricks tags are mapped to corresponding Atlan tags through case-insensitive name match - multiple Databricks tags can be matched to a single tag in Atlan.
  • You can also attach Databricks tags, including tag values, to your Databricks assets in Atlan - allowing you to categorize your assets at a more granular level.
  • You can filter your assets by Databricks tags and tag values.
  • You can enable reverse sync to push any tag updates for your Databricks assets back to Databricks - including tag values added to assets in Atlan.
Did you know?

Enabling reverse sync only updates existing tags in Databricks. It neither creates nor deletes any tags in Databricks.

Prerequisites​

You must have a Unity Catalog-enabled workspace and SQL warehouse configured to import Databricks tags in Atlan.

Before you can import tags from and push tag updates to Databricks using personal access token, AWS service principal, or Azure service principal authentication, you need to do the following:

Import Databricks tags to Atlan​

Who can do this?

You need to be an admin user in Atlan to import Databricks tags to Atlan. You also need to work with your Databricks administrator to grant permissions to import tags from Databricks - you may not have access yourself.

You can import your Databricks tags to Atlan through one-way tag sync. The synced Databricks tags are matched to corresponding tags in Atlan through case-insensitive name match and your Databricks assets are enriched with their synced tags from Databricks.

To import Databricks tags to Atlan, you can either:

  • Create a new Databricks workflow and configure the crawler to import tags.
  • Modify the crawler's configuration for an existing Databricks workflow to change Import Tags to Yes. If you subsequently modify the workflow to disable tag import, for any tags already imported, Atlan preserves those tags.

Once the crawler has completed running, tags imported from Databricks are available to use for tagging assets! 🎉

View Databricks tags in Atlan​

Once you've imported your Databricks tags, you can view and manage your Databricks tags in Atlan.

To view Databricks tags:

  1. From the left menu of any screen, click Governance.
  2. Under the Governance heading of the _Governance cente_r, click Tags.
  3. (Optional) Under Tags, click the funnel icon to filter tags by source type. Click Databricks to filter for tags imported from Databricks.
  4. From the left menu under Tags, select a synced tag.
  5. In the Overview section, you can view a total count of synced Databricks tags. To the right of Overview, click Synced tags to view additional details - including tag name, description, tag values, total count of linked assets, connection, database, and schema names, and timestamp for last synced.
  6. (Optional) Click the Linked assets tab to view linked assets for your Databricks tag.
  7. (Optional) In the top right, click the pencil icon to add a description and change the tag icon. You can't rename tags synced from Databricks.

Push tag updates to Databricks​

Who can do this?

Any admin or member user in Atlan can configure reverse sync for tag updates to Databricks. You also need to work with your Databricks administrator to grant additional permissions to push updates - you may not have access yourself.

You can enable reverse sync for your imported Databricks tags in Atlan and push all tag updates for your Databricks assets back to source. Once you have enabled reverse sync, any Databricks assets with tags updated in Atlan are also updated in Databricks.

To enable reverse sync for imported Databricks tags:

  1. From the left menu of any screen, click Governance.
  2. Under the Governance heading of the _Governance cente_r, click Tags.
  3. (Optional) Under Tags, click the funnel icon to filter tags by source type. Click Databricks to filter for tags imported from Databricks.
  4. In the left menu under Tags, select a synced Databricks tag - synced tags display the Databricks icon next to the tag name. 
  5. On your selected tag page, to the right of Overview, click Synced tags.
  6. Under Synced tags, in the upper right, turn on Enable reverse sync to synchronize tag updates from Atlan to Databricks.
  7. In the corresponding confirmation dialog, click Yes, enable it to enable reverse tag sync or click Cancel.

Now when you attach Databricks tags to your Databricks assets in Atlan, these tag updates are also pushed to Databricks! 🎉

Did you know?

Enabling reverse sync won't trigger any updates in Databricks until synced tags are attached to Databricks assets in Atlan.