Skip to main content

Troubleshooting Databricks errors

When creating or querying foreign Iceberg tables in Databricks Unity Catalog, you may encounter errors related to S3 access, metadata staleness, or missing workspace features. This page covers the most common errors and how to fix them.

S3 access denied when running notebooks​

Error

Access denied: The storage credential cannot read from the specified S3 location

Cause​

The create or refresh notebook ran before Atlan granted read access to the S3 bucket. The storage credential can't read the Lakehouse metadata files until Atlan updates their IAM trust policy.

Solution​

Wait for Atlan to confirm that read access has been granted, then re-run the notebook. See Set up Unity Catalog access for the full setup sequence.

Foreign tables not reflecting latest data​

warning

Query results are missing recent assets or show outdated metadata

Cause​

The refresh notebook hasn't been run since the Lakehouse data was last updated. Overnight compaction jobs can also delete old Iceberg manifest files that the foreign tables were still pointing to, causing queries to return stale or incomplete results.

Solution​

Re-run the refresh notebook to update the metadata pointers:

To prevent this from recurring, schedule the refresh notebook to run at ~4:00 AM daily, after overnight compactions complete at ~3:00 AM.

Private Preview feature not available on workspace​

Error

Unsupported feature: CREATE TABLE ... UNIFORM ICEBERG ... METADATA_PATH is not available

Cause​

The Foreign Iceberg Tables Private Preview feature hasn't been enabled on your Databricks workspace. This feature is required before any foreign Iceberg tables can be created.

Solution​

Contact your Databricks account representative and request enablement of the Foreign Iceberg Tables Private Preview feature. Once enabled, notify Atlan Support before proceeding. See Enable Databricks private preview.

See also​

Need help​

If you need assistance after trying these steps, contact Atlan Support.