Skip to main content

Troubleshooting BigQuery errors

When querying Lakehouse Iceberg tables from BigQuery using external Iceberg tables, you may encounter errors related to region configuration, permissions, or metadata staleness. This page covers the most common errors and how to fix them.

Dataset not found in location​

Error

Not found: Dataset <project>:<dataset> was not found in location <region>

Cause​

The BigQuery connection, dataset, or query execution location doesn't match the region of the GCS bucket containing the Lakehouse data. All four resources must be in the same region.

Solution​

Make sure the following resources are all in the same region:

  • GCS bucket location (provided by Atlan)
  • BigQuery Cloud Resource connection location
  • BigQuery dataset location
  • Query execution location

If any of these are in a different region, recreate the connection or dataset in the correct region. Contact Atlan Support to confirm the GCS bucket region if needed.

Access denied to GCS bucket​

Error

Access denied: The service account does not have permission to read the GCS bucket

Cause​

The service account associated with the BigQuery Cloud Resource connection hasn't been granted read access to the GCS bucket containing the Lakehouse data.

Solution​

  1. Confirm that you have shared the Service Account ID from your BigQuery connection with Atlan Support. See Set up external tables in BigQuery.
  2. Confirm that Atlan has completed the data sharing step. The service account requires the roles/storage.objectViewer role on the GCS bucket.
  3. If Atlan has confirmed access but the error persists, re-run the setup script to refresh the external table metadata.

Invalid source URI​

Error

Invalid source URI: The URI type is not supported in this region

Cause​

The URI scheme of the Iceberg metadata files (GCS vs S3) doesn't match the connection type or region configured in BigQuery.

Solution​

Verify that your BigQuery Cloud Resource connection is of type Cloud Resource and is in the same region as the GCS bucket. The Lakehouse stores metadata in GCS, so the connection must point to a GCS-compatible region. If you created the connection in the wrong region, recreate it in the region provided by Atlan.

Manifest files not found after refresh​

warning

Error while reading data: Failed to expand table—matched no files

Cause​

An overnight compaction job has deleted old Iceberg manifest files that the external tables were still pointing to. This can occur when the refresh script was last run before the compaction completed.

Solution​

Re-run the setup script to refresh the external table metadata:

python bq_external_iceberg_tables_create_refresh.py

To prevent this from recurring, schedule the script to run at ~4:00 AM daily, after overnight compactions (which complete at ~3:00 AM).

See also​

Need help​

If you need assistance after trying these steps, contact Atlan Support.