Troubleshooting Google BigQuery connectivity

TL;DR

Symptom-first troubleshooting for Google BigQuery connectivity - VPC-SC denials, generic connection errors, WIF failures, key rotation, and cross-project permission issues.

Connect

Resolve common Google BigQuery connectivity issues when integrating BigQuery with Atlan. Find the symptom you're seeing in the quick index - each entry includes what you can check yourself before opening a ticket.

Quick index

You see	Most likely cause	Jump to
`403 … Request is prohibited by organization's policy` (anywhere)	VPC Service Controls perimeter	VPC-SC denials
"Unable to connect to the configured BigQuery instance" at setup or preflight	One of four very different things - walk the checklist	The generic error
WIF test fails at "Step 2/3" with an STS 403 or 400	Perimeter rule scoped to the wrong identity, or provider ID format	WIF failures
`invalid_grant … Invalid JWT Signature` / "Provided Service account JSON is invalid" on runs that used to work	Key rotated or deleted in Google Cloud	Key rotation
"Access denied. The service account does not have the required permissions." with correct-looking grants	Role missing on the account's home project	Cross-project
Crawl fails only on some external tables: "Permission denied while globbing file pattern"	Delta tables need Cloud Storage permissions	Special cases
Crawl "succeeds" but newly added projects never show up	Silent VPC-SC denial on the new projects	VPC-SC denials

VPC-SC denials - organization's-policy 403

Symptom. Anywhere in a test, preflight, crawl, or miner log:

403 VPC Service Controls: Request is prohibited by organization's policy.
vpcServiceControlsUniqueIdentifier: <identifier>
-- audit log: violationReason: NETWORK_NOT_IN_SAME_SERVICE_PERIMETER

Why. Your organization's VPC Service Controls perimeter is blocking the call. This is a policy decision on your Google Cloud side - credentials, permissions, and PSC cannot fix it, and it also strikes tenants that already have PSC.

Check yourself. Give the vpcServiceControlsUniqueIdentifier to your security team with the Cloud Logging query from VPC Service Controls & allowlisting. The log names the blocked service, method, and caller. Three recurring shapes:

Blocked service is sts.googleapis.com or iamcredentials.googleapis.com → WIF rules scoped to the wrong identity - see WIF failures.
Blocked call targets a newly added project → that project isn't in the ingress rules yet. Note: the crawl may still report success while skipping it.
No violation found in the project you searched → you're searching the wrong project; these logs usually live in the org-level/centralized audit sink.

Fix. Ingress rules per VPC Service Controls & allowlisting. Mention any perimeter change on your Atlan ticket too - Atlan may need to mirror it.

"Unable to connect to the configured BigQuery instance"

Symptom. The catch-all error at test connection or preflight:

Unable to connect to the configured bigquery instance,
please check your credentials and configs

Why. This one message wraps at least four unrelated root causes. Walk this list in order:

Cloud Resource Manager API disabled (most common). Preflight needs it; the crawl doesn't - so this is often a false alarm on an otherwise perfect setup. Fix: gcloud services enable cloudresourcemanager.googleapis.com --project=example-project, then re-run preflight.
Your project filter matches nothing. If the connection's include/exclude regex excludes every project the service account can see, the run dies looking auth-broken. Re-check filters against the actual project list - especially right after a filter edit.
The stored key is stale. See Key rotation.
None of the above and auth tests green? Possibly an Atlan-side issue. Open a ticket and say explicitly which of steps 1–3 you ruled out.

WIF failures - STS step 403 / 400

Symptom. Test authentication fails partway through the three-step WIF handshake:

Step 2/3: GCP STS error: status 403:
{"error":"access_denied","error_description":"Request is prohibited by
organization's policy. vpcServiceControlsUniqueIdentifier: <id>"}
-- or: Failed to exchange token via Google STS: 400

Why and fix.

STS 403 with the policy text → VPC-SC ingress rules for sts.googleapis.com / iamcredentials.googleapis.com are scoped to the service account instead of the WIF principal. Hand your security team the table in VPC Service Controls & allowlisting.
STS 400 → the trust setup itself: wrong WIF Pool Provider ID format (must start //iam.googleapis.com/projects/<number>/…/providers/<provider> - if your string contains /subject/ it's the wrong one), audience mismatch with the OAuth Client ID, or the service account missing roles/iam.workloadIdentityUser on the pool.
Error mentions "Service account JSON" but the connection uses WIF? Misattributed error text - there is no JSON key to fix; debug the WIF chain above instead, and say "this connection uses WIF, not a key file" in any ticket.

Key rotation - scheduled runs break overnight

Symptom. Crawls/miners that ran fine for months start failing every run:

400 Bad Request {"error":"invalid_grant",
"error_description":"Invalid JWT Signature."}
-- or: Provided Service account JSON is invalid.

Why. Someone rotated, disabled, or deleted the service account key (or the account itself) in Google Cloud; the copy stored in Atlan is now dead. This can cause multi-week silent failure streaks - nobody connects the GCP key hygiene job to the Atlan schedule.

Check yourself. In Google Cloud: IAM & Admin → Service Accounts → your Atlan account → Keys - does a key with the same ID as the one you uploaded still exist and show enabled?

Fix. Generate a new key, update the credential in Atlan (UI, or API - see the rotation runbook, and avoid the /credentials/test trap), re-run Test connection, then revoke the old key. Prevention: add the Atlan credential to your rotation runbook, or move to WIF and stop having keys.

Cross-project permission denials

Symptom. "Access denied. The service account does not have the required permissions." - yet the role is definitely granted on the project you're crawling.

Why. Test connection verifies permissions against the service account's home project (the one it was created in), via cloudresourcemanager.googleapis.com…:testIamPermissions. If the role is bound only on the target projects, the home-project check finds nothing. Google's API also can't report which permission is missing, so the error stays vague.

Fix. Bind the Atlan custom role on the home project as well as every target project. If previews/querying are enabled, confirm bigquery.tables.getData is present too.

PSC questions

"What's my private endpoint DNS?" - It's tenant-specific and not shown in the product today; ask on a support ticket (Phase 1 wording).
"Is there something we must approve?" - No. BigQuery PSC endpoints are auto-approved by Google; there is no pending state on your side.
"Can we point other tools/sources at it?" - No. The endpoint serves Google-managed APIs for your Atlan tenant; a database on a GCE VM, for example, needs its own connectivity.
"We have PSC - why are we still getting policy 403s?" - PSC changes the road, not the rules. VPC-SC evaluates the calls regardless; see VPC-SC denials.

Opening ticket that gets solved fast

Include these six things - they usually turn multi-day exchanges into one reply:

Your Atlan tenant URL and the failing workflow run link
Auth method - service account key or WIF (support cannot always tell from the error, and it changes the whole diagnosis)
Network path - public endpoint or Private Service Connect
The exact error text from the workflow logs - and for any policy 403, the vpcServiceControlsUniqueIdentifier value
Whether your organization uses VPC Service Controls, and whether the perimeter or the project list changed recently
Whether it ever worked, and if so when it stopped (check for key rotations and filter edits around that date)

Other known issues

How to debug test authentication and preflight check errors?

Invalid project ID

Provided GCP project ID is invalid, please check and try again.

Ensure that the project ID is non-empty and matches the expected project in your Google Cloud console.

Invalid service account JSON

Following are the possible error messages for this issue:

Provided Service account JSON is invalid, please check and try again.

Private key in the service account JSON is invalid, please check and try again.

Failed to sign service account access token request with the provided private key. Check the service account JSON and try again.

Malformed JSON, please check and try again.

These indicate issues with the service account JSON key, possibly invalid or malformed data, or incorrect private key or formatting.
Verify that the service account JSON is correctly formatted, and the private key is correctly specified.
Regenerate the service account key if needed, and ensure that all required fields are included.
Ensure that the file is not corrupted and follows a proper JSON structure.

Insufficient permissions

Service account doesn't have permission to create jobs, please ensure that the service account has the 'bigquery.jobs.create' permission.

Ensure that you have assigned the bigquery.jobs.create permission to the service account.
Review the roles and permissions assigned to the service account in your Google Cloud IAM settings.

Cloud Resource Manager API disabled

Cloud Resource Manager API has not been used in the configured project before or it is disabled. Please enable it and try again after some time.

Open your Google Cloud console and enable the Cloud Resource Manager API for the project.
Wait for the API to be fully activated before retrying the operation.

Invalid grant, service account not found

Unable to get access token for the provided service account, ensure the service account is active and try again.

The service account is either inactive or does not exist.
Ensure that the service account you created still exists in your Google Cloud console and has neither been disabled nor deleted.

General connection failure

Following are the possible error messages for this issue:

Unable to connect to the configured BigQuery instance, please check your credentials and configs and then try again.

Cannot create poolable connection factory.

These indicate a general connection failure to your Google BigQuery instance, possibly due to misconfigured credentials or network issues. Walk the checklist in "Unable to connect to the configured BigQuery instance" above.
Verify that your credentials are correctly configured.
Ensure that there are no network issues blocking the connection.
If the problem still persists after verifying all of the above, contact Atlan support.

Does Atlan support nested columns beyond level 15?

Atlan supports nested columns up to level 15, because BigQuery's maximum supported nested (RECORD) depth is 15 levels. For more information, see the official BigQuery documentation on nested and repeated columns.

What are the known limitations of generating lineage for nested columns?

The following examples illustrate the known limitations of generating lineage for Google BigQuery nested columns in Atlan:

Lineage will not be generated if you use the nested column of a RECORD in the source table or view to create another nested column for a RECORD REPEATED column in the target table or view. For example:
```
CREATE VIEW orders_view AS
SELECT
  ARRAY(SELECT AS STRUCT 
    customer.name AS customer_name,
    order_id AS order_id
  ) AS orders
FROM
  orders_raw;
```
- Lineage will be generated for: orders_raw.order_id → orders_view.orders.order_id
- Lineage will not be generated for: orders_raw.customer.name → orders_view.orders.customer_name
Lineage will not be generated for a nested column if you use the RECORD column as well as a level 1 nested column in the source table or view to create a target column in another table or view. For example:
```
CREATE VIEW orders_view AS
SELECT
  customer AS customer;
  customer.name AS customer_name,
  order_id AS order_id
FROM
  orders_raw;
```
- Lineage will be generated for:
  - orders_raw.customer → orders_view.customer
  - orders_raw.customer.address.city → orders_view.city
- Lineage will not be generated for: orders_raw.customer.name → orders_view.customer_name. Instead, lineage will be generated between orders_raw.customer → orders_view.customer_name.
Lineage will not be generated for a nested column if you use a table alias to refer to nested columns in a source table or view to create a target column in another table or view. For example:
```
CREATE VIEW orders_view AS
SELECT
  o.order_id AS order_id
  o.customer.name as customer_name
  o.customer.address.city as city
FROM
  orders_raw o;
```
- Lineage will be generated for: orders_raw.order_id → orders_view.order_id
- Lineage will not be generated for:
  - orders_raw.customer.name → orders_view.customer_name
  - orders_raw.customer.address.city → orders_view.city
Lineage will not be generated for a nested column if you use a source table or view CROSS JOIN with UNNEST to create a target table or view. For example:
```
CREATE VIEW orders_view AS
SELECT
  order_id,
  customer.name as customer_name,
  customer.address.city as city,
  i.item_name AS item_name
FROM
  orders_raw CROSS JOIN UNNEST(items) AS i;
```
- Lineage will be generated for: orders_raw.order_id → orders_view.order_id
- Lineage will not be generated for:
  - orders_raw.customer.name → orders_view.customer_name
  - orders_raw.customer.address.city → orders_view.city
  - orders_raw.items.item_name → orders_view.item_name

Quick index​

VPC-SC denials - organization's-policy 403​

"Unable to connect to the configured BigQuery instance"​

WIF failures - STS step 403 / 400​

Key rotation - scheduled runs break overnight​

Cross-project permission denials​

PSC questions​

Opening ticket that gets solved fast​

Other known issues​

How to debug test authentication and preflight check errors?​

Does Atlan support nested columns beyond level 15?​

What are the known limitations of generating lineage for nested columns?​