Skip to main content

Preflight and lineage errors

Connect docs via MCP
TL;DR

Troubleshoot preflight failures and missing lineage in the Google Dataflow connector.

Resolve preflight failures and missing lineage errors in the Google Dataflow connector.

Permission denied on preflight​

Error

Preflight failed: caller does not have permission

Cause​

The service account is missing the roles/datalineage.viewer role on one or more of the configured GCP projects.

Solution​

  1. Open IAM & Admin in the Google Cloud console for each project.
  2. Find the service account used by the connector.
  3. Confirm it has the Data Lineage Viewer (roles/datalineage.viewer) role.
  4. If missing, add the role and re-run the workflow.

Data Lineage API not enabled​

Error

Preflight failed: API datalineage.googleapis.com is not enabled

Cause​

The Data Lineage API isn't enabled on the project.

Solution​

Enable datalineage.googleapis.com on every project you want Atlan to read lineage from:

APIs & Services → Enable APIs and services → search for Data Lineage API → Enable.

Job appears without lineage edges​

Error

No lineage events found for job

Cause​

One of the following:

  • The job's source or sink isn't supported by the Data Lineage API (for example, Apache Iceberg writes).
  • Lineage records haven't propagated yet—Google Cloud can take a few minutes after a job run completes.

Solution​

  1. Check that the job's sources and sinks are in the supported list: Pub/Sub, Cloud Storage, Bigtable, Spanner, Apache Kafka, JDBC.
  2. If the sink is unsupported, lineage edges won't appear—this is a Google Cloud limitation.
  3. If the sink is supported, wait a few minutes and re-run the workflow.

No runs found​

Error

No Dataflow runs found in lookback window

Cause​

One of the following:

  • The job was launched without the enable_lineage=true service option.
  • The job ran before the connector's lookback window.

Solution​

  1. Confirm enable_lineage=true is set in the job launch command:

    --dataflowServiceOptions=enable_lineage=true      # Java
    --dataflow_service_options=enable_lineage=true    # Python

  2. For template-based jobs:

    gcloud dataflow jobs run <job> ... \
      --additional-experiments=enable_lineage=true

  3. Re-run the job with the flag enabled, then re-run the workflow.

See also​

Need help​

If you need assistance after trying these steps, contact Atlan support: Submit a request.