Google Dataflow Private Preview
TL;DR
Integrate, catalog, and visualize Google Dataflow job lineage in Atlan.
Discovery
Lineage
Browser Extension
Self-deployed Runtime
Enrichment
Popularity
Run status
Overview: The Google Dataflow connector reads job-run lineage that Dataflow publishes to Google Cloud's Data Lineage API and creates the jobs, datasets, and lineage in Atlan. Gain visibility into your batch and streaming pipelines and their upstream and downstream datasets.
Get started
References
- What does Atlan crawl from Google Dataflow?: Detailed list of Google Dataflow metadata that Atlan can extract and visualize.
Troubleshooting
- Preflight and lineage errors: Resolve permission errors, missing lineage edges, and jobs with no runs.