Skip to main content

Set up Google Cloud Knowledge Catalog

Configure Google Cloud Knowledge Catalog authentication using either a service account key or Workload Identity Federation (WIF). Atlan uses the configured credentials to extract metadata from Knowledge Catalog.

Prerequisites​

Before you begin, make sure you have:

  • Google Cloud administrator access to create roles and service accounts
  • Access to the Google Cloud project where Knowledge Catalog is configured
  • Either Google Cloud console access or Google Cloud CLI installed and initialized

Authentication options​

The Knowledge Catalog connector supports two authentication methods:

  • Service account key: A JSON key file downloaded from GCP. Suitable for most deployments.
  • Workload Identity Federation (WIF): Keyless authentication using GCP's WIF. Suitable for environments where managing long-lived service account keys isn't desirable.

Both methods require a service account with the appropriate IAM permissions. Create the service account first, then configure the authentication method of your choice.

Permissions​

Atlan requires the following permissions to connect to Google Cloud Knowledge Catalog. Create a custom role with the relevant permissions and assign it to the Atlan service account.

Metadata crawling​

Add these permissions for the baseline metadata crawl. They're required for all connector configurations.

  • dataplex.entryTypes.list enables Atlan to list entry types.
  • dataplex.entryTypes.get enables Atlan to retrieve entry type metadata.
  • dataplex.entryGroups.list enables Atlan to list entry groups.
  • dataplex.entryGroups.get enables Atlan to retrieve entry group metadata.
  • dataplex.entries.list enables Atlan to list entries.
  • dataplex.entries.get enables Atlan to retrieve entry metadata.
  • dataplex.aspectTypes.list enables Atlan to list Aspect Types.
  • dataplex.aspectTypes.get enables Atlan to retrieve Aspect Type metadata.
  • dataplex.locations.list enables Atlan to dynamically discover all available Knowledge Catalog locations. Without this permission, the connector uses a built-in fallback list of locations.
  • resourcemanager.projects.get enables Atlan to list all GCP projects accessible to the service account via the Cloud Resource Manager API for the Include/Exclude filter.

Data quality and data profiling​

Add these permissions if you enable Ingest Data Quality or Ingest Data Profiling in the connector configuration.

  • dataplex.datascans.list enables Atlan to list data scans in the project.
  • dataplex.datascans.get enables Atlan to retrieve data scan metadata and configuration.
  • dataplex.datascans.getData enables Atlan to retrieve data scan results.

Aspects reverse-sync​

Add these permissions if you enable Enable Aspects Reverse Sync. They enable the service account to write Aspect value changes made in Atlan back to Knowledge Catalog entries.

  • dataplex.entries.update enables writing updated Aspect values back to Knowledge Catalog entries.
  • dataplex.aspectTypes.use enables attaching or modifying Aspects of a specific Aspect Type on an entry. GCP enforces this in addition to dataplex.entries.update; without it, Aspect writes are rejected even if the entry itself is writable.

Create service account​

Create a service account with the required Knowledge Catalog permissions. You can use either the Google Cloud console or Google Cloud CLI.

Create custom role​

Create a custom role and assign the permissions relevant to your configuration (see Permissions).

  1. Open the Google Cloud console.
  2. From the left menu under IAM and admin, click Roles.
  3. Using the dropdown list at the top of the page, select the project where you want to create the role.
  4. From the upper left of the Roles page, click Create Role.
  5. In the Create role page, enter the following details:
    1. For Title, enter a meaningful name for the custom role - for example, Atlan Knowledge Catalog Role.
    2. (Optional) For Description, enter a description for the custom role.
    3. For ID, the Google Cloud console generates a custom role ID based on the custom role name. Edit the ID if necessary - the ID can't be changed later.
    4. (Optional) For Role launch stage, assign a stage for the custom role - for example, Alpha or General Availability.
    5. Click Add permissions and add the permissions from the Permissions section that apply to your configuration.
    6. Click Create to finish custom role setup.

Create service account​

  1. Open the Google Cloud console.
  2. From the left menu under IAM and admin, click Service accounts.
  3. Select a Google Cloud project.
  4. From the upper left of the Service accounts page, click Create Service Account.
  5. For Service account details, enter the following details:
    1. For Service account name, enter a service account name to display in the Google Cloud console.
    2. For Service account ID, the Google Cloud console generates a service account ID based on this name. Edit the ID if necessary - the ID can't be changed later.
    3. (Optional) For Service account description, enter a description for the service account.
    4. Click Create and continue to proceed to the next step.
  6. For Grant this service account access to the project, enter the following details:
    1. Click the Select a role dropdown and then select the custom role you created in the previous step - for example, Atlan Knowledge Catalog Role.
    2. Click Continue to proceed to the next step.
  7. Click Done to finish the service account setup.

Create service account key​

  1. Open the Google Cloud console.
  2. From the left menu under IAM and admin, click Service accounts.
  3. Select the Google Cloud project for which you created the service account.
  4. On the Service accounts page, click the email address of the service account that you want to create a key for.
  5. From the upper left of your service account page, click the Keys tab.
  6. On the Keys page, click the Add Key dropdown and then click Create new key.
  7. In the Create private key dialog, for Key type, click JSON and then click Create. This creates a service account key file. Download the key file and store it in a secure location—you won't be able to download it again.

Set up workload identity federation (WIF)​

Use this option to avoid managing long-lived service account keys. The service account created below is still required—WIF is an alternative way to authenticate as that service account.

  1. Create an OAuth client in Atlan and securely store:
    • OAuth Client ID
    • OAuth Client Secret
  2. In Google Cloud, create a Workload Identity Pool and OIDC provider that trusts your Atlan tenant issuer.
  3. Configure attribute mapping for the audience and add your Atlan OAuth Client ID as the audience.
  4. Grant roles/iam.workloadIdentityUser on the service account created below to the workload identity principal set.
  5. Copy the WIF Pool Provider ID in this format:
    • //iam.googleapis.com/projects/<project-number>/locations/global/workloadIdentityPools/<pool-id>/providers/<provider-id>
  6. Use these values when configuring the connector:
    • Project ID
    • Service Account Email
    • WIF Pool Provider ID
    • Atlan OAuth Client ID
    • Atlan OAuth Client Secret

For the detailed WIF setup flow, refer to Set up Workload Identity Federation for Google BigQuery. The same Atlan OAuth and Google WIF concepts apply.

Next steps​