Skip to main content

Crawl SAP Datasphere

Configure and run the crawler to extract metadata from your SAP Datasphere instance and catalog it in Atlan. Once configured, the crawler uses OAuth to connect to your SAP Datasphere instance and discovers spaces, views, analytical models, and columns.

Prerequisites​

Before you begin, make sure you have:

  • Reviewed the order of operations for connecting data sources.
  • OAuth client ID, client secret, and refresh token for your SAP Datasphere instance.
  • Asset endpoint and token endpoint URLs for your SAP Datasphere tenant.
  • If your SAP Datasphere instance requires IP allowlisting, contact Atlan support to obtain Atlan IP addresses and add them before running the crawler.
info

Analytical models (when enabled) are cataloged as views in Atlan.

Create crawler workflow​

To create and configure the SAP Datasphere crawler workflow, complete the following steps.

  1. In the top right corner of any screen, navigate to New and then click New workflow.
  2. From the list of packages, select SAP Datasphere Assets, and click Set up workflow.

Configure credentials​

On the credentials page, configure OAuth authentication:

  1. For Client ID, enter the OAuth client ID used for authentication and REST API calls.

  2. For Client Secret, enter the OAuth client secret.

  3. For Refresh Token, enter the OAuth refresh token. The refresh token has a fixed validity period. Once it expires, you must obtain a new authorization code and re-run the workflow.

  4. For Asset Endpoint, enter the endpoint URL used to extract asset information from SAP Datasphere.

    Example: somename-ds-prod.us10.hcs.cloud.sap/api/v1/dwc/catalog/assets

  5. For Token Endpoint, enter the endpoint URL used to generate a new access token from the refresh token.

    Example: somename-ds-prod.us10.hana.ondemand.com/oauth/token

  6. For Database Name, enter the name to use in Atlan for the SAP Datasphere spaces being cataloged.

    Example: somename-ds-prod

  7. Click Test Authentication to confirm connectivity.

  8. Click Next after completing the configuration.

Configure connection​

To complete the SAP Datasphere connection configuration:

  1. For Connection Name, enter a unique name to identify this SAP Datasphere connection in Atlan (for example, sap-datasphere-prod). The name must be unique across all SAP Datasphere connections in Atlan.
  2. To change the users able to manage this connection, change the users or groups listed under Connection Admins. If you don't specify any user or group, nobody can manage the connection, not even admins.
  3. Click Next to proceed.

Configure crawler​

Configure which SAP Datasphere assets to include in the crawl:

  • Analytical Models: Choose No (default) to crawl only spaces and views, or Yes to include analytical models. When Yes is selected, analytical models are cataloged as views in Atlan.

Run crawler​

Choose how and when to run the crawler to extract metadata from your SAP Datasphere instance.

  1. You can either:
    • To run the crawler once immediately, click Run
    • To schedule the crawler to run hourly, daily, weekly, or monthly, click Schedule & Run and select your preferred frequency

Once the crawler completes, you can view the discovered SAP Datasphere assets in Atlan's asset search.

See also​