Skip to main content

Set up Microsoft Azure Data Factory

Atlan supports service principal authentication for fetching metadata from Microsoft Azure Data Factory. This method requires a client ID, client secret, and tenant ID to fetch metadata.

Register app with Microsoft Entra ID​

Who can do this?

To complete these steps, you need your Cloud Application Administrator or Application Administrator - you may not have access yourself. This is required if the creation of registered applications isn't enabled for the entire organization.

To complete this, you need to register your service principal application with Microsoft Entra ID and note down the values of the tenant ID, client ID, and client secret.

To register your app with Microsoft Entra ID:

  1. Log in to the Azure portal.
  2. In the search bar, search for Microsoft Entra ID, and select it from the dropdown list.
  3. From the left menu of the Microsoft Entra ID page, click App registrations.
  4. From the toolbar on the App registrations page, click + New registration.
  5. On the Register an application page, for Name, enter a name for your service principal application and then click Register.
  6. On the homepage of your newly created application, from the Overview screen, copy the values for the following fields and store them in a secure location:
    • Application (client) ID
    • Directory (tenant) ID
  7. From the left menu of your newly created application page, click Certificates & secrets.
  8. On the Certificates & secrets page, under Client secrets, click + New client secret.
  9. In the Add a client secret screen, enter the following details:
    1. For Description, enter a description for your client secret.
    2. For Expiry, select when the client secret expires.
    3. Click Add.
  10. On the Certificates & secrets page, under Client secrets, for the newly created client secret, click the clipboard icon to copy the Value and store it in a secure location.

Set permissions​

Who can do this?

To complete these steps, you need your Microsoft Azure Data Factory administrator - you may not have access yourself.

To proceed, add the service principal to the Reader role. This lets the service principal have read-only access to your Microsoft Azure Data Factory account.

To add the service principal to the Reader role:

  1. Log in to the Azure portal.
  2. Open the menu and search for or select Data factories.
  3. On the Data factories page, select the data factory you want to crawl in Atlan.
  4. From the left menu of your data factory page, click Access control (IAM).
  5. From the tabs along the top of the Access control (IAM) page, click Add and then click Add role assignment.
  6. On the Add role assignment page, configure the following:
    1. In the Roles tab, from the list of roles under Job function roles, select Reader - this gives read-only access to your data factory - and then click Next. Assign this role to all the data factories you want to crawl in Atlan.
    2. In the Members tab, enter the following details:
      1. For Assign access to, click User, group, or service principal.
      2. For Members, click + Select members and then select the service principal you created. Click Next to proceed to the next step.
    3. In the Review + assign tab, click Review + assign to add role assignment.

Atlan extracts metadata from all the data factories you specified in your Microsoft Azure Data Factory account with Reader access.