Skip to main content
Community Hub

How to crawl Microsoft Power BI in Atlan

TL;DR

Crawl Microsoft Power BI in Atlan to catalog reports, dashboards, datasets, and workspaces. Set up Microsoft Entra ID and the Power BI tenant settings before establishing the connection.

Your AI can read this via Docs MCPcurl -fsSL "https://docs.atlan.com/install-docs-mcp" | bashConnect

Once you have set up Microsoft Power BI, you can establish a connection between Atlan and Microsoft Power BI.

To crawl metadata from Microsoft Power BI, review the order of operations and then complete the following steps.

Prerequisites​

Before you begin, make sure you have:

Create crawler workflow​

To create a workflow choose Microsoft Power BI as your source:

  1. In your Atlan workspace, click Connectors in the left sidebar.

    • If you are using the Old UI (Classic), click New Workflow in the top navigation.

  2. Click Marketplace.

  3. Search for Power BI Assets and select it.

  4. Click Install.

  5. Once installation completes, click Setup Workflow on the same tile.

Provide credentials​

To enter your Microsoft Power BI credentials:

  1. For Authentication, choose the method you want to use to access Microsoft Power BI:

    • For Service Principal authentication, enter the Tenant Id, Client Id, and Client Secret you configured when setting up Microsoft Entra ID. Use the Enable Scanner API Access toggle to control how metadata is extracted. When enabled, the crawler uses only scanner APIs. If disabled, both scanner and non-scanner APIs are used. For limitations when you use only scanner APIs, see Capabilities & limitations.
    • For Delegated User authentication, enter the Username and Password of a Power BI user, plus the same Tenant ID, Client ID, and Client Secret. This method needs different setup; see Delegated user authentication. Use Service Principal unless you can't.

  2. At the bottom of the form, click the Test Authentication button to confirm connectivity to Microsoft Power BI using these details. See Authentication for what this validates and why the secret field appears empty after saving.

  3. Once successful, at the bottom of the screen click the Next button.

Configure connection​

To complete the Microsoft Power BI connection configuration:

  1. Provide a Connection Name that represents your source environment. For example, you might want to use values like production, development, gold, or analytics.

  2. (Optional) To change the users able to manage this connection, change the users or groups listed under Connection Admins. If you don't specify any user or group, nobody can manage the connection — not even admins.

  3. At the bottom of the screen, click the Next button to proceed.

Configure crawler​

Before running the Microsoft Power BI crawler, configure metadata extraction and advanced options. You can override the default settings for the following fields. If a workspace appears in both the include and exclude filters, the exclude filter takes precedence.

  • Include Workspaces: Select Microsoft Power BI workspaces to include. Defaults to all workspaces when left blank. Use Advanced Search to filter workspaces using the following options:

    • Contains: Matches workspaces that contain the given substring.
    • Starts with: Matches workspaces that begin with the specified text.
    • Ends with: Matches workspaces that end with the specified text.
    • Regex pattern: Matches workspaces based on a regular expression.
    Include Workspaces is an allowlist

    Setting Include Workspaces to a few entries means only those are crawled. On a connection that previously crawled everything, this makes the next run look like a mass deletion (assets from all other workspaces "gone"), which trips the 80% deletion circuit breaker. Narrow scope deliberately, and expect the breaker — see Operating Microsoft Power BI.

    All selected filters apply using an AND condition.

  • Exclude Workspaces: Select workspaces to exclude. No workspaces are excluded by default. Advanced Search is also available for exclusion, with the same filtering options as mentioned previously.

  • Exclude Workspaces Regex: Use a regular expression to exclude workspaces by name. Excludes none by default. The expression is case-sensitive. To match text anywhere in the name, wrap it in .*. For example, .*Test.* excludes OLS_Testing and Dev-Test. A workspace that matches is excluded even if you select it in Include Workspaces.

    Exclude workspaces that are re-created

    Advanced Search selects from the workspaces that exist when you save the crawler. Use Exclude Workspaces Regex for dev or test workspaces that are deleted and re-created. A re-created workspace gets a new ID, so a workspace you selected before isn't excluded again. The regex matches every new workspace with the same name pattern on each run.

  • Include Dashboard and Reports Regex: Use a regular expression to include dashboards and reports based on naming patterns. Includes all by default.

  • Exclude Dashboard and Reports Regex: Use a regular expression to exclude dashboards and reports based on naming patterns. Excludes none by default.

  • Attach Endorsements from Power BI: Automatically certify assets endorsed in Power BI. To manually review before applying, change this setting to Send a Request. For more details, see What does Atlan crawl from Microsoft Power BI?

  • Fetch Report Definition Extracts: On by default. Reads each report's definition through the Fabric API to build column and measure to page lineage. Turn it off to skip those calls; the rest of the crawl is unaffected. See What lineage does Atlan extract from Microsoft Power BI?

  • Fetch Extracts for Context Agents: Off by default. Downloads the complete report (PBIR) and semantic model (TMDL) definitions that Atlan's Context Agents Studio and Talk-to-Data agents use, and keeps the latest copy for this connection in your tenant's object storage. Each run replaces the previous copy. This adds Fabric API calls for every report and semantic model, so runs take longer. It creates no new assets in the catalog.

    Fabric prerequisites and content of the extracts

    Both toggles read from the Fabric API and need Enable Scanner API Access off, the Service principals can call Fabric public APIs tenant setting, and Contributor or higher on the workspaces. See Power BI's two permission layers. If every report and semantic model is refused, the run fails with PERMISSION_DENIED_DEFINITION_API rather than keeping a stale copy.

    Report and semantic model definitions are stored as Microsoft returns them. They can include data source connection details, Power Query and DAX expressions, and literal values embedded in the model. Enable Fetch Extracts for Context Agents only for connections where storing that content in your Atlan tenant is acceptable.

Configure advanced settings​

  • Source Connections: When your tenant has multiple connections available for the same source system that share the similar metadata, confirm the advanced options and choose the correct connections from the Source Connections list drop down to avoid creating duplicate lineage to such connections.
  • Enable ODBC DSN Connectivity Mapping: Power BI provides multiple ways of connecting to a SQL source, including ODBC connectivity for building Reports and Dashboards. When datasets are populated using ODBC, provide a mapping of the DSN ( Data Source Name ) names to their appropriate database qualified names after enabling this toggle.
  • Enable Power BI OLS Dataset Enrichment: The Power BI Admin APIs don't return table metadata for semantic models protected by Object Level Security (OLS), so those tables and columns are missing in Atlan and their upstream lineage is broken. Enable this toggle and provide the BIM File Bucket, BIM File Bucket Region, BIM File Prefix, and BIM Bucket Role ARN so that the crawler reads each semantic model's BIM file from Amazon S3 and creates the missing assets and lineage. You need to export the BIM files before the crawl. See Enrich Power BI OLS datasets.

Run crawler​

To run the Microsoft Power BI crawler:

  1. To check for any permissions or other configuration issues before running the crawler, click Preflight checks. Get all checks green before the first crawl.

  2. You can either:

    • To run the crawler once immediately, at the bottom of the screen, click the Run button.
    • To schedule the crawler to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the Schedule Run button.

Once the crawler has completed running, you can see the assets in Atlan's asset page! 🎉

Before you rely on a schedule:

  • Order it after the crawlers for your source databases, and before the Power BI miner, so lineage resolves against current source assets.
  • Configure failure alerting (email, Teams, or Slack webhook) and send a test alert.

Delegated user authentication​

Delegated user authentication signs in as a named Power BI user with that user's password. Microsoft recommends against this sign-in flow: it doesn't work for accounts that require multifactor authentication, federated accounts, or guest accounts invited from another tenant. Use it only when a service principal isn't an option.

It needs a different setup from the service principal path:

  1. Fabric Administrator role: a Microsoft 365 administrator assigns the user the Fabric Administrator role (Microsoft 365 admin center → Users → Active users → the user → Manage roles).
  2. Delegated API permissions: in the app registration, go to API permissions → Add a permission → Power BI Service → Delegated permissions, add Capacity.Read.All, Dataflow.Read.All, Dataset.Read.All, Report.Read.All, Tenant.Read.All, and Workspace.Read.All, and then select Grant admin consent. These permissions are for delegated user authentication only; don't add them for a service principal.
  3. Tenant settings: enable Enhance admin APIs responses with detailed metadata and Enhance admin APIs responses with DAX and mashup expressions, scoped to a security group that contains the user. See Configure Power BI tenant settings.

See also​