Skip to main content

Snowflake connectivity and crawling

This FAQ answers common questions about Snowflake connectivity and crawling.

Why are some assets from databases or schemas missing?​

  1. Check the grants on the role attached to the user defined for the crawler. Confirm that the missing database or schema is present in these grants.

    SHOW GRANTS TO ROLE atlan_user_role;

Why are new tables or views missing?​

  1. When using incremental extraction, consider running a one-time full extraction to capture any newly introduced metadata.

  2. Make sure the role attached to the user defined for the crawler has grants for future tables and views being created in the database:

    GRANT USAGE ON FUTURE SCHEMAS IN DATABASE "<database-name>" TO ROLE atlan_user_role;
    GRANT REFERENCES ON FUTURE TABLES IN DATABASE "<database-name>" TO ROLE atlan_user_role;
    GRANT REFERENCES ON FUTURE VIEWS IN DATABASE "<database-name>" TO ROLE atlan_user_role;
    GRANT REFERENCES ON FUTURE EXTERNAL TABLES IN DATABASE "<database-name>" TO ROLE atlan_user_role;

  3. Make sure you run the below commands as well so that new tables and views you've created in-between are also visible to the user:

    GRANT USAGE ON ALL SCHEMAS IN DATABASE "<database-name>" TO role atlan_user_role;
    GRANT REFERENCES ON ALL TABLES IN DATABASE "<database-name>" TO role atlan_user_role;
    GRANT REFERENCES ON ALL EXTERNAL TABLES IN DATABASE "<database-name>" TO atlan_user_role;
    GRANT REFERENCES ON ALL VIEWS IN DATABASE "<database-name>" TO role atlan_user_role;

Why is some lineage missing?​

  • The query miner mines query history for up to the previous two weeks. It doesn't mine queries older than that. If the queries that created your assets ran earlier, lineage for those assets isn't available.
  • Lineage is unsupported for parameterized queries. Snowflake currently does not resolve values for parameterized queries before logging them in query history. This limits Atlan from generating lineage in such cases.
  1. To mine more than two weeks of query history, use S3-based query mining or contact Atlan support. Snowflake retains query history for a limited time. If Snowflake no longer retains the query history, Atlan can't mine it for lineage.

Missing attributes and lineage​

  • When using the account usage extraction method, there are currently some limitations. Atlan works with Snowflake to find workarounds for crawling the following:
    • External table location data
    • Procedures
    • Primary key designation
  • Furthermore, only database-level filtering is currently possible.

What views does Atlan require access to for account usage method?​

When using the account usage method for fetching metadata, Atlan requires access to the following views in Snowflake:

  • For the crawler: DATABASES, SCHEMATA, TABLES, VIEWS, COLUMNS, FUNCTIONS, and PIPES
  • For the miner and popularity metrics: QUERY_HISTORY, ACCESS_HISTORY, and SESSIONS

Why do I get destination URL mismatch error when authenticating via Okta SSO?​

This error can occur when you're connecting to Snowflake through Okta SSO and enter the URL of your Snowflake instance in a format different from the one used in Okta.

Snowflake follows two URL formats:

  • Legacy format -  <AccountLocator>.<Region>.snowflakecomputing.com or <AccountLocator>.<Region>.<cloud>.snowflakecomputing.com
  • New URL format - <Orgname>-<AccountName>.snowflakecomputing.com

Use the same Snowflake URL format in Snowflake and Okta. Refer to Snowflake documentation to learn more.

You may see errors like java.net.UnknownHostException or "Name or service not known" when connecting to Snowflake over private link if all of the following are true:

  • You are using Snowflake JDBC driver version 3.13.25 or newer
  • Your Snowflake account name contains an underscore (_)
  • You are connecting via private link (for example, https://my_account.us-west-2.privatelink.snowflakecomputing.com)

The newer Snowflake JDBC driver automatically replaces underscores (_) with hyphens (-) in the hostname. For normal (non-private link) connections, this works because Snowflake accepts both formats. For private link connections, the DNS endpoint only recognizes the original hostname. Because of this mismatch, the hostname can't be resolved and the connection fails.

  1. Update your Snowflake hostname to use hyphens (-) instead of underscores (_).
  2. Retry the connection.

For more details, refer to the Snowflake documentation.