Skip to main content

Manage popularity

Popularity gives additional context to the usage of assets. From popularity, you can see various information such as:

  • users who have queried an asset, including:
    • those who have most recently queried the asset
    • those who have most frequently queried the asset
  • details about queries against the asset, including:
    • the most frequently run queries
    • the slowest-running queries
    • the most expensive queries
Popularity is only populated on certain assets by Atlan

Out-of-the-box, only specific crawlers currently populate popularity information on assets. However, the attributes involved are actually available behind-the-scenes on all assets.

Make details visible in UI​

Usage details not visible by default

By default, only the crawlers that support usage details out-of-the-box will show the usage details in the Atlan UI. To make usage details visible for other assets you must enable this at a connection level.

During connection creation​

To create a popularity-enabled connection:

Create a popularity-enabled connection
String adminRoleGuid = client.getRoleCache().getIdForName("$admin"); // (1)
Connection connection = Connection.creator( // (2)
        "development", // (3)
        AtlanConnectorType.HIVE, // (4)
        List.of(adminRoleGuid), // (5)
        List.of("group2"), // (6)
        List.of("jsmith")) // (7)
    .hasPopularityInsights(true) // (8)
    .build();
AssetMutationResponse response = connection.save(client); // (9)

  1. Retrieve the GUID for the admin role, to use later for defining the roles that can administer the connection.

  2. Build up the minimum request to create a connection.

  3. Provide a human-readable name for your connection.

  4. Set the type of connection.

    Determines the icon

This determines the icon that Atlan will use for all the assets in the connection. ::: 5. List the workspace roles that should be able to administer the connection (or null if none). All users with that workspace role (current and future) will be administrators of the connection. Note that the values here need to be the GUIDs of the workspace roles. At least one of adminRoles, adminGroups, or adminUsers must be provided. 6. List the group names that can administer this connection (or null if none). All users within that group (current and future) will be administrators of the connection. Note that the values here are the names of the groups. At least one of adminRoles, adminGroups, or adminUsers must be provided. 7. List the user names that can administer this connection (or null if none). Note that the values here are the usernames of the users. At least one of adminRoles, adminGroups, or adminUsers must be provided. 8. Set the hasPopularityInsights property to true—this is the key piece to ensuring usage details will be visible in the UI for assets in this connection. 9. Actually call Atlan to create the connection. Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant.

Update existing connection​

To update an existing connection:

Update connection to make popularity visible
Connection connection = Connection.get("default/hive/1657025257"); // (1)
AssetMutationResponse updated = connection.trimToRequired() // (2)
    .hasPopularityInsights(true) // (3)
    .build() // (4)
    .save(client); // (5)
  1. Build an object referencing the existing connection. In this example we will retrieve it first, but you could also search for it or use the updater() method to construct an update.
  2. If starting from an existing object, remember to trimToRequired() to get a builder with the minimal required information for an update.
  3. Set the hasPopularityInsights property to true.
  4. Build the object, so it's ready to be persisted.
  5. Persist the object by saving it. Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant.

Retrieve usage details​

Since popularity details are only available on certain assets, you will need to retrieve one of those assets to see the usage details:

Retrieve popularity
Table table = Table.get(client,  // (1)
        "default/hive/1657025257/OPS/DEFAULT/RUN_STATS"); // (2)
table.getPopularityScore(); // (3)
table.getSourceReadCount();
table.getSourceReadQueryCost();
table.getSourceReadUserCount();
table.getSourceTotalCost();
table.getSourceQueryComputeCosts(); // (4)
table.getSourceReadRecentUsers();
table.getSourceReadTopUsers();
table.getSourceQueryComputeCostRecords(); // (5)
table.getSourceReadRecentUserRecords();
table.getSourceReadTopUserRecords();
table.getSourceReadExpensiveQueryRecords(); // (6)
table.getSourceReadPopularQueryRecords();
table.getSourceReadSlowQueryRecords();

  1. Use the get() method to retrieve all details about a specific asset. Because this operation will retrieve the asset from Atlan, you must provide it an AtlanClient through which to connect to the tenant.

  2. Provide the full, case-sensitive qualifiedName of the asset.

  3. You can retrieve single quantified metrics for various aspects, such as:

    • An overall popularity score for the asset (higher is better), using getPopularityScore().
    • The total number of all read operations on the asset, using getSourceReadCount().
    • The total cost of all read operations on the asset, using getSourceReadQueryCost().
    • The total number of unique users that read data from the asset, using getSourceReadUserCount().
    • The total cost of all operations on the asset, using getSourceTotalCost().

  4. You can retrieve lists of various aspects, such as:

    • The top query running engines (warehouses), using getSourceQueryComputeCosts().
    • The users who most recently queried the asset, using getSourceReadRecentUsers().
    • The users who most frequently query the asset, using getSourceReadTopUsers().

  5. You can also list more detailed information about individual metrics, such as:

    • The top query running engines (warehouses) and their related cost, using getSourceQueryComputeCostRecords().
    • The users who most recently queried the asset, the number of times they've queried it and when they last queried it, using getSourceReadRecentUserRecords().
    • The users who most frequently query the asset, the number of times they've queried it and when they last queried it, using getSourceReadTopUserRecords().

  6. You can also list details about specific queries, such as:

    • The list of most expensive queries against this asset, using getSourceReadExpensiveQueryRecords().
    • The list of the most frequently run queries against this asset, using getSourceReadPopularQueryRecords().
    • The list of the slowest-running queries against this asset, using getSourceReadSlowQueryRecords().

Add your own usage details​

In cases where Atlan doesn't popularity details from the source, you may want to add your own. You can do this by either adding the usage details when creating the asset (programmatically) or by updating the attributes on an existing asset:

Add or update usage details
Table table = Table.updater( // (1)
        "default/hive/1657025257/OPS/DEFAULT/RUN_STATS",
        "RUN_STATS")
    .popularityScore(1.2345) // (2)
	.sourceReadCount(123L)
    .sourceReadQueryCost(5.4321)
    .sourceReadUserCount(5L)
    .sourceTotalCost(5.4321)
    .sourceQueryComputeCosts(List.of("A", "B")) // (3)
    .sourceReadRecentUsers(List.of("c", "d", "e"))
    .sourceReadRecentUser("f")
    .sourceReadRecentUser("g")
    .sourceReadTopUsers(List.of("g", "f", "e"))
    .sourceReadTopUser("d")
    .sourceReadTopUser("c")
    .sourceQueryComputeCostRecord( // (4)
        PopularityInsights.builder()
            .recordComputeCost(4.4321)
            .recordComputeCostUnit(SourceCostUnitType.CREDITS)
            .recordWarehouse("A")
            .build())
    .sourceQueryComputeCostRecord(
        PopularityInsights.builder()
            .recordComputeCost(1.0)
            .recordComputeCostUnit(SourceCostUnitType.CREDITS)
            .recordWarehouse("B")
            .build())
    .sourceReadRecentUserRecord(
        PopularityInsights.builder()
            .recordUser("c")
            .recordLastTimestamp(1234567899000L)
            .build())
    .sourceReadRecentUserRecord(
        PopularityInsights.builder()
            .recordUser("d")
            .recordLastTimestamp(1234567898000L)
            .build())
    .sourceReadRecentUserRecord(
        PopularityInsights.builder()
            .recordUser("e")
            .recordLastTimestamp(1234567897000L)
            .recordQueryCount(3L)
            .build())
    .sourceReadTopUserRecord(
        PopularityInsights.builder()
            .recordUser("g")
            .recordQueryCount(100L)
            .recordLastTimestamp(1234567895000L)
            .build())
    .sourceReadTopUserRecord(
        PopularityInsights.builder()
            .recordUser("f")
            .recordQueryCount(20L)
            .recordLastTimestamp(1234567896000L)
            .build())
    .sourceReadTopUserRecord(
        PopularityInsights.builder()
            .recordUser("e")
            .recordQueryCount(3L)
            .recordLastTimestamp(1234567897000L)
            .build())
    .sourceReadExpensiveQueryRecord(
        PopularityInsights.builder()
            .recordComputeCost(5.4321)
            .recordMaxComputeCost(5.4321)
            .recordComputeCostUnit(SourceCostUnitType.CREDITS)
            .recordQuery("SELECT * from RUN_STATS")
            .recordQueryCount(123L)
            .recordTotalUserCount(5L)
            .build())
    .sourceReadPopularQueryRecord(
        PopularityInsights.builder()
            .recordComputeCost(5.4321)
            .recordComputeCostUnit(SourceCostUnitType.CREDITS)
            .recordQueryCount(123L)
            .recordQuery("SELECT * from RUN_STATS")
            .recordTotalUserCount(5L)
            .build())
    .sourceReadSlowQueryRecord(
        PopularityInsights.builder()
            .recordUser("g")
            .recordLastTimestamp(124567895000L)
            .recordQueryDuration(321L)
            .recordQuery("SELECT * from RUN_STATS")
            .build())
    .build();
AssetMutationResponse response = table.save(client); // (5)
  1. Use the updater() method to update an existing asset, providing the required details for that particular asset type (for more details, see Updating an asset).
  2. For single quantified metrics, you can directly set the metric.
  3. For lists, you can directly set the lists or individually add elements (or a combination of the two).
  4. For the more detailed records, you need to build the PopularityInsights object with its embedded details. As with the lists you can associate these detailed records with the asset many-at-a-time or one-by-one.
  5. Finally, you must save the object you've built up to persist this information in Atlan. Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant.
Was this page helpful?