Skip to main content

Searching for assets

Searching is a very flexible operation in Atlan. This also makes it a bit more complex to understand than the other operations. To encapsulate the full flexibility of Atlan's search, the SDK provides a dedicated IndexSearchRequest object and a FluentSearch class for configuring such a request using a fluent builder pattern.

More details on the power and flexibility of searching

See the dedicated Searching section of this site for more details on Atlan's search. This covers the various kinds of searches you can run, and the detailed attributes you can search against.

Build query​

To run a search in Atlan, you need to define the query using Elastic's structures. While you can always use Elastic's own structures to make use of its full power, for the vast majority of cases you may find it easier to use the helpers built-in to the SDK:

Build the query
FluentSearch.FluentSearchBuilder<?,?> builder = client.assets.select() // (1)
    .active() // (2)
    .where(Asset.TYPE_NAME.eq(GlossaryTerm.TYPE_NAME)); // (3)

  1. You can start building a query across all assets using the select() method on the assets member of any client. You can chain as many conditions as you want:

    • where() is mandatory inclusion
    • whereNot() is mandatory exclusion
    • whereSome() for conditions where some of them must match

  2. This helper provides a query that ensures results are active (not archived) assets.

    Equivalent Elastic query
    Query beActive = TermQuery.of(m -> m
          .field("__state")
          .value(AtlanStatus.ACTIVE.getValue()))
      ._toQuery();

  3. This condition provides a query that restricts results to a specific type of assets (glossary terms in this example).

    Equivalent Elastic query
    Query beTerm = TermQuery.of(m -> m
          .field("__typeName.keyword")
          .value(GlossaryTerm.TYPE_NAME))
      ._toQuery();

Build request​

Once the query is defined, we can then build up the search request. The request includes not only the query, but also parameters like paging and which attributes to include in the response:

Build the request
IndexSearchRequest index = builder // (1)
    .pageSize(100) // (2)
    .includeOnResults(GlossaryTerm.ANCHOR) // (3)
    .includeOnRelations(Asset.CERTIFICATE_STATUS) // (4)
    .toRequest(); // (5)
  1. You can then chain additional parameters onto the fluent search. (You could of course do this all directly as part of the same chain, you don't need to store the interim builder variable.)
  2. The number of results to include (per page).
  3. You can chain as many attributes as you want to include in each result. In this case we will return the anchor attribute for terms, which gives the relationship from the term to its parent glossary.
  4. You can chain as many attributes to include on each related asset to each result. Since we're returning anchor relationships, this will make sure that the certificateStatus of those related glossaries is also included in each result.
  5. You can now build all of this search configuration into a request.

(Optional) Build request directly from JSON​

While we recommend using FluentSearch to build search requests, there might be scenarios where quick prototyping or testing of search queries is needed. For such use cases, we also support directly constructing search requests from raw JSON.

Coming soon

To now run the search, we call the search() method against our request object:

Run the search
IndexSearchResponse response = index.search(client);
log.info(response.getApproximateCount()) // (1)
  1. The getApproximateCount() method gives the total number of results overall (not restricted by page).

Iterate through results​

One page of results​

To iterate through one page of results, loop through the list of assets:

Iterate through one page of results
List<Asset> results = response.getAssets(); // (1)
for (Asset result : results)
}
  1. The page of results itself can be accessed through the getAssets() method on the response.
  2. You can then iterate through these results from a single page.
  3. Remember that each result is a generic Asset. In our example we searched for a specific type, but another example may search for any asset with a given name (or Atlan tag)—so each result could be a different type. So again we should check and cast the results as-needed.

Multiple pages of results​

To iterate through multiple pages of results:

Iterate through multiple pages of results
for (Asset result : response)
  1. You can simply iterate over the reponse itself. This will lazily load and loop through each page of results until the loop finishes or you break out of it. (You could also use response.forEach(), which uses the same iteratable-based implementation behind-the-scenes.)
Iterate through multiple pages of results (streaming)
response.stream() // (1)
    .limit(100) // (2)
    .filter(a -> !(a instanceof ILineageProcess)) // (3)
    .forEach(a -> log.info("Do something with each result: {}", a)); // (4)

  1. Alternatively, you can also stream the results direct from the response. This will also lazily load and loop through each page of results.

    Can be chained without creating a request in-between

You can actually chain the stream() method directly onto the end of your query and request construction, without creating a request or response object in-between. ::: 2. With streaming, you can apply your own limits to the maximum number of results you want to process.

Independent of page size

Note that this is independent of page size. You could page through results 50 at a time, but only process a maximum of 100 total results this way. Since the results are lazily-loaded when streaming, only the first two pages of results would be retrieved in such a scenario.

  1. You can also apply your own logical filters to the results.

    Push-down as much as you can to the query

You should of course push-down as many of the filters as you can to the query itself, but if you have a particular complex check to apply that can't be encoded in the query this can be a useful secondary filter over the results. ::: 4. The forEach() on the resulting stream will then apply whatever actions you want with the results that come through.

Was this page helpful?