Skip to main content

Crawl Talend assets

Create a workflow in Atlan to discover and catalog Talend assets from project files stored in a GitHub or Atlassian Stash repository. Each workflow connects to one repository. To catalog assets from multiple repositories, create a separate workflow for each.

Prerequisites​

Before you begin, make sure you have:

Create crawler workflow​

  1. In Atlan, navigate to New > New Workflow.
  2. In the Marketplace tab, search for or select Talend Connector.
  3. Click Set up workflow.
  4. Select GitHub Cloud Store as the cloud store type. Both GitHub and Atlassian Stash (Bitbucket Server) are supported through this connection type.

Configure connection​

Enter the repository URL, access token, and connection settings for this workflow.

  1. Enter your repository details:
    • Repository URL: Enter the repository path ending with .git, for example github.com/organization/repository.git. The https:// prefix is added automatically.
    • Git access token: Enter your personal access token with read permissions.
    • Branch name: Enter the branch to crawl, for example main or master.
    • Talend project path: If your Talend projects are in a subdirectory, enter the path, for example talend_projects/. Leave this blank to crawl from the repository root.
  2. Enter a Connection name to identify this Talend connection in Atlan, for example production-talend.
  3. To change who manages this connection, update the users or groups listed under Connection admins.

Run crawler​

Choose to run the crawler once immediately or schedule it to run at a recurring interval.

  1. Click Run to run once, or Schedule & Run to run hourly, daily, weekly, or monthly.
  2. Monitor progress in the workflow execution details.

After the crawler completes, assets appear in Atlan. Navigate to Assets and filter by Talend to browse projects, jobs, components, and component fields.

See also​

What does Atlan crawl from Talend: Review the Talend asset types and metadata Atlan extracts

Need help​

If you encounter issues while crawling Talend assets, see Connection and authentication issues. For additional assistance, contact Atlan support by submitting a request.