Skip to main content

Set up Inventory reports

Set up inventory reports for Amazon S3 to enable inventory-based ingestion through the crawler. This guide shows you how to configure inventory reports in the format required by Atlan's S3 crawler.

Prerequisites​

Before you begin, make sure you have:

  • AWS permissions: Access to configure inventory reports on source buckets. Follow the official AWS documentation on inventory report configuration for permissions.
  • Destination bucket: A dedicated S3 bucket to store inventory reports.

Create destination bucket​

First, create a dedicated S3 bucket to store your inventory reports.

  1. Sign in to the AWS Management Console.
  2. Navigate to S3 → Buckets.
  3. Click Create bucket.
  4. Enter a unique bucket name (for example, atlan-inventory-reports). Make a note of the bucket name as it’s required when configuring the Atlan workflow for inventory-based ingestion.
  5. Select the appropriate region (keep this consistent for all inventory reports). Make a note of the region as it’s required when configuring the Atlan workflow for inventory-based ingestion.
  6. Configure other settings as needed.
  7. Click Create bucket.

Configure inventory reports​

Now configure inventory reports for each S3 bucket you want to catalog in Atlan.

  1. Navigate to S3 → Buckets.

  2. Select the source bucket you want to catalog.

  3. Go to the Management tab.

  4. Scroll down to Inventory configurations

  5. Click Create inventory configuration and configure the following settings:

    • Inventory configuration name: Enter a meaningful name, such as atlan-inventory-config
    • Inventory scope: Optionally choose a prefix to limit the report to specific objects. You can also use filters in your workflow.
    • Object versions: Select Current version only (Atlan doesn't support Include all versions).

  6. Configure the Report details:

    • Destination bucket: Select the destination bucket you created earlier. Optionally specify a prefix to organize reports in a folder. Note: If you use a prefix, remember it for your Atlan workflow configuration and keep it consistent across all bucket reports.
    • Report frequency: Choose daily or weekly.
    • Report format: Select CSV or Apache Parquet (only these formats are supported).
    • Status: Enable the inventory report by selecting Enabled.

  7. Encryption: Leave encryption disabled. Atlan's S3 crawler requires unencrypted inventory reports.

  8. Metadata fields: Select all available metadata fields. This ensures Atlan receives complete metadata information about your S3 objects.

  9. Review all settings and click Create.

  10. For multiple inventory reports, your destination bucket must follow a specific structure. See Inventory Report Structure for details.

Need help?​

If you run into issues while setting up inventory reports:

Next steps​

Once you've configured your inventory reports: