Skip to main content

S3 Manage AWS S3 assets

Operations on S3 assets (connections, buckets, objects).

In general, these should be:

  • Created in top-down order (connection, then bucket, then object)
  • Deleted in bottom-up order (objects, then buckets, then connections)1

Asset structure​

Connection​

An AWS S3 connection requires a name and qualifiedName. For creation, specific settings are also required to distinguish it as an AWS S3 connection rather than another type of connection. In addition, at least one of adminRoles, adminGroups, or adminUsers must be provided.

Create an S3 connection
String adminRoleGuid = client.getRoleCache().getIdForName("$admin"); // (1)
Connection connection = Connection.creator( // (2)
        "aws-s3-connection", // (3)
        AtlanConnectorType.S3, // (4)
        List.of(adminRoleGuid), // (5)
        List.of("group2"), // (6)
        List.of("jsmith")) // (7)
    .build();
AssetMutationResponse response = connection.save(client); // (8)
String connectionQualifiedName = response.getCreatedAssets().get(0).getQualifiedName(); // (9)
  1. Retrieve the GUID for the admin role, to use later for defining the roles that can administer the connection.
  2. Build up the minimum request to create a connection.
  3. Provide a human-readable name for your connection, such as production or development.
  4. Set the type of connection to S3.
  5. List the workspace roles that should be able to administer the connection (or null if none). All users with that workspace role (current and future) will be administrators of the connection. Note that the values here need to be the GUIDs of the workspace roles. At least one of adminRoles, adminGroups, or adminUsers must be provided.
  6. List the group names that can administer this connection (or null if none). All users within that group (current and future) will be administrators of the connection. Note that the values here are the names of the groups. At least one of adminRoles, adminGroups, or adminUsers must be provided.
  7. List the user names that can administer this connection (or null if none). Note that the values here are the usernames of the users. At least one of adminRoles, adminGroups, or adminUsers must be provided.
  8. Actually call Atlan to create the connection. Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant.
  9. Retrieve the qualifiedName for use in subsequent creation calls. (You'd probably want to do some null checking first.)

S3Bucket​

An AWS S3 bucket requires both a name and a qualifiedName. During creation, you also need to specify the connectionQualifiedName of the connection associated with the bucket, and optionally provide a unique awsArn.

Create an S3 bucket
S3Bucket s3Bucket = S3Bucket.creator( // (1)
		"mybucket", // (2)
		connectionQualifiedName, // (3)
		"arn:aws:s3:::mybucket") // (4)
    .s3ObjectCount(10) // (5)
    .build();
AssetMutationResponse response = s3Bucket.save(client); // (6)
s3Bucket = response.getResult(s3Bucket); // (7)
  1. Build up the minimum request to create a bucket.
  2. Provide a human-readable name for your bucket.
  3. Provide the qualifiedName of the connection for this bucket.
  4. (Optional) If awsArn is provided, it will be used to construct the qualifiedName for the bucket; otherwise, the name of the bucket will be used.
  5. (Optional) To make sure the UI displays the correct count of S3Object's, set the s3ObjectCount directly on the S3Bucket instance.
  6. Actually call Atlan to create the bucket. Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant.
  7. Retrieve the created bucket for use in subsequent creation calls. (You'd probably want to do some null checking first.)

S3Object​

An AWS S3 object requires a name and a qualifiedName. For creation, you also need to specify the connectionQualifiedName of the connection for the object, and a unique awsArn or prefix. You should also specify the bucket the object is in, along with its s3BucketName and s3BucketQualifiedName.

By AWS ARN​

Create an S3 object using AWS ARN:

Create an S3 object using AWS ARN
S3Object s3Object = S3Object.creator( // (1)
		"myobject.csv", // (2)
		s3Bucket, // (3)
		"arn:aws:s3:::mybucket/prefix/myobject.csv") // (4)
    .build();
AssetMutationResponse response = s3Object.save(client); // (5)
  1. Build up the minimum request to create an object.
  2. Provide a human-readable name for your object.
  3. Provide the bucket in which this object should be created.
  4. Provide the unique ARN from AWS for this object.
  5. Actually call Atlan to create the object. Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant.

By prefix​

Create an S3 object using prefix:

Create an S3 object using prefix
S3Object s3Object = S3Object.creatorWithPrefix( // (1)
		"myobject.csv", // (2)
		s3Bucket, // (3)
		"/some/folder/structure") // (4)
    .build();
AssetMutationResponse response = s3Object.save(client); // (5)
  1. Build up the minimum request to create an object.
  2. Provide a human-readable name for your object.
  3. Provide the bucket in which this object should be created.
  4. Provide the folder path where the object is located within the bucket.
  5. Actually call Atlan to create the object. Because this operation will persist the asset in Atlan, you must provide it an AtlanClient through which to connect to the tenant.

Available relationships​

Every level of the object store structure is an Asset, and can therefore be related to the following other assets.

Footnotes​

  1. Although if you want to delete everything in a connection, your better avenue is the packaged connection delete utility in the UI. ↩

Was this page helpful?