Skip to main content

Create custom metadata

Like other objects in the SDK that you can create, custom metadata implements the builder pattern. This allows you to progressively build-up the structure you want to create.

There are limits to the number of custom metadata properties you can create

Atlan currently preserves details of custom metadata in its audit log. This allows Atlan to retain an audit trail of actions users took on custom metadata on each asset, even if the custom metadata definition itself is deleted.

However, this also places an upper limit on the number of custom metadata properties you can (structurally) define in Atlan. Even if you delete the custom metadata definitions, any that you have previously defined will still take up "space" within this limit.

More details

By default this is ~1000 properties. Note that this limit applies only to the structural definition of the properties themselves, not the values captured for assets. If you see an error like the following, it means you have reached this limit:

{
  "errorCode": "ATLAS-500-00-001",
  "errorMessage": "Unable to push entity audits to ES",
  "errorCause": "[{type=mapper_parsing_exception, reason=failed to parse, caused_by={type=illegal_argument_exception, reason=Limit of total fields [1000] has been exceeded while adding new fields [5]}}]"
}

You will need to contact Atlan support to extend this threshold if you reach it.

Build minimal object needed​

For example, to create a custom metadata structure to capture RACI assignments:

Build custom metadata definition for creation
CustomMetadataDef customMetadataDef = CustomMetadataDef.creator("RACI") // (1)
    .attributeDef( // (2)
        AttributeDef.of(client, "Responsible", // (3)
            AtlanCustomAttributePrimitiveType.USERS, // (4)
            null, // (5)
            false)) // (6)
    .attributeDef(AttributeDef.of(client, "Accountable", AtlanCustomAttributePrimitiveType.USERS, false))
    .attributeDef(AttributeDef.of(client, "Consulted", AtlanCustomAttributePrimitiveType.GROUPS, true))
    .attributeDef(AttributeDef.of(client, "Informed", AtlanCustomAttributePrimitiveType.GROUPS, true))
    .options(CustomMetadataOptions.withImage("https://example.com/logo.png", true)) // (7)
    .options(CustomMetadataOptions.withEmoji("👪")) // (8)
    .options(CustomMetadataOptions.withIcon(AtlanIcon.ROCKET_LAUNCH, AtlanTagColor.RED)) // (9)
    .build(); // (10)
  1. When creating the custom metadata structure, you must provide a name (RACI in this example).
  2. You can then add as many attributes to that structure as you want.
  3. Each attribute must have a name. Because this operation may need to retrieve information from Atlan, you must provide it an AtlanClient through which to connect to the tenant.
  4. Each attribute must have a type.
  5. If the type is AtlanCustomAttributePrimitiveType.OPTIONS then you must also specify the enumeration that defines the valid values for this attribute (in this example the type isn't an enumeration, so this is null and could even be left out as in the subsequent lines).
  6. You must also specify whether the attribute allows multiple values to be captured on it (true) or only a single value (false).
  7. You can also provide a custom logo for the custom metadata by providing a URL to an image. The second argument controls whether this custom metadata is editable via the UI—for false it's editable via the UI, while for true the custom metadata will only be editable via APIs (including via SDK).
  8. Or you can use an emoji as the custom icon for the custom metadata.
  9. Or you can use a built-in icon for the custom metadata. The second argument controls the color to use for the icon.
  10. As with all other builder patterns, you must build() the object you've defined.

Create custom metadata from object​

Now that the object is built, this customMetadataDef object will have the required information for Atlan to create it. You can then actually create the custom metadata definition in Atlan by calling the create() method on the object itself:

Create the custom metadata definition
CustomMetadataDef response = customMetadataDef.create(client); // (1)
  1. The create() operation will actually create the custom metadata definition within Atlan, including all the attributes that were defined as part of it. Because this operation will persist the structure in Atlan, you must provide it an AtlanClient through which to connect to the tenant.

Limit applicability of attribute​

You can also limit the assets the custom metadata applies to in Atlan. Anywhere you create an attribute definition, you can:

Limit applicability of an attribute
AttributeDef responsible = AttributeDef.of(client, "Responsible", // (1)
    AtlanCustomAttributePrimitiveType.USERS,
    false);
responsible = responsible.toBuilder() // (2)
    .options(responsible.getOptions().toBuilder() // (3)
        .clearApplicableConnections() // (4)
        .clearApplicableAssetTypes()
        .clearApplicableGlossaries()
        .clearApplicableGlossaryTypes()
        .applicableConnection("default/snowflake/1234567890") // (5)
        .applicableAssetType(Database.TYPE_NAME) // (6)
        .applicableGlossary(Glossary.findByName("Example").getQualifiedName()) // (7)
        .applicableGlossaryType(GlossaryTerm.TYPE_NAME) // (8)
        .build()) // (9)
    .build(); // (10)

  1. We still recommend creating the attribute using the Attribute.of() factory method. This ensures all required settings are configured based on the type of the attribute.

  2. You can then clone the attribute into a mutable form using toBuilder().

  3. Set the options on this clone to change its applicability. You can use the toBuilder() on the options themselves to get a mutable clone of the options that have already been setup by the AttributeDef.of() factory method.

  4. By default, the AttributeDef.of() method will make sure a custom metadata attribute applies to all assets. To limit its applicability, you need to remove these "grants" by clearing out:

    • Connections the custom metadata attribute applies to (by default, all assets in all connections that existed when the attribute was created will be capable of using this custom metadata attribute).
    • Asset types the custom metadata attribute applies to (by default, all asset types will be capable of using this custom metadata attribute).
    • Glossaries the custom metadata attribute applies to (by default, all objects in a glossary that existed when the attribute was created will be capable of using this custom metadata attribute).
    • Glossary asset types the custom metadata applies to (by default, glossaries, terms and categories will be capable of using this custom metadata attribute).

  5. You can chain any number of applicableConnection() calls to specify the qualiedNames of connections. The custom metadata attribute will only be available to assets within these connections.

    To use all connections

To select all connections, instead chain .applicableConnections(Connection.getAllQualifiedNames()). ::: 6. You can chain any number of applicableAssetType() calls to specify the types of assets for the custom metadata attribute. The custom metadata attribute will only be available to assets of these types, within the connections specified in the line above.

To use all asset types

To select all asset types, instead chain .applicableAssetTypes(AttributeDefOptions.ALL_ASSET_TYPES).

  1. You can chain any number of applicableGlossary() calls to specify the qualifiedNames of glossaries. The custom metadata attribute will only be available to assets within these glossaries.

    To use all glossaries

To select all glossaries, instead chain .applicableGlossaries(Glossary.getAllQualifiedNames()). ::: 8. You can chain any number of applicableGlossaryType() calls to specify the types of glossary assets for the custom metadata attribute. The custom metadata attribute will only be available to glossary assets of these types, within the glossaries specified in the line above.

To use all glossary asset types

To select all glossary asset types, instead use .applicableGlossaryTypes(AttributeDefOptions.ALL_GLOSSARY_TYPES).

  1. You then need to build all of these options.
  2. And finally you need to build the changes back into the attribute definition. You can then use the attribute definition (responsible in this example) as you would any other attribute definition, for example passing it to the chained .attributeDef() as part of CustomMetadataDef.creator() shown earlier.
Was this page helpful?