Skip to main content
Community Hub
TL;DR

Import glossaries, glossary term links, descriptions, metadata forms, and Glue skills from AWS Glue business context into Atlan during an AWS Glue crawl.

Your AI can read this via Docs MCPcurl -fsSL "https://docs.atlan.com/install-docs-mcp" | bashConnect

Import AWS Glue business context Private Preview

Private Preview

Importing AWS Glue business context is in private preview and is enabled per tenant. The Import business context option stays hidden in the workflow setup until the feature is enabled for your tenant. It works only in the AWS regions us-east-1, us-east-2, us-west-2, and eu-west-1. To request access, contact your Atlan customer team. Import Glue skills has the same private preview access. It appears under Import business context, and only when that option is Yes.

The AWS Glue business context APIs that Atlan reads are also in preview at AWS, so AWS can change them. For details, see the AWS announcement and the AWS Glue business context user guide.

Atlan can import the business context you maintain in AWS Glue, such as glossaries, glossary term links, descriptions, and metadata forms, and add it to the Glue assets in Atlan. The import runs after the technical crawl, in the same workflow run. Atlan only reads from AWS Glue and never writes anything back.

The import is off by default. Once you crawl AWS Glue with Import business context set to Yes, your Glue tables, views, and columns carry the terms, descriptions, and form values set on them in AWS.

With Import business context on, you can also turn on Import Glue skills to import the markdown files that AWS Glue links to your tables as Atlan Skills. For details, see Import Glue skills.

Prerequisites​

Before you import business context, make sure that:

  • Business context is turned on for your tenant. Contact your Atlan customer team. The Import business context option stays hidden until it is.
  • You're an admin user in Atlan.
  • Your AWS Glue connection uses one of the AWS regions in the preview: us-east-1, us-east-2, us-west-2, or eu-west-1. In any other region, Atlan skips business context and the rest of the crawl runs normally.
  • Your AWS administrator has added the business context read permissions to the IAM policy used by the connection, as described in the next section.

Grant IAM permissions for business context​

Add the following read actions to the IAM policy you created for Atlan, either in the existing statement or in a new one:

{
"Effect": "Allow",
"Action": [
"glue:ListGlossaries",
"glue:ListGlossaryTerms",
"glue:GetGlossaryTerm",
"glue:GetAsset",
"glue:BatchGetIterableForms",
"glue:ListFormTypes",
"glue:GetFormType"
],
"Resource": "*"
}

If any of these actions is missing, Atlan skips business context and the rest of the crawl runs normally. Preflight checks list the missing actions.

note

The Atlan CloudFormation template doesn't include these permissions yet. If you created the IAM policy from the template, add them to the policy by hand.

Import AWS Glue business context to Atlan​

To import business context, either create a new AWS Glue workflow or modify the crawler's configuration of an existing one. On the metadata step of the workflow:

  1. For Import business context, click Yes.
  2. Under Preflight checks, click Check to confirm that the region and the IAM permissions support the import. For the messages you may see, see Check your setup with preflight checks.
  3. Run or schedule the crawler.

For an existing workflow, editing the workflow and saving it with Import business context set to Yes is enough. The next run imports business context.

Once the crawler has completed running, the imported glossaries, terms, descriptions, and form values are on your Glue assets in Atlan. Atlan imports business context only for the tables and views that the crawl includes, so your include and exclude filters apply.

What Atlan imports​

In AWS GlueIn Atlan
GlossaryA glossary named <Glue glossary name> (<connection name>), for example Sales (production). The connection name keeps it apart from glossaries in Atlan that have the same name.
Glossary termA term in that glossary. The term description is the Glue long description, or the short description when there's no long description.
Glossary term linked to a table, view, or columnThe term linked to the same asset in Atlan.
Description set in Glue business context on a table or view, for example with update-assetThe description of that asset. It takes precedence over the Glue table description. Descriptions that users edit in Atlan are kept separately and aren't overwritten.
Customer-defined metadata form typeA custom metadata set named AWS Glue: <form type name>, with the Glue logo. The set is locked, so it's read-only in Atlan, and it applies to tables, views, and columns.
Form values on an assetThe values of that custom metadata set on the same asset.

Atlan skips AWS-managed form types.

How form field types map​

Each field in a Glue form type becomes a field in the custom metadata set.

Glue field typeAtlan field type
StringText
IntegerInteger
LongInteger (large)
DoubleDecimal
BooleanBoolean
TimestampDate
Any other typeText

Atlan imports up to 25 form types, and up to 250 fields per form type. Form types beyond those limits are skipped, and the run ends as a partial success. Custom metadata sets never delete or retype fields. If the type of a field changes in Glue, Atlan skips that field.

How changes in Glue are reflected in Atlan​

Each run reads business context again, so changes in AWS Glue reach Atlan on the next run:

Change in AWS GlueResult in Atlan on the next run
A form is removed from an assetThe values from that form are cleared on the asset.
A term is unlinked from an assetThe term link is removed from the asset.
A term is deletedThe term is archived.
A glossary is deletedThe glossary and its terms are archived.

The connector only adds and removes links to terms from its own glossaries, the ones named <Glue glossary name> (<connection name>). Links to any other term, including terms users link to assets directly in Atlan, are never changed.

Turn off business context import​

Set Import business context to No and save the workflow. Glossaries, terms, and term links already imported stay in Atlan. Table and view descriptions go back to the Glue table description on the next run.

Limitations​

  • Region. Business context is available only in us-east-1, us-east-2, us-west-2, and eu-west-1 during the AWS preview.
  • Scope. Atlan imports business context for the tables and views that the crawl includes. AWS business context doesn't use Lake Formation, so Atlan reads business context only for tables that the crawl itself can see.
  • Federated catalogs. Business context isn't imported yet for tables in federated catalogs, such as S3 Tables. Atlan still crawls these tables as usual, and imports business context for tables in the default AWS Glue Data Catalog.
  • Catalog-level descriptions. These aren't imported yet.
  • Run time. Atlan makes about two AWS calls per table, one at a time. Crawls of large catalogs take longer with the import turned on.

Import Glue skills​

Glue skills are markdown files in Amazon S3 that AWS Glue links to tables. When you turn on Import Glue skills, Atlan imports them as Atlan Skills, with their markdown and lineage from the tables they're linked to. The option is off by default.

Glue skills follow the same limits as business context: they work only in the four AWS regions that business context supports, and a link to a table in a federated catalog doesn't count, because business context skips those tables.

Grant IAM permissions for Glue skills​

On top of the business context permissions, add the following to the IAM policy you created for Atlan:

  • glue:SearchAssets, to list skills
  • s3:GetObject on the bucket and prefix that hold the skill files
  • kms:Decrypt on the key, only when the skill files use a customer-managed KMS key

Add these statements to the policy:

{
"Effect": "Allow",
"Action": "glue:SearchAssets",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::<bucket-name>/<prefix>/*"
},
{
"Effect": "Allow",
"Action": "kms:Decrypt",
"Resource": "arn:aws:kms:<region>:<account-id>:key/<key-id>"
}

Replace <bucket-name> and <prefix> with the bucket and prefix that hold your skill files. Replace <region>, <account-id>, and <key-id> with the region, AWS account ID, and ID of the KMS key. Leave out the last statement when your skill files don't use a customer-managed key.

note

The Atlan CloudFormation template doesn't include these permissions either. If you created the IAM policy from the template, add them to the policy by hand.

If the skill files are in another AWS account, that bucket's policy must also let the Atlan IAM role read them, and so must the key policy when the files use a customer-managed key.

Without glue:SearchAssets, Atlan skips Glue skills and keeps the skills it already imported. Without s3:GetObject or kms:Decrypt, Atlan still imports the skills with their links but can't read their markdown. The rest of the crawl runs normally in both cases. Preflight checks report both.

Turn on Glue skills import​

To import Glue skills, either create a new AWS Glue workflow or modify the crawler's configuration of an existing one. On the metadata step of the workflow:

  1. For Import business context, click Yes. The Import Glue skills option appears under it.
  2. For Import Glue skills, click Yes.
  3. Under Preflight checks, click Check to confirm that the permissions support the import. Check your setup with preflight checks explains the messages.
  4. Run or schedule the crawler.

For an existing workflow, editing the workflow and saving it with Import Glue skills set to Yes is enough. The next run imports Glue skills.

What Atlan imports for Glue skills​

In AWS GlueIn Atlan
Glue skills linked to the crawled tables of a catalogA context repository named Glue skills (<catalog ID>), one per catalog, for example Glue skills (AwsDataCatalog).
Glue skillA Skill. Its name is the Glue skill name, or the skill identifier when the skill has no name. It carries the description and the S3 URI of the markdown as its source path.
Markdown file of a skillA skill artifact on the Skill, holding the markdown content.
Link between a skill and a table or viewA lineage process from the table or view to the Skill, named <skill name> describes <table name>.

To find a skill, search for its name in Atlan, or open a linked table and go to the Lineage tab.

The identity of a Skill is its Glue skill identifier, so renaming a skill in Glue keeps the same asset in Atlan.

Atlan cuts markdown over 256 KiB at 256 KiB and adds a note saying the markdown was cut. Atlan imports a skill without content when its URI isn't an s3:// URI.

Which skills Atlan imports​

Glue lists skills for the whole AWS account and region, across all catalogs. Atlan imports only the skills linked to a table or view that this connection crawls. A link counts whichever side declared it, the skill or the table.

Atlan skips skills linked only to tables outside the crawl, for example tables your filters exclude or that Lake Formation hides. It never reads the markdown of a skipped skill, and the run logs how many skills it skipped.

How Glue skill changes reach Atlan​

Each run reads Glue skills again, so changes reach Atlan on the next run:

ChangeResult in Atlan on the next run
A skill is renamed in AWS GlueThe Skill stays the same asset.
A skill is deleted in AWS GlueThe Skill is archived.
You remove a skill's last link to a crawled tableThe Skill is archived on the next run. Link it again and it comes back as the same asset.
You turn off Import Glue skills or Import business contextAll imported Skills are archived on the next run. Turn it back on, with Import Glue skills on, and they return as the same assets.

When a read fails, Atlan archives nothing and the run ends as a partial success:

What failsWhat stays in Atlan
Business context is skipped or failsEvery imported Skill stays as it was.
glue:SearchAssets is deniedEvery imported Skill stays as it was.
One skill can't be readThat Skill keeps its imported copy while the other Skills update.
A markdown file can't be read from S3The Skill updates and keeps its imported markdown.
Atlan can't read a tableThat table's Skills stay, because their link might be on it.

Turn off Glue skills import​

Set Import Glue skills to No and save the workflow. The next run archives every imported Skill. Set it back to Yes to restore them. Setting Import business context to No also archives every imported Skill on the next run.

Check your setup with preflight checks​

With Import business context set to Yes, preflight checks add two checks. Both always pass, so they never block you from running the crawler. The message tells you what Atlan skips.

CheckMessageWhat it means
Business context regionBusiness context is available in <region>.The import can run.
Business context regionBusiness context is not available in <region> during the AWS preview. The rest of the crawl runs as normal.Atlan skips business context. Use a connection in a supported region to import it.
Business context permissionsBusiness context permissions validated.The IAM principal can call the business context actions.
Business context permissionsBusiness context will be skipped. Missing IAM permissions: <actions>. The rest of the crawl runs as normal.Add the listed actions to the IAM policy, then run the crawler again.

If the region isn't supported or permissions are missing, the technical crawl still completes. The workflow run ends as a partial success and shows the reason business context was skipped.

With Import Glue skills also set to Yes, preflight checks add two more checks. They're advisory and never block the crawler.

CheckMessageWhat it means
Glue skills search assetsGlue skills permissions validated. or Glue skills are readable, but none were found.Atlan can list Glue skills.
Glue skills search assetsGlue skills will be skipped. Missing IAM permission: glue:SearchAssets. The rest of the crawl runs as normal.Add glue:SearchAssets to the IAM policy, then run the crawler again.
Glue skills search assetsCould not check glue:SearchAssets. The crawl will report it if skills cannot be read. The rest of the crawl runs as normal.Atlan couldn't test the permission. The crawl reports it if it can't read skills.
Glue skills S3 readSkill markdown is readable from S3.Atlan can read the markdown of a skill.
Glue skills S3 readSkill markdown will be skipped: s3:GetObject is denied on the skill files, or kms:Decrypt on their customer-managed key. Skills still import with their links.Add s3:GetObject on the skill files, and kms:Decrypt on the key when the files use a customer-managed key. Then run the crawler again.
Glue skills S3 readCould not read a skill file from S3. Skills still import with their links.The read of one skill file failed. Skills still import with their links.
Glue skills S3 readNo S3 skill file to check; skill markdown will be read during the crawl where allowed.Atlan found no S3 skill file to test. The crawl reads the markdown it has permission to read.

If Atlan can't read skills or their markdown, the technical crawl still completes and the workflow run ends as a partial success. Markdown that isn't in S3 and markdown that gets cut make no difference to the run status.

See also​