Manage AI assets

✦TL;DR

Create and manage AI models and AI applications using the Atlan Python SDK (pyatlan). Programmatically manage AI model assets and their relationships.

||

|Connect

AI model

Creating an AI model is a 2-step process:

1. Step 1: Create the minimal AI model with basic information
2. Step 2: Create processes to link the AI model with datasets used for training, testing, inference, validation, and output

An AI model requires a name and ai_model_status. The model can be associated with training and output datasets through processes.

Java

Coming soon

Python

Create an AI model

from pyatlan.client.atlan import AtlanClient
from pyatlan.model.assets import AIModel
from pyatlan.model.enums import AIDatasetType, AIModelStatus
from pyatlan.model.fluent_search import FluentSearch
from pyatlan.model.assets import Asset

client = AtlanClient()

# Step 1: Create the minimal AI model
model = AIModel.creator( # (1)
    name="test-ai-model", # (2)
    ai_model_status=AIModelStatus.ACTIVE # (3)
)
response = client.asset.save(model) # (4)
ai_model = response.mutated_entities.CREATE[0] # (5)

# Step 2: Create processes to link with datasets
query = (
    FluentSearch()
    .where(Asset.TYPE_NAME.eq("View"))
    .include_on_results(Asset.NAME)
    .include_on_results(Asset.GUID)
    .include_on_results(Asset.TYPE_NAME)
).to_request()

list_training = []
for results in client.asset.search(query):
    list_training.append(results)

query = (
    FluentSearch()
    .where(Asset.TYPE_NAME.eq("Table"))
    .include_on_results(Asset.NAME)
    .include_on_results(Asset.GUID)
    .include_on_results(Asset.TYPE_NAME)
).to_request()

list_output = []
for results in client.asset.search(query):
    list_output.append(results)

dataset_dict = {
    AIDatasetType.TRAINING: list_training,
    AIDatasetType.OUTPUT: list_output
}
process = AIModel.processes_creator(ai_model, dataset_dict) # (6)
response = AIModel.processes_batch_save(client, process) # (7)

1. Build up the minimum request to create an AI model.
2. Provide a human-readable name for your AI model, such as gpt-4-model or bert-classifier.
3. Set the status of the AI model.
4. Actually call Atlan to create the AI model.
5. Retrieve the AI model response object to be used in the next step.
6. Create a process to link the AI model with its associated datasets.
7. Save the processes in batches of 20 to map the relationships between the AI model and the datasets.

Kotlin

Coming soon

Raw REST API

POST /api/meta/entity/bulk

{
  "entities": [ // (1)
    }
  ]
}

1. All assets must be wrapped in an entities array.
2. Specify the entity type as AIModel to create an AI model asset.
3. Provide a human-readable name for your AI model, such as gpt-4-model or bert-classifier.
4. Set the unique qualified name for the AI model in the format default/ai/aiapplication/{name} where {name} should be in camelCase.
5. Specify the connector name as ai for AI assets.
6. Set the status of the AI model (for example, ACTIVE, INACTIVE).
7. (optional) Specify the version of the AI model (for example, "1.2", "2.0").
8. (optional) Assign owner groups of this AI model.
9. (optional) Assign owner users of this AI model.
10. Set the cover image for the AI model asset.

POST /api/meta/entity/bulk

{
  "entities": [ // (1)
        ],
        "outputs": [ // (7)
        ],
        "__state": "ACTIVE" // (8)
      }
    },
    {
      "typeName": "Process",
      "attributes": {
        "name": "ai-model -> asset-name",
        "qualifiedName": "default/ai/dataset/4578fd36d3956f38b3123555bc5fac951e76ea7b3517e7b988e71b8191f36f3e",
        "aiDatasetType": "OUTPUT",
        "inputs": [
          {
            "typeName": "AIModel",
            "guid": "9065bba5-22b8-4331-bff8-1bdf017c5cfb"
          }
        ],
        "outputs": [
          {
            "typeName": "Table",
            "guid": "9065bba5-22b8-4331-bff8-1bdf017c5cfb"
          }
        ],
        "__state": "ACTIVE"
      }
    }
  ]
}

1. All assets must be wrapped in an entities array.
2. Specify the entity type as Process to create relationships between datasets and AI models.
3. Provide a descriptive name for the process showing the relationship (for example, "asset-name -> model").
4. Set the unique qualified name for the process in the format default/ai/dataset/{hash}. The MD5 hash is generated internally by the SDK based on process attributes.
5. Specify the dataset type (for example, TRAINING, TESTING, INFERENCE, VALIDATION, OUTPUT).
6. Define the input datasets that feed into the AI model.
7. Define the output AI model that receives the data.
8. Set the process state to ACTIVE to enable the relationship.

AI application

An AI application requires a name, ai_application_version, and ai_application_development_stage.

Java

Coming soon

Python

Create an AI application

from pyatlan.client.atlan import AtlanClient
from pyatlan.model.assets import AIApplication
from pyatlan.model.enums import AIApplicationDevelopmentStage

client = AtlanClient()

ai_application = AIApplication.creator( # (1)
    name="ai-app", # (2)
    ai_application_version="1.1", # (3)
    ai_application_development_stage=AIApplicationDevelopmentStage.PRODUCTION # (4)
)
response = client.asset.save(ai_application) # (5)

1. Build up the minimum request to create an AI application.
2. Provide a human-readable name for your AI application, such as chatbot-app or recommendation-engine.
3. Specify the version of the AI application (for example, "1.1", "2.12").
4. Set the development stage of the application.
5. Actually call Atlan to create the AI application.

Kotlin

Coming soon

Raw REST API

POST /api/meta/entity/bulk

{
  "entities": [ // (1),
      "relationshipAttributes": {
        "models": [] // (11)
      }
    }
  ]
}

1. All assets must be wrapped in an entities array.
2. Specify the entity type as AIApplication to create an AI application asset.
3. Provide a human-readable name for your AI application, such as chatbot-app or recommendation-engine.
4. Set the unique qualified name for the AI application in the format default/ai/aiapplication/{name} where {name} should be in camelCase.
5. Specify the version of the AI application (for example, "1.1", "2.12").
6. Set the development stage of the application (for example, PROPOSAL, DEVELOPMENT, PRODUCTION).
7. Set the certificate status.
8. (optional) Assign owner groups of this AI application.
9. (optional) Assign owner users of this AI application.
10. Set the cover image for the AI application asset.
11. (optional) Provide the AI models objects to be used by this application.

Next Step

• Learn about managing AI assets—common operations like adding descriptions, owners, and tags
• Search for AI assets—find specific AI models or applications
• Update AI assets—modify existing AI assets
• Delete AI assets—remove AI assets when no longer needed