Skip to main content

40 docs tagged with "application-sdk"

View all tags

@needs_lock decorator

Reference documentation for the @needs_lock decorator that applies distributed locking to Temporal activities.

Atlas transformer

Complete reference for AtlasTransformer that converts metadata into Atlas entities using pyatlan library classes.

BaseHandler

Complete API reference for BaseHandler - the concrete implementation of HandlerInterface for non-SQL data sources.

BaseSQLHandler

Complete API reference for BaseSQLHandler - the concrete implementation of HandlerInterface for SQL-based data sources.

CleanupInterceptor

Automatically cleans up temporary artifacts and activity state when workflows complete or fail. Supports multiple cleanup base paths and comprehensive error handling.

Common utilities

Reference documentation for utility functions in the transformers module for text processing, qualified name building, and template path mapping.

Connect to source systems

Learn how handlers enable connections to source systems in the Application SDK, providing the operational abstraction for data source interactions.

Distributed locking

Frequently asked questions about distributed locking in the Application SDK including decorator usage, testing, and configuration.

Distributed locking

Understand distributed locking in the Application SDK and how it coordinates resource access across distributed Atlan applications to prevent race conditions and implement rate limiting.

Distributed locking issues

Troubleshoot distributed locking errors with Redis connection failures, lock acquisition timeouts, and Sentinel configuration issues.

EventInterceptor

Tracks workflow and activity execution events for monitoring and observability. Automatically publishes events when workflows and activities start and end.

HandlerInterface

Complete API reference for HandlerInterface - the abstract base class defining the contract that all handlers must fulfill.

IcebergInput

Reads data from Apache Iceberg tables using daft. Provides support for reading Iceberg table data as DataFrames with lazy evaluation.

IcebergOutput

Writes data to Apache Iceberg tables using daft. Supports table creation, schema inference, and multiple write modes.

Inputs

Complete reference for input classes that read data from various sources including SQL queries, Parquet files, JSON files, and Iceberg tables.

Interceptors

Complete reference for Temporal interceptors that provide cross-cutting concerns including event tracking, distributed locking, and automatic cleanup.

JsonInput

Reads data from JSON files, supporting both single files and directories containing multiple JSON files. Supports JSONL (JSON Lines) format where each line is a separate JSON object.

JsonOutput

Writes data to JSON files (JSONL format) with support for chunking, buffering, null field processing, and automatic object store uploads.

Metadata extraction workflows

Complete reference for workflows that orchestrate metadata extraction from SQL databases, including databases, schemas, tables, columns, and procedures.

Outputs

Complete reference for output classes that write data to various destinations including Parquet files, JSON files, and Iceberg tables.

ParquetInput

Reads data from Parquet files, supporting both single files and directories containing multiple Parquet files. Automatically handles local and object store paths.

ParquetOutput

Writes data to Parquet files with support for chunking, consolidation, Hive partitioning, and automatic object store uploads.

Query extraction workflows

Complete reference for workflows that orchestrate SQL query extraction from database query logs and history.

Query-based transformer

Complete reference for QueryBasedTransformer that transforms metadata using SQL queries defined in YAML templates.

RedisLockInterceptor

Manages distributed locks for activities using Redis. Limits concurrent execution to a specified number of activities with the same lock name.

Retry policies

Configuration reference for retry policies that control automatic retry behavior when operations fail.

Retry strategies

Understand retry strategies, timeouts, and heartbeats in the Application SDK to build resilient apps that handle failures gracefully.

SQLQueryInput

Reads data from SQL databases by executing SQL queries. Supports both SQLAlchemy engines and connection strings, with automatic handling of async and sync operations.

Timeouts

Timeout configuration parameters for Temporal workflows and activities.

Transformers

Overview of metadata transformers that convert raw metadata into Atlas-compatible entities for ingestion into Atlan.

Typedefs

Learn about Atlan Typedefs and how they define the structure and attributes of your metadata assets in Atlan.

Usage patterns

Common implementation patterns for using output classes to write data to various destinations.

Use distributed locking

Apply distributed locking to control concurrent operations across workflow instances and prevent resource overload.

Workflows

Overview of Temporal workflows that orchestrate long-running processes for metadata extraction and query mining in Atlan applications.