Skip to main content

52 docs tagged with "application-sdk"

View all tags

@needs_lock decorator

Reference documentation for the @needs_lock decorator that applies distributed locking to Temporal activities.

Atlas transformer

Complete reference for AtlasTransformer that converts metadata into Atlas entities using pyatlan library classes.

Authentication & metadata

Database connectivity and metadata operations including credential validation, schema retrieval, and preflight checks.

BaseHandler

Complete API reference for BaseHandler - the concrete implementation of HandlerInterface for non-SQL data sources.

BaseSQLHandler

Complete API reference for BaseSQLHandler - the concrete implementation of HandlerInterface for SQL-based data sources.

Build custom app

Create a local file extraction application from scratch using the patterns and tools you learned

CleanupInterceptor

Automatically cleans up temporary artifacts and activity state when workflows complete or fail. Supports multiple cleanup base paths and comprehensive error handling.

Common utilities

Reference documentation for utility functions in the transformers module for text processing, qualified name building, and template path mapping.

Configuration management

Workflow and system configuration management including configuration retrieval, updates, and configuration maps.

Configure tests

Complete reference for configuring tests in the Application SDK, including naming conventions, configuration parameters, and structure patterns.

Connect to source systems

Learn how handlers enable connections to source systems in the Application SDK, providing the operational abstraction for data source interactions.

Distributed locking

Frequently asked questions about distributed locking in the Application SDK including decorator usage, testing, and configuration.

Distributed locking

Understand distributed locking in the Application SDK and how it coordinates resource access across distributed Atlan applications to prevent race conditions and implement rate limiting.

Distributed locking issues

Troubleshoot distributed locking errors with Redis connection failures, lock acquisition timeouts, and Sentinel configuration issues.

EventInterceptor

Tracks workflow and activity execution events for monitoring and observability. Automatically publishes events when workflows and activities start and end.

HandlerInterface

Complete API reference for HandlerInterface - the abstract base class defining the contract that all handlers must fulfill.

IcebergInput

Reads data from Apache Iceberg tables using daft. Provides support for reading Iceberg table data as DataFrames with lazy evaluation.

IcebergOutput

Writes data to Apache Iceberg tables using daft. Supports table creation, schema inference, and multiple write modes.

Inputs

Complete reference for input classes that read data from various sources including SQL queries, Parquet files, JSON files, and Iceberg tables.

Interceptors

Complete reference for Temporal interceptors that provide cross-cutting concerns including event tracking, distributed locking, and automatic cleanup.

JsonInput

Reads data from JSON files, supporting both single files and directories containing multiple JSON files. Supports JSONL (JSON Lines) format where each line is a separate JSON object.

JsonOutput

Writes data to JSON files (JSONL format) with support for chunking, buffering, null field processing, and automatic object store uploads.

Metadata extraction workflows

Complete reference for workflows that orchestrate metadata extraction from SQL databases, including databases, schemas, tables, columns, and procedures.

Outputs

Complete reference for output classes that write data to various destinations including Parquet files, JSON files, and Iceberg tables.

ParquetInput

Reads data from Parquet files, supporting both single files and directories containing multiple Parquet files. Automatically handles local and object store paths.

ParquetOutput

Writes data to Parquet files with support for chunking, consolidation, Hive partitioning, and automatic object store uploads.

Query extraction workflows

Complete reference for workflows that orchestrate SQL query extraction from database query logs and history.

Query-based transformer

Complete reference for QueryBasedTransformer that transforms metadata using SQL queries defined in YAML templates.

RedisLockInterceptor

Manages distributed locks for activities using Redis. Limits concurrent execution to a specified number of activities with the same lock name.

Retry policies

Configuration reference for retry policies that control automatic retry behavior when operations fail.

Retry strategies

Understand retry strategies, timeouts, and heartbeats in the Application SDK to build resilient apps that handle failures gracefully.

Server APIs

Comprehensive REST APIs for Application SDK server management, workflow operations, authentication, and configuration.

Server management

Monitor and manage the Application SDK server with health checks, readiness probes, and graceful shutdown operations.

Set up E2E tests

Create end-to-end tests for your Application SDK application with directory structure, configuration files, and test implementations.

SQLQueryInput

Reads data from SQL databases by executing SQL queries. Supports both SQLAlchemy engines and connection strings, with automatic handling of async and sync operations.

Test your apps

Understand the Application SDK testing framework and how different testing approaches complement each other.

Testing best practices

Recommended approaches for writing effective tests in the Application SDK based on framework requirements and patterns.

Timeouts

Timeout configuration parameters for Temporal workflows and activities.

Transformers

Overview of metadata transformers that convert raw metadata into Atlas-compatible entities for ingestion into Atlan.

Typedef toolkit

Learn about typedef toolkit and how it simplifies the process of defining and managing typedefs for your connector applications.

Typedefs

Learn about Atlan Typedefs and how they define the structure and attributes of your metadata assets in Atlan.

UI & Documentation

UI and documentation endpoints for serving the application homepage, observability dashboard, and static documentation files.

Usage patterns

Common implementation patterns for using output classes to write data to various destinations.

Use distributed locking

Apply distributed locking to control concurrent operations across workflow instances and prevent resource overload.

Workflow management

Complete workflow lifecycle management including HTTP-triggered execution, status monitoring, and workflow termination.

Workflows

Overview of Temporal workflows that orchestrate long-running processes for metadata extraction and query mining in Atlan applications.