Skip to main content

Configure tests

This reference provides configuration options, parameters, and structure patterns for testing applications built with the Application SDK. Use this document to look up specific configuration options, naming patterns, and parameter details.

Test structure and naming​

Configure test file organization and naming conventions to match framework requirements.

Test file naming​

test_<module_name>.pystring
Required

Test file naming pattern. Replace <module_name> with the source module name being tested. For example, `statestore.py` becomes `test_statestore.py`.

Examples:
  • test_statestore.py
  • test_metadata.py
  • test_workflow.py

Test class naming​

Test<ClassName>string
Required

Test class naming pattern. Use the class name being tested with 'Test' prefix. For example, `StateStore` class becomes `TestStateStore` test class.

Examples:
  • TestStateStore
  • TestMetadataHandler
  • TestWorkflowClient

Test method naming​

test_<method_name>_<scenario>string
Required

Test method naming pattern. Use the method name being tested followed by the test scenario. For example, `test_get_state_success` or `test_get_state_not_found_returns_empty_dict`.

Examples:
  • test_get_state_success
  • test_save_state_updates_existing
  • test_save_state_object_creates_new

Test directory structure​

test_type_directoriesdirectory structure
Required

Separate test types into distinct directories. Unit tests in `tests/unit/`, integration tests in `tests/integration/`, and E2E tests in `tests/e2e/`.

Examples:
  • tests/unit/
  • tests/integration/
  • tests/e2e/

Unit test decorators and markers​

Configure unit tests with decorators and markers for async operations, mocking, and error handling.

Async test marker​

@pytest.mark.asynciodecorator
Required

Decorator to mark async test functions. Required for all tests that use async/await syntax.

Example:@pytest.mark.asyncio

Example:

@pytest.mark.asyncio
async def test_get_state_success():
    state = await StateStore.get_state("test-id", StateType.WORKFLOWS)
    assert state is not None

Mocking decorator​

@patchdecorator
Required

Use unittest.mock.patch to mock external dependencies. Use AsyncMock for asynchronous methods.

Examples:
  • @patch("application_sdk.services.statestore.ObjectStore.get_content", new_callable=AsyncMock)
  • @patch("application_sdk.services.api_client.requests.get")

Example:

@patch(
    "application_sdk.services.statestore.ObjectStore.get_content",
    new_callable=AsyncMock
)
async def test_get_state_mocked(mock_get_content):
    mock_get_content.return_value = '{"status": "running"}'
    state = await StateStore.get_state("test-id", StateType.WORKFLOWS)
    assert state["status"] == "running"

Exception testing​

pytest.raisescontext manager
Required

Context manager to verify that exceptions are raised correctly. Use for testing error handling and invalid inputs.

Examples:
  • pytest.raises(json.JSONDecodeError)
  • pytest.raises(ValueError)
  • pytest.raises(ConnectionError)

Example:

async def test_get_state_invalid_json():
    with pytest.raises(json.JSONDecodeError):
        await StateStore.get_state("invalid-id", StateType.WORKFLOWS)

Property-based testing parameters​

Configure Hypothesis property-based testing to automatically generate diverse test cases.

Hypothesis strategies​

hypothesis strategiesstrategy definition
Optional

Reusable strategies defined in `application_sdk/test_utils/hypothesis/strategies` that generate random test data following specific formats. Strategies generate data that matches your application's requirements.

Examples:
  • sql_credentials_strategy
  • parquet_input_config_strategy
  • workflow_args_strategy

Strategy decorator​

@givendecorator
Required

Hypothesis decorator that applies a strategy to generate test inputs. The strategy automatically generates many test cases with diverse inputs.

Example:@given(config=parquet_input_config_strategy)

Example:

from hypothesis import given
from application_sdk.test_utils.hypothesis.strategies import sql_credentials_strategy

@given(credentials=sql_credentials_strategy)
def test_credential_validation(credentials: Dict[str, Any]) -> None:
    result = validate_credentials(credentials)
    assert result is not None
    assert result["username"] == credentials["username"]

Strategy definition example​

sql_credentials_strategy = st.fixed_dictionaries(
    {
        "username": st.text(),
        "password": st.text(),
        "database": st.text(),
        "schema": st.text(),
        "warehouse": st.text().map(lambda x: x.upper() + "_WH"),
        "role": st.sampled_from(["ACCOUNTADMIN", "SYSADMIN", "USERADMIN", "PUBLIC"]),
    }
)

E2E test configuration​

Configure end-to-end tests using YAML configuration files and Pandera schema files.

E2E test folder structure​

test_{app}_{workflow}_{scenario}directory name
Required

E2E test scenario folder naming pattern. Each scenario gets its own folder with this naming convention.

Examples:
  • test_redshift_workflow_iam_user_authentication
  • test_postgresql_workflow_basic_auth
  • test_snowflake_workflow_oauth

Each test scenario folder must contain:

  • config.yaml: Test configuration file
  • schema/: Directory containing Pandera schema files
    • raw/: Schema files for raw/unprocessed data
    • transformed/: Schema files for processed/transformed data
  • test_{scenario_name}.py: Python test file implementing the test

Config.yaml parameters​

The config.yaml file contains all configuration needed to run your E2E test.

test_workflow_argsdict
Required

Workflow parameters including credentials, metadata configuration, and connection details. Credentials use environment variable format `$VARIABLE_NAME` for security.

Examples:
  • credentials: {username: $E2E_USERNAME, password: $E2E_PASSWORD}
  • metadata: {include-filter: '{"^dev$":["^workflows$"]}'}
  • connection: {connection_name: "dev"}

server_configdict
Required

Server host and API version information for API client configuration.

Examples:
  • server_host: "http://localhost:8000"
  • server_version: "workflows/v1"

expected_api_responsesdict
Required

Expected responses from each API endpoint for validation. Defines success conditions for auth, metadata, and preflight_check endpoints.

Examples:
  • auth: {success: true, message: "Authentication successful"}
  • metadata: {success: true, data: [...]}
  • preflight_check: {success: true, data: {...}}

test_namestring
Required

Unique identifier for the test scenario. Used for test reporting and identification.

Examples:
  • "test_redshift_workflow_empty_exclude"
  • "test_postgresql_workflow_basic_auth"

Example config.yaml:

test_workflow_args:
  credentials:
    username: $E2E_REDSHIFT_USERNAME
    password: $E2E_REDSHIFT_PASSWORD
    host: $E2E_REDSHIFT_HOST
    port: $E2E_REDSHIFT_PORT
    extra:
      database: "dev"
    authType: "basic"
    type: "all"
    database: "dev"
  metadata:
    exclude-filter: "{}"
    include-filter: '{"^dev$":["^workflows$","^public$"]}'
    temp-table-regex: ""
    extraction-method: "direct"
  connection:
    connection_name: "dev"
    connection_qualified_name: "default/redshift/1738930739"
test_name: "test_redshift_workflow_empty_exclude"
server_config:
  server_host: "http://localhost:8000"
  server_version: "workflows/v1"
expected_api_responses:
  auth:
    success: true
    message: "Authentication successful"
  metadata:
    success: true
  preflight_check:
    success: true
    data:
      databaseSchemaCheck:
        success: true
        successMessage: "Schemas and Databases check successful"

Credentials management​

$VARIABLE_NAMEenvironment variable reference
Required

Format for referencing environment variables in config.yaml. GitHub Actions populate these from secrets, ensuring sensitive information never appears in your codebase.

Examples:
  • $E2E_REDSHIFT_USERNAME
  • $E2E_REDSHIFT_PASSWORD
  • $E2E_REDSHIFT_HOST

Pandera schema parameters​

Pandera-based YAML files define the expected structure and validation rules for your test data.

schema_typestring
Required

Type of schema. Use 'dataframe' for tabular data validation.

Example:"dataframe"

columnsdict
Required

Column definitions with data types, validation rules, and constraints. Each column specifies dtype, nullable, checks, and required status.

Examples:
  • table_catalog: {dtype: str, nullable: false, required: true}
  • table_schema: {dtype: str, nullable: false, checks: {isin: {value: ['workflows', 'public']}}}

checksdict
Optional

Dataframe-level validation checks. Common checks include record count validation (check_record_count_ge, check_record_count_le).

Examples:
  • check_record_count_ge: {value: 1776}
  • check_record_count_le: {value: 2000}

Example schema.yaml:

schema_type: dataframe
version: 0.22.1
columns:
  table_catalog:
    title: Table Catalog Name Check
    description: Check if the catalog name matches expected value
    dtype: str
    nullable: false
    checks:
      str_matches:
        value: dev
        options:
          raise_warning: false
    required: true
  table_schema:
    title: Table Schema Name Check
    dtype: str
    nullable: false
    checks:
      isin:
        value: ['workflows', 'public']
    required: true
checks:
  check_record_count_ge:
    value: 1776
    options:
      raise_warning: false

E2E test class parameters​

BaseTestbase class
Required

Base class for E2E tests that extends TestInterface. Provides standard implementation of common E2E operations including metadata fetching, preflight checks, and workflow execution.

Example:from application_sdk.test_utils.e2e.base import BaseTest

extracted_output_base_pathstring
Required

Path where the workflow stores its output artifacts. Used by BaseTest to locate and validate workflow results.

Examples:
  • "./local/dapr/objectstore/artifacts/apps/redshift/workflows"
  • "./local/dapr/objectstore/artifacts/apps/postgresql/workflows"

Example test class:

import unittest
from application_sdk.test_utils.e2e.base import BaseTest

class TestRedshiftWorkflowEmptyExclude(unittest.TestCase, BaseTest):
    extracted_output_base_path = (
        "./local/dapr/objectstore/artifacts/apps/redshift/workflows"
    )

GitHub Actions configuration​

Configure GitHub Actions for automated test execution and coverage reporting.

Unit test action​

.github/actions/unit-tests/action.yamlGitHub Action
Optional

GitHub Action that runs unit tests, generates coverage reports, posts results as pull request comments, and optionally uploads reports to storage.

E2E test action​

.github/actions/e2e-apps/action.yamlGitHub Action
Optional

GitHub Action that executes end-to-end workflows. Provides access to credentials from GitHub secrets securely and validates application behavior in realistic scenarios.

Coverage threshold​

coverage_thresholdconfiguration
Optional

Configured threshold that causes tests to fail if overall code coverage falls below the specified percentage. Prevents coverage from decreasing as the application evolves.

Examples:
  • 80
  • 85
  • 90

See also​