Set up E2E tests

End-to-end tests validate complete workflows from start to finish in your Application SDK application. This guide walks you through creating the directory structure, configuration files, and test implementations for E2E testing.

Prerequisites

Before you begin, make sure you have:

An Application SDK application project
Understanding of the Application SDK test framework
Access to test credentials for your data source

Prepare test environment

Create a tests folder at the root of your application:
```
mkdir -p tests/e2e
```
The tests folder can also contain other test types like unit and integration tests in their own subdirectories.
Create a dedicated folder for each testing scenario within the e2e directory using the format test_{app}_{workflow}_{scenario}.

For example, to test a Redshift workflow with IAM user authentication:
```
mkdir -p tests/e2e/test_redshift_workflow_iam_user_authentication
```

Create a schema subfolder in your test scenario folder:

mkdir -p tests/e2e/test_redshift_workflow_iam_user_authentication/schema

Create a raw subfolder for unprocessed data schemas:

mkdir -p tests/e2e/test_redshift_workflow_iam_user_authentication/schema/raw

Create schema/raw/schema.yaml with your Pandera schema definition:

Example raw data schema

schema_type: dataframe
version: 0.22.1
columns:
  table_catalog:
    title: Table Catalog Name Check
    description: Check if the catalog name of records in the raw data is matched as expected
    dtype: str
    nullable: false
    checks:
      str_matches:
        value: dev
        options:
          raise_warning: false
    unique: false
    coerce: false
    required: true
    regex: false
  table_schema:
    title: Table Schema Name Check
    description: Check if the schema name of records in the raw data is matched as expected
    dtype: str
    nullable: false
    checks:
      isin:
        value: ['workflows', 'public']
        options:
          raise_warning: false
    unique: false
    coerce: false
    required: true
    regex: false
  table_name:
    title: Check Table Names
    description: Check if the table names are within the expected list
    dtype: str
    nullable: false
    unique: false
    coerce: false
    required: true
    regex: false
checks:
  check_record_count_ge:
    value: 1776
    options:
      raise_warning: false
index: null
dtype: null
coerce: false
strict: false
name: null
ordered: false
unique: null
report_duplicates: all
unique_column_names: true
add_missing_columns: false
title: Raw Column Pandera Schema
description: Pandera schema definition for the raw column output

This schema validates that the raw data contains the expected columns with appropriate data types and constraints, and verifies that the data includes at least the specified number of records.

Create a transformed subfolder for processed data schemas:

mkdir -p tests/e2e/test_redshift_workflow_iam_user_authentication/schema/transformed

Create similar YAML files in schema/transformed/ for data after your application has processed and transformed it.

Create config.yaml in your test scenario folder with the test configuration:

Example configuration file

test_workflow_args:
  credentials:
    username: $E2E_REDSHIFT_USERNAME
    password: $E2E_REDSHIFT_PASSWORD
    host: $E2E_REDSHIFT_HOST
    port: $E2E_REDSHIFT_PORT
    extra:
      database: "dev"
    authType: "basic"
    type: "all"
    database: "dev"
  metadata:
    exclude-filter: "{}"
    include-filter: '{"^dev$":["^workflows$","^public$"]}'
    temp-table-regex: ""
    extraction-method: "direct"
  connection:
    connection_name: "dev"
    connection_qualified_name: "default/redshift/1738930739"
test_name: "test_redshift_workflow_empty_exclude"
server_config:
  server_host: "http://localhost:8000"
  server_version: "workflows/v1"
expected_api_responses:
  auth:
    success: true
    message: "Authentication successful"
  metadata:
    success: true
  preflight_check:
    success: true
    data:
      databaseSchemaCheck:
        success: true
        successMessage: "Schemas and Databases check successful"
        failureMessage: ""
      tablesCheck:
        success: true
        successMessage: "Tables check successful. Table count: 155"
        failureMessage: ""

Replace credential values with environment variables (like $E2E_REDSHIFT_USERNAME) that GitHub Actions populates from secrets.

Create a Python test file in your test scenario folder. Name it to match the folder name. For example:
test_redshift_workflow_iam_user_authentication.py
```
import unittest

from application_sdk.test_utils.e2e.base import BaseTest


class TestRedshiftWorkflowIamUserAuthentication(unittest.TestCase, BaseTest):
    extracted_output_base_path = (
        "./local/dapr/objectstore/artifacts/apps/redshift/workflows"
    )
```
The test class:
- inherits from both unittest.TestCase and BaseTest
- Sets extracted_output_base_path to specify where the workflow stores its output, and inherits all standard E2E test methods from BaseTest.

You can override methods from BaseTest to customize test behavior or add additional test methods specific to your scenario.

Run your E2E test to verify the setup:

pytest tests/e2e/test_redshift_workflow_iam_user_authentication/

The test discovers the config.yaml file, loads schema files from the schema/ directory, executes the test workflow, validates data using the schemas, and reports results.

Prerequisites​

Prepare test environment​

See also​

Prerequisites

Prepare test environment

See also