## Mine PostgreSQL

URL: https://docs.atlan.com/apps/connectors/database/postgresql/how-tos/mine-postgresql

> Once you have [crawled assets from PostgreSQL](/apps/connectors/database/postgresql/how-tos/crawl-postgresql), you can mine its query history to construct lineage.

Use the PostgreSQL Miner workflow to extract query history from `pg_stat_statements` and build lineage for your crawled PostgreSQL assets. This page walks you through creating, configuring, and running the miner.

## Prerequisites

Before you begin:

- Make sure you have [crawled PostgreSQL](https://docs.atlan.com/llms/connectors/postgresql/crawl-postgresql/llms.txt) at least once so a connection exists to mine.
- Make sure [`pg_stat_statements` is enabled](https://docs.atlan.com/llms/connectors/postgresql/generate-lineage-with-postgres-miner/llms.txt) on your PostgreSQL instance. The miner extracts query history from `pg_stat_statements` and can't run without it.

## Create miner workflow

To create the PostgreSQL miner workflow:

1. In the top right of any screen, navigate to _+New_ and then click **New workflow**.
2. Under _Marketplace_, from the filters along the top, click **Miner**.
3. From the list of packages, select **PostgreSQL Miner** and then click **Setup Workflow**.

### Configure miner

The PostgreSQL miner uses `pg_stat_statements`, which doesn't support time-based filtering like some other sources. The miner extracts the current query statistics; for incremental updates, run the miner on a schedule.

To configure the PostgreSQL miner:

1. For _Connection_, select the connection to mine. (To select a connection, [the crawler](https://docs.atlan.com/llms/connectors/postgresql/crawl-postgresql/llms.txt) must have already run.)
2. For _Miner Extraction Method_, choose your extraction method:
 - In **Query History**, Atlan connects to your database and mines query history directly from `pg_stat_statements`.
 - In **Agent**, Atlan uses a Self-Deployed Runtime deployed within your network to mine query history. For details on deploying the runtime, see:
 - [Install via Docker Compose](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-docker-compose/llms.txt)
 - [Install on Kubernetes](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-kubernetes/llms.txt)
 - In **Offline**, you need to first mine query history yourself and [make it available in S3](https://docs.atlan.com/llms/catalog/connector-capabilities/mine-queries-through-cloud-object-storage/llms.txt).
3. For _Advanced Config_, keep _Default_ for the default configuration or click **Advanced** to configure the miner:
 - For _Cross Connection_, click **Yes** to extract lineage across all available data source connections or click **No** to only extract lineage from the selected PostgreSQL connection.
 - For _Control Config_, if Atlan support has provided you with a custom control configuration, select **Custom** and enter the configuration into the _Custom Config_ box.

### Run miner

To run the PostgreSQL miner, after completing the configuration:

- To run the miner once, immediately, at the bottom of the screen, click the **Run** button.
- To schedule the miner to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the **Schedule & Run** button.

Once the miner completes running, you see lineage for PostgreSQL assets based on query history from `pg_stat_statements`.

## Need help

If you need help configuring the miner or enabling `pg_stat_statements`, contact Atlan Support by [submitting a request](https://docs.atlan.com/support/submit-request).

## See also

- [Generate lineage for PostgreSQL assets](https://docs.atlan.com/llms/connectors/postgresql/generate-lineage-with-postgres-miner/llms.txt): Enable `pg_stat_statements` and configure your PostgreSQL instance so the miner can extract query history

---