## Mine ClickHouse

URL: https://docs.atlan.com/apps/connectors/database/clickhouse/how-tos/mine-clickhouse

> Once you have [crawled assets from ClickHouse](/apps/connectors/database/clickhouse/how-tos/crawl-clickhouse), you can mine its query history to construct lineage.

Use the ClickHouse Miner workflow to extract query history from `system.query_log` and build lineage for your crawled ClickHouse assets. This page walks you through creating, configuring, and running the miner.

## Prerequisites

Before you begin:

- Make sure you have [crawled ClickHouse](https://docs.atlan.com/llms/connectors/clickhouse/crawl-clickhouse/llms.txt) at least once so a connection exists to mine.
- Make sure [query logging is enabled](https://docs.atlan.com/llms/connectors/clickhouse/generate-lineage-with-clickhouse-miner/llms.txt) on your ClickHouse instance. The miner extracts query history from the `system.query_log` table and can't run without it.

## Create miner workflow

To create the ClickHouse miner workflow:

1. In the top right of any screen, navigate to _+New_ and then click **New workflow**.
2. Under _Marketplace_, from the filters along the top, click **Miner**.
3. From the list of packages, select **ClickHouse Miner** and then click **Setup Workflow**.

### Configure miner

The miner restricts you to the past two weeks of query history. If you need more history (for example, during an initial load), use [Generic Miner](https://docs.atlan.com/llms/catalog/connector-capabilities/mine-queries-through-cloud-object-storage/llms.txt) first, then [switch to query history extraction](https://docs.atlan.com/llms/catalog/connector-capabilities/manage-connectivity/llms.txt) afterward.

To configure the ClickHouse miner:

1. For _Connection_, select the connection to mine. (To select a connection, [the crawler](https://docs.atlan.com/llms/connectors/clickhouse/crawl-clickhouse/llms.txt) must have already run.)
2. For _Miner Extraction Method_, choose your extraction method:
 - In **Query History**, Atlan connects to your database and mines query history directly from `system.query_log`.
 - In **Agent**, Atlan uses a Self-Deployed Runtime deployed within your network to mine query history. For details on deploying the runtime, see:
 - [Install via Docker Compose](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-docker-compose/llms.txt)
 - [Install on Kubernetes](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-kubernetes/llms.txt)
 - In **Offline**, you need to first mine query history yourself and [make it available in S3](https://docs.atlan.com/llms/catalog/connector-capabilities/mine-queries-through-cloud-object-storage/llms.txt).
3. For _Start date_, choose the earliest date from which to mine query history.
4. For _Advanced Config_, keep _Default_ for the default configuration or click **Advanced** to configure the miner:
 - For _Cross Connection_, click **Yes** to extract lineage across all available data source connections or click **No** to only extract lineage from the selected ClickHouse connection.
 - For _Control Config_, if Atlan support has provided you with a custom control configuration, select **Custom** and enter the configuration into the _Custom Config_ box.

### Run miner

To run the ClickHouse miner, after completing the configuration:

- For your first run, set the start date to three days before today, then click **Schedule & Run** to run the miner daily. The miner requires a 24–48 hour lag to capture all session transformations, so building up history gradually avoids delays. Learn more about [miner logic](https://docs.atlan.com/llms/catalog/lineage/troubleshooting-lineage/llms.txt).
- To run the miner once, immediately, at the bottom of the screen, click the **Run** button.
- To schedule the miner to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the **Schedule & Run** button.

Once the miner completes running, you see lineage for ClickHouse assets based on query history from `system.query_log`.

## Need help

If you need help configuring the miner or enabling query logging, contact Atlan Support by [submitting a request](https://docs.atlan.com/support/submit-request).

## See also

- [Generate lineage for ClickHouse assets](https://docs.atlan.com/llms/connectors/clickhouse/generate-lineage-with-clickhouse-miner/llms.txt): Enable query logging and configure your ClickHouse instance so the miner can extract query history

---