Skip to main content

What does Atlan crawl from Google BigQuery?

Once you have crawled Google BigQuery, you can use connector-specific filters for quick asset discovery. The following filters are currently supported for these assets:

Atlan doesn't run any table scans. Atlan leverages the table preview options from Google BigQuery that enable you to view data for free and without affecting any quotas using the tabledata.list API. Hence, table asset previews in Atlan are already cost-optimized. However, this doesn't apply to views and materialized views.

For Google BigQuery views and materialized views, Atlan sends you a cost nudge before viewing a sample data preview. This informs you about the precise bytes that are spent during the execution of the query, helping you decide if you still want to run the preview.

Did you know?

You also receive a cost nudge before querying your Google BigQuery assets.

Atlan crawls and maps the following assets and properties from Google BigQuery.

Databases

Atlan maps projects from Google BigQuery to its Database asset type.

Source propertyAtlan propertyWhere in Atlan
Project IDnameasset preview, profile, and filter, overview sidebar

Schemas

Atlan maps datasets from Google BigQuery to its Schema asset type.

Source propertyAtlan propertyWhere in Atlan
TABLE_SCHEMAnameasset preview and profile, overview sidebar
TABLE_COUNTtableCountasset preview and profile
VIEW_COUNTviewsCountasset preview and profile, overview sidebar
TABLE_CATALOGdatabaseNameasset preview
REMARKSdescriptionasset preview and profile, overview sidebar
CREATEDsourceCreatedAtasset profile and properties sidebar
MODIFIEDsourceUpdatedAtasset profile and properties sidebar

Tables

Did you know?

Table asset previews are already cost-optimized. Google BigQuery enables you to use the table preview options to view data for free and without affecting any quotas. Note that this isn't currently supported for Google BigQuery views and materialized views in Atlan.

Atlan maps tables from Google BigQuery to its Table asset type.

Source propertyAtlan propertyWhere in Atlan
TABLE_NAMEnameasset preview and profile, overview sidebar
REMARKSdescriptionasset preview and profile, overview sidebar
COLUMN_COUNTcolumnCountasset preview and profile, overview sidebar
ROW_COUNTrowCountasset preview, profile, and filter, overview sidebar
SIZE_BYTESsizeBytesasset filter and overview sidebar
TABLE_TYPEsubTypeasset preview and profile
LABELSassetTagsoverview sidebar
CREATEDsourceCreatedAtasset profile and properties sidebar
MODIFIEDsourceUpdatedAtasset profile and properties sidebar
OPTION_NAMES (require_partition_filter)isPartitionedAPI only

Views

Atlan maps views from Google BigQuery to its View asset type.

Source propertyAtlan propertyWhere in Atlan
TABLE_NAMEnameasset preview and profile, overview sidebar
REMARKSdescriptionasset preview and profile, overview sidebar
COLUMN_COUNTcolumnCountasset preview and profile, overview sidebar
TABLE_TYPEsubTypeasset preview and profile
CREATEDsourceCreatedAtasset profile and properties sidebar
MODIFIEDsourceUpdatedAtasset profile and properties sidebar
OPTION_NAMES (require_partition_filter)isPartitionedAPI only
DDLdefinitionasset profile and overview sidebar

Materialized views

Atlan maps materialized views from Google BigQuery to its MaterialisedView asset type.

Source propertyAtlan propertyWhere in Atlan
TABLE_NAMEnameasset preview and profile, overview sidebar
REMARKSdescriptionasset preview and profile, overview sidebar
COLUMN_COUNTcolumnCountasset preview and profile, overview sidebar
ROW_COUNTrowCountasset preview, profile, and filter, overview sidebar
SIZE_BYTESsizeBytesasset filter and overview sidebar
TABLE_TYPEsubTypeasset preview and profile
CREATEDsourceCreatedAtasset profile and properties sidebar
MODIFIEDsourceUpdatedAtasset profile and properties sidebar
OPTION_NAMES (require_partition_filter)isPartitionedAPI only
DDLdefinitionasset profile and overview sidebar

Columns

Atlan supports nested columns up to level 1 for Google BigQuery to help you enrich your semi-structured data types. You can view nested columns in the asset sidebar for your table assets. Atlan maps columns from Google BigQuery to its Column asset type.

Important

Atlan doesn't crawl primary key (PK) and foreign key (FK) information from Google BigQuery.

Source propertyAtlan propertyWhere in Atlan
COLUMN_NAMEnameasset preview and profile, overview sidebar
REMARKS, DESCRIPTIONdescriptionasset preview and profile, overview sidebar
ORDINAL_POSITIONorderasset profile
TYPE_NAMEdataTypeasset preview, profile, and filter, overview sidebar
IS_NULLABLEisNullableAPI only
IS_PARTITIONING_COLUMNisPartitionasset preview, profile, and filter
CLUSTERING_COLUMN_LISTisClusteredasset preview, profile, and filter

Stored procedures

Atlan maps stored procedures from Google BigQuery to its Procedure asset type.

Source propertyAtlan propertyWhere in Atlan
PROCEDURE_NAMEnameAPI only
REMARKSdescriptionAPI only
PROCEDURE_TYPEsubTypeAPI only
ROUTINE_DEFINITIONdefinitionAPI only
CREATEDsourceCreatedAtAPI only
MODIFIEDsourceUpdatedAtAPI only