> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reelevant.com/llms.txt
> Use this file to discover all available pages before exploring further.

# BigQuery

> Synchronize data from Google BigQuery tables into Reelevant

<img src="https://mintcdn.com/reelevant/10tXeCE_biVlzX8s/images/datahub/source-bigquery-v2.png?fit=max&auto=format&n=10tXeCE_biVlzX8s&q=85&s=e8901e21f19b07b995eb38238c039bca" alt="BigQuery source configuration form" width="1280" height="800" data-path="images/datahub/source-bigquery-v2.png" />

## Overview

The BigQuery source synchronizes data from a Google BigQuery table directly into Reelevant. This is ideal for teams that centralize their data in BigQuery and want to leverage it for personalization.

## Configuration

### Required Fields

| Field       | Description                                                      |
| ----------- | ---------------------------------------------------------------- |
| `projectId` | The Google Cloud project ID that contains your BigQuery dataset. |
| `dataset`   | The BigQuery dataset name.                                       |
| `table`     | The table name to synchronize.                                   |

### Authentication

BigQuery supports two authentication methods:

<Tabs>
  <Tab title="Service Account">
    Authenticate using a Google Cloud service account key.

    | Field         | Description                                                                       |
    | ------------- | --------------------------------------------------------------------------------- |
    | `clientEmail` | The service account email address (e.g. `my-sa@project.iam.gserviceaccount.com`). |
    | `privateKey`  | The service account private key (from the JSON key file).                         |

    <Tip>
      The minimum required role is **BigQuery Data Viewer** (`roles/bigquery.dataViewer`) on the target dataset. If you use a custom SQL query, the service account also needs **BigQuery Job User** (`roles/bigquery.jobUser`) on the project.
    </Tip>
  </Tab>

  <Tab title="OAuth">
    Authenticate using an OAuth connection configured in Reelevant. This is useful when you need to access BigQuery resources on behalf of a user rather than a service account.
  </Tab>
</Tabs>

### Optional Fields

| Field                 | Description                                                                                                                           |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| `query`               | A custom SQL query to filter or transform data before import. When provided, the query is executed instead of reading the full table. |
| `tableProjectId`      | If the table belongs to a different GCP project than the service account, specify the table's project ID here.                        |
| `customOAuthClientId` | DatasourceAuth ID for OAuth-based authentication (alternative to service account).                                                    |

### Custom SQL Queries

When you provide a custom SQL query, Reelevant executes it against BigQuery instead of reading the entire table. This is useful for:

* Filtering rows (e.g. `SELECT * FROM my_table WHERE active = true`)
* Selecting specific columns
* Joining multiple tables
* Applying transformations

<Warning>
  The query must be valid BigQuery Standard SQL. Make sure the service account has permission to run jobs in the project.
</Warning>

## Incremental Sync

BigQuery supports incremental synchronization when used with the **Keep Previous Data** update mode. To enable this:

| Field                | Description                                                        |
| -------------------- | ------------------------------------------------------------------ |
| `timestampField`     | The column used to track new or updated rows since the last sync.  |
| `timestampFieldType` | Whether the field is a `datetime` or `timestamp` type in BigQuery. |

When configured, Reelevant only fetches rows where the timestamp field is greater than the last recorded value, appending new data to the existing dataset.

<Info>
  If your datasource uses the **Override Previous Data** update mode (the default), the timestamp field is not needed — the entire table or query result is re-imported on each sync.
</Info>

## Supported Data Types

BigQuery data types are automatically converted:

| BigQuery Type                           | Behavior                                  |
| --------------------------------------- | ----------------------------------------- |
| `STRING`, `INT64`, `FLOAT64`, `BOOL`    | Passed through as-is.                     |
| `DATE`, `DATETIME`, `TIME`, `TIMESTAMP` | Converted to their string representation. |
| `NUMERIC`, `BIGNUMERIC`                 | Converted to JavaScript numbers.          |
| `STRUCT` (nested records)               | Flattened into individual fields.         |
| `ARRAY` (repeated fields)               | Preserved as arrays.                      |

## How It Works

1. Reelevant authenticates with BigQuery using the provided credentials.
2. If a custom query is provided, it is executed. Otherwise, the full table is exported.
3. For large tables, Reelevant uses an optimized export-to-GCS pipeline for better performance.
4. Fields are extracted and made available for [mapping](/advanced-guide/datahub/field-mapping).
5. On subsequent syncs, the query or table is re-read (or incrementally fetched if configured).

<Warning>
  Ensure the service account has read access to the specified dataset and table. The minimum required role is **BigQuery Data Viewer**. For custom queries, the service account also needs **BigQuery Job User**.
</Warning>