Skip to main content
GCS source configuration form

​
Overview

The GCS (Google Cloud Storage) source retrieves data files from a GCS bucket. Use it when your data is stored as files in Google Cloud and you want to import them into Reelevant for personalization.

​
Configuration

​
Required Fields

​
Optional Fields

The minimum required role is Storage Object Viewer (roles/storage.objectViewer) on the target bucket.

​
Supported File Formats

GCS automatically detects the file format. The following formats are supported:
Compressed files (.gz, .zip) are automatically decompressed before parsing.

​
PGP Decryption

The GCS source supports PGP-encrypted files. When pgpPrivateKey is configured, files are decrypted transparently before decompression and parsing. Both armored (.asc) and binary (.pgp, .gpg) encrypted files are supported.
See the PGP Decryption guide for details on key generation, supported formats, and error handling.

​
Wildcard Paths

You can use a wildcard (*) in the file path to match multiple files. This is useful when:
  • A new file is exported periodically with a different name (e.g. exports/products_20240101.csv, exports/products_20240102.csv)
  • Data is split across multiple files in the same directory
Example patterns:
  • exports/products_*.csv — matches all CSV files starting with products_ in the exports/ folder
  • data/*.json — matches all JSON files in the data/ folder

​
File Processing Modes

When using wildcard paths, you can configure how files are selected via the processType field:

​
How It Works

  1. Reelevant authenticates using the provided service account credentials.
  2. The specified bucket and path are accessed. If a wildcard is used, matching files are listed.
  3. Files are downloaded, decompressed if needed, and parsed based on the detected format.
  4. Fields are extracted and made available for mapping.
  5. On subsequent syncs, files are re-fetched according to the configured processing mode.
Ensure the service account has read access to the specified bucket and objects. The minimum required role is Storage Object Viewer.