> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reelevant.com/llms.txt
> Use this file to discover all available pages before exploring further.
# FTP & SFTP
> Connect to FTP or SFTP servers to retrieve data files for your datasource
## Overview
The FTP and SFTP sources let you retrieve data files from remote file servers. This is useful for scheduled feeds from partners, legacy systems, or any infrastructure that publishes data via FTP/SFTP.
## FTP Configuration
### Required Fields
| Field | Description |
| ---------- | -------------------------------------------------------------------------- |
| `host` | The FTP server hostname or IP address. |
| `port` | The FTP port (default: `21`). |
| `username` | FTP login username. |
| `password` | FTP login password. |
| `path` | The file path on the server (see [Wildcard Paths](#wildcard-paths) below). |
### Optional Fields
| Field | Description |
| -------------- | -------------------------------------------------------------------------------------------------------- |
| `secure` | Enable FTPS (FTP over TLS) for encrypted connections. |
| `processType` | File processing mode when using wildcard paths: `last_unprocess` *(default)*, `all_unprocess`, or `all`. |
| `disableProxy` | Disable the SOCKS5 proxy for the FTP connection (optional). |
## SFTP Configuration
### Required Fields
| Field | Description |
| ---------- | -------------------------------------------------------------------------- |
| `host` | The SFTP server hostname or IP address. |
| `port` | The SFTP port (default: `22`). |
| `username` | SFTP login username. |
| `path` | The file path on the server (see [Wildcard Paths](#wildcard-paths) below). |
### Authentication
SFTP supports two authentication methods:
Authenticate using a username and password.
| Field | Description |
| ---------- | -------------------- |
| `password` | SFTP login password. |
Authenticate using an SSH private key.
| Field | Description |
| ------------------------- | ------------------------------------------------------- |
| `privateKey` | The SSH private key in PEM format. |
| `passphrase` *(optional)* | The passphrase for the private key, if it is encrypted. |
When a private key is provided, password authentication is disabled.
Use SFTP when security is a concern — it encrypts the connection and all data transfer. If your SFTP server supports key-based authentication, prefer it over passwords.
## Supported File Formats
FTP/SFTP automatically detects the file format. The following formats are supported:
| Format | Description |
| ----------- | --------------------------------------------------------------------------------- |
| **CSV** | Comma-separated values. Delimiter is auto-detected (comma, semicolon, tab, pipe). |
| **JSON** | Standard JSON files with a root array or object. |
| **NDJSON** | Newline-delimited JSON (one JSON object per line). |
| **XML** | XML files — the root element path is auto-detected. |
| **Parquet** | Apache Parquet columnar format. |
| **Avro** | Apache Avro serialization format. |
| **XLSX** | Microsoft Excel files. |
Compressed files (`.gz`, `.zip`) are automatically decompressed before parsing.
## PGP Decryption
FTP and SFTP sources support fetching **PGP-encrypted files**. When `pgpPrivateKey` is configured, files are decrypted transparently before decompression and parsing.
| Field | Required | Description |
| --------------- | -------- | -------------------------------------------------- |
| `pgpPrivateKey` | Yes | The PGP/GPG private key in armored (ASCII) format. |
| `pgpPassphrase` | No | The passphrase for the private key, if encrypted. |
Both armored (`.asc`) and binary (`.pgp`, `.gpg`) encrypted files are supported.
See the [PGP Decryption guide](/advanced-guide/datahub/pgp-decryption) for details on key generation, supported formats, and error handling.
## Wildcard Paths
You can use a wildcard (`*`) in the file name to match multiple files. This is useful when:
* A new file is exported periodically with a different name (e.g. `/exports/products_20240101.csv`, `/exports/products_20240102.csv`)
* Data is split across multiple files in the same directory
**Example patterns:**
* `/exports/products_*.csv` — matches all CSV files starting with `products_` in the `/exports/` directory
* `/data/*.json` — matches all JSON files in the `/data/` directory
## File Processing Modes
When using wildcard paths, you can configure how files are selected via the `processType` field:
| `processType` value | Description |
| ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `last_unprocess` *(default)* | Process only the most recent file that hasn't been processed yet. Ideal for full-dataset exports where only the latest file matters. |
| `all_unprocess` | Process all files that haven't been processed yet. Useful for incremental/delta exports. |
| `all` | Process all matching files on every sync, regardless of whether they were processed before. Useful for full datasets split across multiple files with the same names. |
## Reelevant-Hosted SFTP
For datasources configured via the platform, Reelevant can also provide an SFTP endpoint where your systems can push files directly. This is useful when:
* Your infrastructure prefers to push data rather than have it pulled
* You want to update data files without accessing the Reelevant platform
The SFTP credentials (host, username, password) are generated automatically when the datasource is created.
## How It Works
1. Reelevant connects to the FTP/SFTP server using the provided credentials.
2. The file at the configured path is located. If a wildcard is used, matching files are listed.
3. Files are downloaded, decompressed if needed, and parsed based on the detected format.
4. Fields are extracted and made available for [mapping](/advanced-guide/datahub/field-mapping).
5. On subsequent syncs, files are re-fetched according to the configured processing mode.