Skip to main content

Data Connectors

A Data Connector represents a location from and to which data can be read and written. For example, an AWS S3 bucket or a Google Cloud Storage bucket. By itself, a Data Connector does nothing, but paired with a batch job or batch exporter, it provides a flexible way to read or export data. When creating a Data Connector, credentials must be provided that grant the required access (read, write, or both).

The following data connector and credential types are currently supported:

Data Connector TypeCredentials Type
Amazon S3 BucketIAM User Access Key
Google Cloud Storage BucketService Account Credentials
Azure Blob Storage ContainerApplication Principal Client Secret Credentials
Mysql DatabaseDatabase connection URI
Postgresql DatabaseDatabase connection URI
BigQueryBigQuery connection URI

For examples on how to create and use these Data Connectors, see the Data Connectors quickstart, or the CLI reference for data connectors.