Set up process #

brew install cloudquery/tap/cloudquery

1. Download CLI and login

See installation options

2. Create source and destination configs

Plugin configuration

cloudquery sync openai.yml postgresql.yml

3. Run the sync

CloudQuery sync

Overview #

The CloudQuery OpenAI source plugin pulls data out of the OpenAI platform and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).

Authentication #

In order to fetch information from OpenAI, cloudquery needs to be authenticated. An organization admin key is required for authentication. See Administration API documentation for more information.

OpenAI Source Plugin Configuration Reference

Example #

This example syncs from OpenAI to a Postgres destination. The (top level) source spec section is described in the Source Spec Reference.

kind: source
# Common source-plugin configuration
spec:
  name: openai
  path: cloudquery/openai
  registry: cloudquery
  version: "v1.0.0"
  tables: ["*"]
  destinations: ["postgresql"]

  spec:
    # required parameters
    admin_api_key: "${OPENAI_ADMIN_API_KEY}"
    
    # optional parameters
    # concurrency: 10000

OpenAI Spec #

This is the (nested) spec used by the OpenAI source plugin.

The API Key can be generated from the OpenAI Cloud platform, under the My Account section.

admin_api_key (string) (required)
An organization admin key is required for authentication. See Administration API page for more information.
concurrency (integer) (optional) (default: 10000)
The best effort maximum number of Go routines to use. Lower this number to reduce memory usage.