Back to plugin list
postgresql
Official

PostgreSQL

This destination plugin lets you sync data from a CloudQuery source to a PostgreSQL compatible database.

Publisher

cloudquery

Repositorygithub.com
Latest version

v8.1.0

Type

Destination

Platforms
Date Published

Price

Free

Overview

PostgreSQL Destination Plugin

This destination plugin lets you sync data from a CloudQuery source to a PostgreSQL compatible database.
Supported database versions:
  • PostgreSQL >= v10
  • CockroachDB >= v20.2

Configuration

Example

This example configures a Postgresql destination using an environment variable called POSTGRESQL_CONNECTION_STRING:
kind: destination
spec:
  name: "postgresql"
  path: "cloudquery/postgresql"
  registry: "cloudquery"
  version: "v8.1.0"

  spec:
    connection_string: "${POSTGRESQL_CONNECTION_STRING}" # set the environment variable in a format like postgres://postgres:pass@localhost:5432/postgres?sslmode=disable
    # you can also specify it in DSN format, which can hold special characters in the password field:
    # connection_string: "user=postgres password=pass+0-[word host=localhost port=5432 dbname=postgres"
    # Optional parameters:
    # pgx_log_level: error
    # batch_size: 10000 # 10K entries
    # batch_size_bytes: 100000000 # 100 MB
    # batch_timeout: 60s
The (top level) spec section is described in the Destination Spec Reference.
The PostgreSQL destination utilizes batching, and supports batch_size and batch_size_bytes.

PostgreSQL Spec

This is the (nested) spec used by the PostgreSQL destination Plugin.
  • connection_string (string) (required)
    Connection string to connect to the database. This can be a URL or a DSN, as per pgxpool
    • "postgres://jack:secret@localhost:5432/mydb?sslmode=prefer" connect with tcp and prefer TLS
    • "postgres://jack:secret@localhost:5432/mydb?sslmode=disable&application_name=pgxtest&search_path=myschema&connect_timeout=5" be explicit with all options
    • "postgres://localhost:5432/mydb?sslmode=disable" connect with os username cloudquery is being run as
    • "postgres:///mydb?host=/tmp" connect over unix socket
    • "dbname=mydb" unix domain socket, just specifying the db name - useful if you want to use peer authentication
    • "user=jack password=jack\\'ssooper\\\\secret host=localhost port=5432 dbname=mydb sslmode=disable" DSN with escaped backslash and single quote
  • pgx_log_level (string) (optional) (default: error)
    Available: error, warn, info, debug, trace. Defines what pgx call events should be logged.
  • batch_size (integer) (optional) (default: 10000)
    Maximum number of items that may be grouped together to be written in a single write.
  • batch_size_bytes (integer) (optional) (default: 100000000 (= 100 MB))
    Maximum size of items that may be grouped together to be written in a single write.
  • batch_timeout (duration) (optional) (default: 60s (= 60 seconds))
    Maximum interval between batch writes.

Verbose logging for debug

The PostgreSQL destination can be run in debug mode.
Note: This will use pgx built-in logging and might output data/sensitive information to logs so make sure to not use it in production but only for debugging.
kind: destination
spec:
  name: postgresql
  path: cloudquery/postgresql
  registry: cloudquery
  version: "v8.1.0"
  spec:
    connection_string: ${PG_CONNECTION_STRING}
    pgx_log_level: debug # Available: error, warn, info, debug, trace. Default: "error"


Subscribe to product updates

Be the first to know about new features.