This syncs data from a CloudQuery source to a PostgreSQL-compatible database. Supports RDS, Prisma, Aiven, Neon, Supabase and many others. It also comes with a built-in support for pgvector extension that allows you to store, query, and index vectors.
Loading plugin documentation
CloudQuery's use of cookies
We use tracking cookies to understand how you use the product and help us improve it. Your consent is required before we can enable these cookies. You can opt out via the link in the footer.
POSTGRESQL_CONNECTION_STRING:kind: destination
spec:
name: "postgresql"
path: "cloudquery/postgresql"
registry: "cloudquery"
version: "v8.14.1"
write_mode: "overwrite-delete-stale"
send_sync_summary: true
# Learn more about the configuration options at https://cql.ink/postgresql_destination
spec:
# set the environment variable in DSN format like "user=postgres password=pass+0-[word host=localhost port=5432 dbname=postgres sslmode=disable"
# you can also specify it in URI format like "postgres://postgres:pass@localhost:5432/postgres?sslmode=disable". any special URI characters need to be percent-encoded
connection_string: "${POSTGRESQL_CONNECTION_STRING}"
# Optional parameters:
# pgx_log_level: error
# batch_size: 10000 # 10K entries
# batch_size_bytes: 100000000 # 100 MB
# batch_timeout: 60s
# create_performance_indexes: false #create indexes that help with performance when using `write_mode: overwrite-delete-stale`
batch_size and batch_size_bytes.connection_string (string) (required)pgxpool"user=user password=pass+0-[word host=localhost port=5432 dbname=mydb sslmode=disable" DSN format"postgres://user:pass@localhost:5432/mydb?sslmode=prefer" connect with tcp and prefer TLS"postgres://user:pass@localhost:5432/mydb?sslmode=disable&application_name=pgxtest&search_path=myschema&connect_timeout=5" be explicit with all options"postgres://localhost:5432/mydb?sslmode=disable" connect with os username cloudquery is being run as"postgres:///mydb?host=/tmp" connect over unix socket"dbname=mydb" unix domain socket, just specifying the db name - useful if you want to use peer authenticationpgx_log_level (string) (optional) (default: error)batch_size (integer) (optional) (default: 10000)batch_size_bytes (integer) (optional) (default: 100000000 (= 100 MB))batch_timeout (duration) (optional) (default: 60s (= 60 seconds))create_performance_indexes (boolean) (optional) (default: false)write_mode: overwrite-delete-stale.pgvector_config (object) (optional)_cq_id column on target tables if this is enabled.tables (array) (required)source_table_name (string) (required)target_table_name (string) (required)embedding vector column, a chunk text column, and the configured metadata columns. The _cq_id column is always included and indexed.embed_columns (array) (required)metadata_columns (array) (optional)_cq_id column will be added automatically and an index will be created on it.text_splitter (object) (optional)recursive_text (object) (required)chunk_size (integer) (required)chunk_overlap (integer) (required)openai_embedding (object) (required)dimensions (integer) (required)text-embedding-3-small, this is 1536. For text-embedding-3-large, this is 3072.api_key (string) (required)model_name (string) (required)text-embedding-3-small and text-embedding-3-large are supported.retry_on_deadlock (integer) (optional) (default: 0)40P01).pgx built-in logging and might output data/sensitive information to logs so make sure to not use it in production but only for debugging.kind: destination
spec:
name: postgresql
path: cloudquery/postgresql
registry: cloudquery
version: "v8.14.1"
send_sync_summary: true
spec:
connection_string: ${PG_CONNECTION_STRING}
pgx_log_level: debug # Available: error, warn, info, debug, trace. Default: "error"
connection_string field in the sync config file.postgres://postgres:pass@localhost:5432/postgres?sslmode=disable. Any special URI characters need to be percent-encoded."user=postgres password=pass+0-[word host=localhost port=5432 dbname=postgres sslmode=disable"\x00) are automatically stripped from string values for PostgreSQL compatibilityjsonb with null characters stripped from string valuestimestamp without time zone in UTC