Official

Premium

Snowflake Config

Snowflake Config plugin pulls configuration information out of Snowflake and loads it into any supported CloudQuery destination

Publisher

cloudquery

Latest version

v4.2.0

Type

Source

Platforms

Date Published

October 22, 2024

Price per 1M rows

Starting from $15

monthly free quota

1M rows

Documentation Tables Changelog Destinations

Overview

Set up process #

brew install cloudquery/tap/cloudquery

1. Download CLI and login

See installation options

2. Create source and destination configs

Plugin configuration

cloudquery sync snowflake-config.yml postgresql.yml

3. Run the sync

CloudQuery sync

Overview #

Snowflake Config Source Plugin

The Snowflake Config source plugin helps you sync configuration information from your Snowflake account.

Example Config #

This example sets the connection string to a value read from the SNOWFLAKE_CONNECTION_STRING environment variable:

kind: source
spec:
  name: snowflake-config
  path: cloudquery/snowflake-config
  registry: cloudquery
  version: "v4.2.0"
  # Learn more about the configuration options at https://cql.ink/snowflake-config_source
  spec:
    connection_string: "${SNOWFLAKE_CONNECTION_STRING}"
    # Optional parameters
    # private_key: ""
    # concurrency: 10000

Authentication #

Authentication of the connection to Snowflake can be specified using:

A username and password in the DSN:

kind: source
spec:
  name: snowflake-config
  # ...
  spec:
    connection_string: "user:pass@account"

A private key inline:

kind: source
spec:
  name: snowflake-config
  # ...
  spec:
    connection_string: "user@account"
    private_key: |
      -----BEGIN PRIVATE KEY-----
      MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQC2ajPRIbPtbxZ1
      3DONLA02eZJuCzsgIkBWov/Me5TL6cKN0gnY+mbA8OnNCH+9HSzgiU9P8XhTUrIN
      85diD+rj6uK+E0sSyxGk6HG17TyR5oBq8nz2hbZlbaNi/HO9qYoHQgAgMq908YBz
      ...
      DUmOIrBYEMf2nDTlTu/QVcKb
      -----END PRIVATE KEY-----

A private key included from a file:

kind: source
spec:
  name: snowflake-config
  # ...
  spec:
    connection_string: "user@account"
    private_key: "${file:./private.key}"

where ./private.key is PEM-encoded private key file with contents of the form:

Private Key Authentication Setup #

The Snowflake guide for Key Pair Authentication demonstrates how to create an RSA private key with the ability to authenticate as a Snowflake user.

Encrypted private keys are not supported by the Snowflake Go SQL driver, and hence not supported by the CloudQuery Snowflake Config plugin. You can decrypt a private key in file enc.key and store it in a file dec.key using the following command:

openssl pkcs8 -topk8 -nocrypt -in enc.key -out dec.key

Snowflake Config Spec #

This is the top level spec used by the Snowflake Config source plugin.

connection_string (string) (required)
Snowflake connection_string.

The connection_string (DSN) value may include the database and schema in them, but these values will limit the results returned from API. We recommend using user[:password]@account value for more results.

Example:

# user[:password]@account
# or
# user[:password]@account/database/schema?warehouse=user_warehouse[&param1=value1&paramN=valueN]
# or
# user[:password]@account/database?warehouse=user_warehouse[&param1=value1&paramN=valueN]
# or
# user[:password]@host:port/database/schema?account=user_account&warehouse=user_warehouse[&param1=value1&paramN=valueN]
# or
# host:port/database/schema?account=user_account&warehouse=user_warehouse[&param1=value1&paramN=valueN]

From Snowflake documentation:

account - Name assigned to your Snowflake account. If you are not on us-west-2 or AWS deployment, append the region and platform to the end, e.g., <account>.<region> or <account>.<region>.<platform>.

private_key (string) (optional)
A PEM-encoded private key for connecting to Snowflake. Equivalent to adding authenticator=snowflake_jwt&privateKey=... to the connection_string but parses, validates, and correctly encodes the key for use with snowflake.
concurrency (integer) (optional) (default: 10000) A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.
scheduler (string) (optional) (default: dfs) The scheduler to use when determining the priority of resources to sync. Supported values are dfs (depth-first search), round-robin, shuffle and shuffle-queue.
For more information about this, see performance tuning.

Underlying library #

We use the same library as the official Terraform plugin for connection.