Official

Premium

Terraform

The CloudQuery Terraform plugin extracts terraform state and loads it into any supported CloudQuery destination

Publisher

cloudquery

Latest version

v4.7.1

Type

Source

Platforms

Date Published

October 9, 2024

Price per 1M rows

Starting from $15

monthly free quota

1M rows

Documentation Tables Changelog Destinations

Overview

Set up process #

brew install cloudquery/tap/cloudquery

1. Download CLI and login

See installation options

2. Create source and destination configs

Plugin configuration

cloudquery sync terraform.yml postgresql.yml

3. Run the sync

CloudQuery sync

Overview #

The CloudQuery Terraform plugin extracts terraform state and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).

Configuration #

kind: source
spec:
  # Source spec section
  name: terraform
  path: cloudquery/terraform
  registry: cloudquery
  version: "v4.7.1"
  tables: ["*"]
  destinations: ["postgresql"]
  # Learn more about the configuration options at https://cql.ink/terraform_source
  spec:
    backends:
      - name: local
        type: local
        config:
          prefix: /tmp/cloudquery
      - name: s3
        type: s3
        config:
          bucket: <bucket>
          prefix: <prefix>
          region: <region>
          role_arn: <role_arn>

You can have multiple backends at the same time, by describing them in the configuration. Every configuration block describes one backend to handle. CloudQuery currently supports local and s3 backends.

Authentication for S3 backends #

To authenticate CloudQuery with your Terraform state in S3 you can use any of the following options (see full documentation at AWS SDK V2):

Static Credentials: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN
Shared configuration files (via aws configure).
SDK defaults to credentials file under .aws folder that is placed in the home folder on your computer
SDK defaults to config file under .aws folder that is placed in the home folder on your computer
If your application uses an ECS task definition or RunTask API operation, IAM role for tasks
If your application is running on an Amazon EC2 instance, IAM role for Amazon EC2

Terraform Spec #

This is the (nested) spec used by the Terraform source plugin:

backends ([]Backend specs) (optional)
concurrency (integer) (optional) (default: 10000)
Best effort maximum number of Go routines to use. Lower this number to reduce memory usage.

Terraform Backend Spec #

name (string) (optional)
Name of the backend. Should be set if there are more than one backend.
type (BackendType, "local" or "s3") (required)
Type of the backend.
config (Backend config) (required)
Backend-type specific configuration

Backend Type Config for Local backend

prefix (string) (required)
Filesystem search prefix to find all .tfstate files.
path (string) (optional)
Deprecated, replaced by prefix! Path to a .tfstate file.

Backend Type Config for S3 backend

bucket (string) (required)
Name of the S3 bucket
prefix (string) (required)
Search prefix to find all .tfstate objects in the specified bucket.
region (string) (optional)
Region to use for S3 client. If not specified, GetBucketRegion will be called to determine the region.
role_arn (string) (optional)
ARN of the role to assume when retrieving the object from S3
key (string) (optional)
Deprecated, replaced by prefix! Path to a .tfstate object in bucket