We just raised $3.5M and we are hiring!

Google Cloud Provider

GCP Provider extends CloudQuery with ability to fetch information on Google Cloud resources and store it in PostgreSQL database.

$ cloudquery init gcp
Category
Public Cloud
Version
v0.7.3
License
MPL-2.0
Published at
Tue May 17 2022

The CloudQuery GCP provider pulls configuration out of GCP resources, normalizes them and stores them in PostgreSQL database.

Install

cloudquery init gcp

Authentication

CloudQuery needs to be authenticated with your GCP account in order to fetch information about your cloud setup. You need to set the GOOGLE_APPLICATION_CREDENTIALS environment variable - see GCP documentation for more details on where to get it.

On Linux/MacOS (Similar for windows):

export GOOGLE_APPLICATION_CREDENTIALS={Path to your google credentials}

Configuration

The following configuration section can be automaticlly generated by cloudquery init gcp:

provider "gcp" { configuration { // Optional. List of folders to get projects from. Required permission: resourcemanager.projects.list // folder_ids = [ "organizations/<ORG_ID>", "folders/<FOLDER_ID>" ] // Optional. Maximum level of folders to recurse into // folders_max_depth = 5 // Optional. If not specified either using all projects accessible. // project_ids = [<CHANGE_THIS_TO_YOUR_PROJECT_ID>] // Optional. ServiceAccountKeyJSON passed as value instead of a file path, can be passed also via env: CQ_SERVICE_ACCOUNT_KEY_JSON // service_account_key_json = <YOUR_JSON_SERVICE_ACCOUNT_KEY_DATA> // Optional. GRPC Retry/backoff configuration, time units in seconds. Documented in https://github.com/grpc/grpc/blob/master/doc/connection-backoff.md // backoff_base_delay = 1 // backoff_multiplier = 1.6 // backoff_max_delay = 120 // backoff_jitter = 0.2 // backoff_min_connect_timeout = 0 } resources = ["*"] }

By default cloudquery will fetch all configurations from all resources in all regions in the default project. You can customize this behaviour with the following arguments:

  • project_ids - Specify multiple projects that you want to fetch configurations from.

Query Examples:

Find all buckets without uniform bucket-level access

SELECT project_id, name, self_link AS link FROM gcp_storage_buckets WHERE iam_configuration_uniform_bucket_level_access_enabled = FALSE;