Back to plugin list
yc

Yandex Cloud

The Yandex Cloud plugin extracts information from many of the supported services by Yandex Cloud and loads it into any supported CloudQuery destination.

Publisher

yandex-cloud

Repositorygithub.com
Latest version

v1.1.1

Type

Source

Platforms
Date Published

Price

Free

Set up process #


brew install cloudquery/tap/cloudquery

1. Download CLI and login

See installation options

2. Create source and destination configs

Plugin configuration

cloudquery sync yc.yml postgresql.yml

3. Run the sync

CloudQuery sync

Overview #

Yandex Cloud Source Plugin

The CloudQuery Yandex Cloud plugin pulls configuration out of Yandex Cloud resources and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).

Authentication #

Authentication is done via IAM tokens/keys. You can read more about it here
It is recommended to grant read-only permissions/roles:
  • auditor role – primitive role which gives access to all configurations, but not the data.
  • *.viewer roles – per-sevice roles to read resource. Sometimes it is necessary to grant this role, if the service doesn't support auditor role.
You can read more about roles here
The plugin will get the following environment variables:
  • YC_SERVICE_ACCOUNT_KEY – Service Account key (in json format)
  • YC_TOKEN – IAM or OAuth token
If none of the variables are set, plugin will try to use Compute Metadata API to get IAM token.

Sentry #

If SENTRY_DSN environment variable is set, plugin will send errors to provided Sentry DSN. Global tags can be set via SENTRY_TAGS_* environment variables (source)

Query Examples #

Find all compute instances having public IPv4 address #

with instances as (select yci.*,
                          iface #>> '{primary_v4_address, one_to_one_nat, address}' address
                   from (select id,
                                folder_id,
                                name,
                                labels,
                                fqdn,
                                jsonb_array_elements(network_interfaces) as iface
                         from yc_compute_instances) yci)
select *
from instances
where address is not null

Find all public object storage buckets (access flags only) #

with buckets as (select *,
                        (anonymous_access_flags #> '{list, value}')::bool        as flags_list,
                        (anonymous_access_flags #> '{read, value}')::bool        as flags_read,
                        (anonymous_access_flags #> '{config_read, value}')::bool as flags_config_read
                 from yc_storage_buckets)
select *
from buckets
where flags_list
   or flags_read
   or flags_config_read


Configuration #

Yandex Cloud Source Plugin Configuration Reference

Example #

kind: source
spec:
  name: "yc"
  registry: cloudquery
  path: yandex-cloud/yc
  version: "v1.0.0"
  destinations: ["postgresql"]
  tables: 
    ["*"]
  spec:
    organization_ids: # sync these organizations only
      - bpf...
    cloud_ids: # sync these clouds only
      - b1g... 
    folder_ids: # sync these folders only
      - b1g... 
---
kind: destination
spec:
  name: "postgresql"
  path: "cloudquery/postgresql"
  registry: "cloudquery"
  version: "v7.1.2"
  spec:
    connection_string: "${PG_CONNECTION_STRING}"

YC Spec #

  • organization_ids ([]string, optional, default: empty): List of Organization IDs to target. If empty, all available Organization will be targeted.
  • cloud_ids ([]string, optional, default: empty): List of Cloud IDs to target. If empty, all available Clouds will be targeted.
  • folder_ids ([]string, optional, default: empty): List of Folder IDs to target. If empty, all available Folders will be targeted.
  • debug_grpc (bool, default false): If true, will log all GRPC calls
  • max_retries (int, default 3): Maxiumum number of retries for YC Client
  • endpoint (string, default api.cloud.yandex.net:443): Yandex Cloud endpoint
  • concurrency (int, optional, default: 10000): A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.
  • scheduler (string, default: shuffle): The scheduler to use when determining the priority of resources to sync. Currently, the only supported values are dfs (depth-first search), round-robin and shuffle.



© 2024 CloudQuery, Inc. All rights reserved.