New
Join our webinar! Building a customizable and extensible cloud asset inventory at scale
Report an issue
Back to source list
alicloud
Official
Premium

Alibaba Cloud

The Alibaba Cloud source plugin for CloudQuery extracts configuration from the Alibaba Cloud API and loads it into any supported CloudQuery destination

Publisher

cloudquery

Latest version

v5.9.4

Type

Source

Platforms
Date Published

Price per 1M rows

Starting from $17

monthly free quota

1M rows

Overview #

Alibaba Cloud Source Plugin

The Alibaba Cloud source plugin for CloudQuery extracts configuration from the Alibaba Cloud (阿里云) API and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).

Configuration #

The following configuration syncs from Alibaba Cloud to a Postgres destination. The (top level) source spec section is described in the Source Spec Reference. The config for the postgresql destination is not shown here. See our Quickstart if you need help setting up the destination.
kind: source
spec:
  name: "alicloud"
  path: "cloudquery/alicloud"
  registry: "cloudquery"
  version: "v5.9.4"
  tables: ["*"]
  destinations: ["postgresql"]
  # Learn more about the configuration options at https://cql.ink/alicloud_source
  spec:
    accounts:
      - name: my_account
        regions:
        - cn-hangzhou
        - cn-beijing
        - eu-west-1
        - us-west-1
        # ...
        access_key: ${ALICLOUD_ACCESS_KEY}
        secret_key: ${ALICLOUD_SECRET_KEY}
    # Optional parameters
    # concurrency: 50000
    # bill_history_months: 12

Alibaba Spec #

  • concurrency (integer, optional, default: 50000):
    A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.
  • scheduler (string, optional, default: dfs):
    The scheduler to use when determining the priority of resources to sync. Supported values are dfs (depth-first search), round-robin, shuffle and shuffle-queue.
    For more information about this, see performance tuning.
  • accounts (array[object], required):
    A list of accounts to sync. Every account must have a unique name, and must specify at least one region. The access_key and secret_key are required and can be specified as environment variables, as shown in the example above.
    • name (string, required): A unique name for the account.
    • regions (array[string], required): A list of regions to sync. For example, ["cn-hangzhou", "cn-beijing"].
    • access_key (string, required): A valid access key for the account
    • secret_key (string, required): A valid secret key for the account, corresponding to the access key
  • bill_history_months (integer, optional):
    The number of months of billing history to fetch for the alicloud_bss_bill and alicloud_bss_bill_overview tables. Defaults to 12.
See the Alibaba documentation for how to obtain an AccessKey pair.

Authentication #

Authentication is done via access keys. See Obtain an AccessKey pair for more information.
It is highly recommended that you use environment variable expansion to store your access key pair. For example, you can set the following environment variables:
  • ALICLOUD_ACCESS_KEY - The access key ID.
  • ALICLOUD_SECRET_KEY - The access key secret.
And then use them in the configuration as follows:
access_key: ${ALICLOUD_ACCESS_KEY}
secret_key: ${ALICLOUD_SECRET_KEY}

Example Queries #

Find all ECS instances in a region #

select
  instance_id,
  os_name,
  region_id,
  start_time,
  tags
from
  alicloud_ecs_instances
where
  region_id = 'eu-west-1';
+------------------------+--------------------------------------+-----------+-------------------+---------------+
| instance_id            | os_name                              | region_id | start_time        | tags          |
|------------------------+--------------------------------------+-----------+-------------------+---------------|
| i-xxxxxxxxxxxxxxxxxxxx | Alibaba Cloud Linux  3.2104 LTS 64位 | eu-west-1 | 2023-01-17T14:40Z | {"Tag": null} |
+------------------------+--------------------------------------+-----------+-------------------+---------------+

Query past bills #

select
  product_name,
  item,
  pip_code,
  currency,
  adjust_amount
from
  alicloud_bss_bill_overview;
+------------------------+----------------+----------+----------+---------------+
| product_name           | item           | pip_code | currency | adjust_amount |
|------------------------+----------------+----------+----------+---------------|
| Object Storage Service | PayAsYouGoBill | oss      | USD      | 0.0           |
+------------------------+----------------+----------+----------+---------------+

Query bucket stats #

select
  account_id,
  bucket_name,
  object_count,
  storage
from
  alicloud_oss_bucket_stats;
+------------+-------------+--------------+---------+
| account_id | bucket_name | object_count | storage |
|------------+-------------+--------------+---------|
| test       | cq-test     | 2            | 29665   |
+------------+-------------+--------------+---------+


Licenses #

The following tools / packages are used in this plugin:
NameLicense
github.com/adrg/xdgMIT
github.com/aliyun/alibaba-cloud-sdk-goApache-2.0
github.com/aliyun/aliyun-oss-go-sdk/ossMIT
github.com/apache/arrow/go/v13Apache-2.0
github.com/apache/arrow-go/v18Apache-2.0
github.com/apapsch/go-jsonmerge/v2MIT
github.com/aws/aws-sdk-go-v2Apache-2.0
github.com/aws/aws-sdk-go-v2/configApache-2.0
github.com/aws/aws-sdk-go-v2/credentialsApache-2.0
github.com/aws/aws-sdk-go-v2/feature/ec2/imdsApache-2.0
github.com/aws/aws-sdk-go-v2/internal/configsourcesApache-2.0
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/iniApache-2.0
github.com/aws/aws-sdk-go-v2/internal/sync/singleflightBSD-3-Clause
github.com/aws/aws-sdk-go-v2/service/internal/accept-encodingApache-2.0
github.com/aws/aws-sdk-go-v2/service/internal/presigned-urlApache-2.0
github.com/aws/aws-sdk-go-v2/service/licensemanagerApache-2.0
github.com/aws/aws-sdk-go-v2/service/marketplacemeteringApache-2.0
github.com/aws/aws-sdk-go-v2/service/ssoApache-2.0
github.com/aws/aws-sdk-go-v2/service/ssooidcApache-2.0
github.com/aws/aws-sdk-go-v2/service/stsApache-2.0
github.com/aws/smithy-goApache-2.0
github.com/aws/smithy-go/internal/sync/singleflightBSD-3-Clause
github.com/bahlo/generic-list-goBSD-3-Clause
github.com/buger/jsonparserMIT
github.com/cenkalti/backoff/v4MIT
github.com/cloudquery/cloudquery-api-goMPL-2.0
github.com/cloudquery/plugin-pb-goMPL-2.0
github.com/cloudquery/plugin-sdk/v2/internal/globMIT
github.com/cloudquery/plugin-sdk/v2/schemaMIT
github.com/cloudquery/plugin-sdk/v2/typesMPL-2.0
github.com/cloudquery/plugin-sdk/v4MPL-2.0
github.com/cloudquery/plugin-sdk/v4/globMIT
github.com/cloudquery/plugin-sdk/v4/scalarMIT
github.com/davecgh/go-spew/spewISC
github.com/ghodss/yamlMIT
github.com/go-logr/logrApache-2.0
github.com/go-logr/stdrApache-2.0
github.com/goccy/go-jsonMIT
github.com/golang/mock/gomockApache-2.0
github.com/google/flatbuffers/goApache-2.0
github.com/google/uuidBSD-3-Clause
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptorsApache-2.0
github.com/grpc-ecosystem/grpc-gateway/v2BSD-3-Clause
github.com/hashicorp/go-cleanhttpMPL-2.0
github.com/hashicorp/go-retryablehttpMPL-2.0
github.com/invopop/jsonschemaMIT
github.com/jmespath/go-jmespathApache-2.0
github.com/json-iterator/goMIT
github.com/klauspost/compressApache-2.0
github.com/klauspost/compress/internal/snaprefBSD-3-Clause
github.com/klauspost/compress/zstd/internal/xxhashMIT
github.com/mailru/easyjsonMIT
github.com/mattn/go-colorableMIT
github.com/mattn/go-isattyMIT
github.com/modern-go/concurrentApache-2.0
github.com/modern-go/reflect2Apache-2.0
github.com/oapi-codegen/runtimeApache-2.0
github.com/opentracing/opentracing-goApache-2.0
github.com/pierrec/lz4/v4BSD-3-Clause
github.com/pkg/errorsBSD-2-Clause
github.com/pmezard/go-difflib/difflibBSD-3-Clause
github.com/rs/zerologMIT
github.com/samber/loMIT
github.com/santhosh-tekuri/jsonschema/v6Apache-2.0
github.com/spf13/cobraApache-2.0
github.com/spf13/pflagBSD-3-Clause
github.com/stretchr/testifyMIT
github.com/thoas/go-funkMIT
github.com/wk8/go-ordered-map/v2Apache-2.0
github.com/zeebo/xxh3BSD-2-Clause
go.opentelemetry.io/otelApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttpApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttpApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptraceApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttpApache-2.0
go.opentelemetry.io/otel/logApache-2.0
go.opentelemetry.io/otel/metricApache-2.0
go.opentelemetry.io/otel/sdkApache-2.0
go.opentelemetry.io/otel/sdk/logApache-2.0
go.opentelemetry.io/otel/sdk/metricApache-2.0
go.opentelemetry.io/otel/traceApache-2.0
go.opentelemetry.io/proto/otlpApache-2.0
golang.org/x/expBSD-3-Clause
golang.org/x/netBSD-3-Clause
golang.org/x/syncBSD-3-Clause
golang.org/x/sysBSD-3-Clause
golang.org/x/textBSD-3-Clause
golang.org/x/time/rateBSD-3-Clause
golang.org/x/xerrorsBSD-3-Clause
google.golang.org/genproto/googleapis/api/httpbodyApache-2.0
google.golang.org/genproto/googleapis/rpc/statusApache-2.0
google.golang.org/grpcApache-2.0
google.golang.org/protobufBSD-3-Clause
gopkg.in/ini.v1Apache-2.0
gopkg.in/yaml.v2Apache-2.0
gopkg.in/yaml.v3MIT


Join our mailing list

Subscribe to our newsletter to make sure you don't miss any updates.

Legal

© 2024 CloudQuery, Inc. All rights reserved.

We use tracking cookies to understand how you use the product and help us improve it. Please accept cookies to help us improve. You can always opt out later via the link in the footer.