Back to source list
Official
Premium
Alibaba Cloud
The Alibaba Cloud source plugin for CloudQuery extracts configuration from the Alibaba Cloud API and loads it into any supported CloudQuery destination
Publisher
cloudquery
Latest version
v5.9.4
Type
Source
Platforms
Date Published
Price per 1M rows
Starting from $17
monthly free quota
1M rows
Overview #
Alibaba Cloud Source Plugin
The Alibaba Cloud source plugin for CloudQuery extracts configuration from the Alibaba Cloud (阿里云) API and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).
Configuration #
The following configuration syncs from Alibaba Cloud to a Postgres destination. The (top level) source spec section is described in the Source Spec Reference. The config for the
postgresql destination is not shown here. See our Quickstart if you need help setting up the destination.kind: source
spec:
name: "alicloud"
path: "cloudquery/alicloud"
registry: "cloudquery"
version: "v5.9.4"
tables: ["*"]
destinations: ["postgresql"]
# Learn more about the configuration options at https://cql.ink/alicloud_source
spec:
accounts:
- name: my_account
regions:
- cn-hangzhou
- cn-beijing
- eu-west-1
- us-west-1
# ...
access_key: ${ALICLOUD_ACCESS_KEY}
secret_key: ${ALICLOUD_SECRET_KEY}
# Optional parameters
# concurrency: 50000
# bill_history_months: 12Alibaba Spec #
concurrency(integer, optional, default:50000):A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.scheduler(string, optional, default:dfs):The scheduler to use when determining the priority of resources to sync. Supported values aredfs(depth-first search),round-robin,shuffleandshuffle-queue.For more information about this, see performance tuning.accounts(array[object], required):A list of accounts to sync. Every account must have a unique name, and must specify at least one region. Theaccess_keyandsecret_keyare required and can be specified as environment variables, as shown in the example above.name(string, required): A unique name for the account.regions(array[string], required): A list of regions to sync. For example,["cn-hangzhou", "cn-beijing"].access_key(string, required): A valid access key for the accountsecret_key(string, required): A valid secret key for the account, corresponding to the access key
bill_history_months(integer, optional):The number of months of billing history to fetch for thealicloud_bss_billandalicloud_bss_bill_overviewtables. Defaults to 12.
See the Alibaba documentation for how to obtain an AccessKey pair.
Authentication #
Authentication is done via access keys. See Obtain an AccessKey pair for more information.
It is highly recommended that you use environment variable expansion to store your access key pair. For example, you can set the following environment variables:
ALICLOUD_ACCESS_KEY- The access key ID.ALICLOUD_SECRET_KEY- The access key secret.
And then use them in the configuration as follows:
access_key: ${ALICLOUD_ACCESS_KEY}
secret_key: ${ALICLOUD_SECRET_KEY}Example Queries #
Find all ECS instances in a region #
select
instance_id,
os_name,
region_id,
start_time,
tags
from
alicloud_ecs_instances
where
region_id = 'eu-west-1';+------------------------+--------------------------------------+-----------+-------------------+---------------+
| instance_id | os_name | region_id | start_time | tags |
|------------------------+--------------------------------------+-----------+-------------------+---------------|
| i-xxxxxxxxxxxxxxxxxxxx | Alibaba Cloud Linux 3.2104 LTS 64位 | eu-west-1 | 2023-01-17T14:40Z | {"Tag": null} |
+------------------------+--------------------------------------+-----------+-------------------+---------------+Query past bills #
select
product_name,
item,
pip_code,
currency,
adjust_amount
from
alicloud_bss_bill_overview;+------------------------+----------------+----------+----------+---------------+
| product_name | item | pip_code | currency | adjust_amount |
|------------------------+----------------+----------+----------+---------------|
| Object Storage Service | PayAsYouGoBill | oss | USD | 0.0 |
+------------------------+----------------+----------+----------+---------------+Query bucket stats #
select
account_id,
bucket_name,
object_count,
storage
from
alicloud_oss_bucket_stats;+------------+-------------+--------------+---------+
| account_id | bucket_name | object_count | storage |
|------------+-------------+--------------+---------|
| test | cq-test | 2 | 29665 |
+------------+-------------+--------------+---------+Licenses #
The following tools / packages are used in this plugin:
| Name | License |
|---|---|
| github.com/adrg/xdg | MIT |
| github.com/aliyun/alibaba-cloud-sdk-go | Apache-2.0 |
| github.com/aliyun/aliyun-oss-go-sdk/oss | MIT |
| github.com/apache/arrow/go/v13 | Apache-2.0 |
| github.com/apache/arrow-go/v18 | Apache-2.0 |
| github.com/apapsch/go-jsonmerge/v2 | MIT |
| github.com/aws/aws-sdk-go-v2 | Apache-2.0 |
| github.com/aws/aws-sdk-go-v2/config | Apache-2.0 |
| github.com/aws/aws-sdk-go-v2/credentials | Apache-2.0 |
| github.com/aws/aws-sdk-go-v2/feature/ec2/imds | Apache-2.0 |
| github.com/aws/aws-sdk-go-v2/internal/configsources | Apache-2.0 |
| github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 | Apache-2.0 |
| github.com/aws/aws-sdk-go-v2/internal/ini | Apache-2.0 |
| github.com/aws/aws-sdk-go-v2/internal/sync/singleflight | BSD-3-Clause |
| github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding | Apache-2.0 |
| github.com/aws/aws-sdk-go-v2/service/internal/presigned-url | Apache-2.0 |
| github.com/aws/aws-sdk-go-v2/service/licensemanager | Apache-2.0 |
| github.com/aws/aws-sdk-go-v2/service/marketplacemetering | Apache-2.0 |
| github.com/aws/aws-sdk-go-v2/service/sso | Apache-2.0 |
| github.com/aws/aws-sdk-go-v2/service/ssooidc | Apache-2.0 |
| github.com/aws/aws-sdk-go-v2/service/sts | Apache-2.0 |
| github.com/aws/smithy-go | Apache-2.0 |
| github.com/aws/smithy-go/internal/sync/singleflight | BSD-3-Clause |
| github.com/bahlo/generic-list-go | BSD-3-Clause |
| github.com/buger/jsonparser | MIT |
| github.com/cenkalti/backoff/v4 | MIT |
| github.com/cloudquery/cloudquery-api-go | MPL-2.0 |
| github.com/cloudquery/plugin-pb-go | MPL-2.0 |
| github.com/cloudquery/plugin-sdk/v2/internal/glob | MIT |
| github.com/cloudquery/plugin-sdk/v2/schema | MIT |
| github.com/cloudquery/plugin-sdk/v2/types | MPL-2.0 |
| github.com/cloudquery/plugin-sdk/v4 | MPL-2.0 |
| github.com/cloudquery/plugin-sdk/v4/glob | MIT |
| github.com/cloudquery/plugin-sdk/v4/scalar | MIT |
| github.com/davecgh/go-spew/spew | ISC |
| github.com/ghodss/yaml | MIT |
| github.com/go-logr/logr | Apache-2.0 |
| github.com/go-logr/stdr | Apache-2.0 |
| github.com/goccy/go-json | MIT |
| github.com/golang/mock/gomock | Apache-2.0 |
| github.com/google/flatbuffers/go | Apache-2.0 |
| github.com/google/uuid | BSD-3-Clause |
| github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors | Apache-2.0 |
| github.com/grpc-ecosystem/grpc-gateway/v2 | BSD-3-Clause |
| github.com/hashicorp/go-cleanhttp | MPL-2.0 |
| github.com/hashicorp/go-retryablehttp | MPL-2.0 |
| github.com/invopop/jsonschema | MIT |
| github.com/jmespath/go-jmespath | Apache-2.0 |
| github.com/json-iterator/go | MIT |
| github.com/klauspost/compress | Apache-2.0 |
| github.com/klauspost/compress/internal/snapref | BSD-3-Clause |
| github.com/klauspost/compress/zstd/internal/xxhash | MIT |
| github.com/mailru/easyjson | MIT |
| github.com/mattn/go-colorable | MIT |
| github.com/mattn/go-isatty | MIT |
| github.com/modern-go/concurrent | Apache-2.0 |
| github.com/modern-go/reflect2 | Apache-2.0 |
| github.com/oapi-codegen/runtime | Apache-2.0 |
| github.com/opentracing/opentracing-go | Apache-2.0 |
| github.com/pierrec/lz4/v4 | BSD-3-Clause |
| github.com/pkg/errors | BSD-2-Clause |
| github.com/pmezard/go-difflib/difflib | BSD-3-Clause |
| github.com/rs/zerolog | MIT |
| github.com/samber/lo | MIT |
| github.com/santhosh-tekuri/jsonschema/v6 | Apache-2.0 |
| github.com/spf13/cobra | Apache-2.0 |
| github.com/spf13/pflag | BSD-3-Clause |
| github.com/stretchr/testify | MIT |
| github.com/thoas/go-funk | MIT |
| github.com/wk8/go-ordered-map/v2 | Apache-2.0 |
| github.com/zeebo/xxh3 | BSD-2-Clause |
| go.opentelemetry.io/otel | Apache-2.0 |
| go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp | Apache-2.0 |
| go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp | Apache-2.0 |
| go.opentelemetry.io/otel/exporters/otlp/otlptrace | Apache-2.0 |
| go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp | Apache-2.0 |
| go.opentelemetry.io/otel/log | Apache-2.0 |
| go.opentelemetry.io/otel/metric | Apache-2.0 |
| go.opentelemetry.io/otel/sdk | Apache-2.0 |
| go.opentelemetry.io/otel/sdk/log | Apache-2.0 |
| go.opentelemetry.io/otel/sdk/metric | Apache-2.0 |
| go.opentelemetry.io/otel/trace | Apache-2.0 |
| go.opentelemetry.io/proto/otlp | Apache-2.0 |
| golang.org/x/exp | BSD-3-Clause |
| golang.org/x/net | BSD-3-Clause |
| golang.org/x/sync | BSD-3-Clause |
| golang.org/x/sys | BSD-3-Clause |
| golang.org/x/text | BSD-3-Clause |
| golang.org/x/time/rate | BSD-3-Clause |
| golang.org/x/xerrors | BSD-3-Clause |
| google.golang.org/genproto/googleapis/api/httpbody | Apache-2.0 |
| google.golang.org/genproto/googleapis/rpc/status | Apache-2.0 |
| google.golang.org/grpc | Apache-2.0 |
| google.golang.org/protobuf | BSD-3-Clause |
| gopkg.in/ini.v1 | Apache-2.0 |
| gopkg.in/yaml.v2 | Apache-2.0 |
| gopkg.in/yaml.v3 | MIT |