Official

Premium

Datadog source integration documentation

The CloudQuery Datadog plugin extracts your Datadog information and loads it into any supported CloudQuery destination

Publisher

cloudquery

Latest version

v5.16.1

Type

Source

Platforms

Date Published

Documentation Tables Changelog Destinations

Overview Configuration Licenses

Overview #

Datadog Source Plugin

The CloudQuery Datadog plugin extracts your Datadog information and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).

Authentication #

CloudQuery requires only DD_CLIENT_API_KEY and DD_CLIENT_APP_KEY. Follow this guide for how to create an API key and app key for CloudQuery.

CloudQuery requires only read permissions (we will never make any changes to your Datadog account), so, following the principle of the least privilege, it's recommended to grant it read-only permissions.

Query Examples #

Find all not verified users #

SELECT 
    "attributes" ->> 'name' AS username
FROM
    datadog_users
WHERE
    ("attributes" ->> 'verified')::boolean is distinct FROM TRUE

Configuration #

CloudQuery Datadog Source Plugin Configuration Reference

Example #

This example connects a single Datadog account to a Postgres destination. The (top level) source spec section is described in the Source Spec Reference.

kind: source
spec:
  # Source spec section
  name: "datadog"
  path: "cloudquery/datadog"
  registry: "cloudquery"
  version: "v5.16.1"
  tables: ["*"]
  destinations: ["postgresql"]

  # Learn more about the configuration options at https://cql.ink/datadog_source
  spec:
    # required
    accounts:
      - # required
        name: example_account
        # required
        api_key: ${DATADOG_EXAMPLE_ACCOUNT_API_KEY}
        # required
        app_key: ${DATADOG_EXAMPLE_ACCOUNT_APP_KEY}
    # Optional parameters
    # max_retries: 5
    # site: datadoghq.eu

Datadog Spec #

This is the (nested) spec used by the Datadog source plugin.

accounts ([]account, required)
Specify which accounts to sync data from.
max_retries (integer, optional, default: 5)
The maximum number of retries to attempt for failed API requests.
site (string, optional, default: "")
The Datadog site to connect to. This is usually one of datadoghq.com or datadoghq.eu - see site documentation for more information.
concurrency (integer, optional, default: 10000)
A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.
scheduler (string, optional, default: dfs)
The scheduler to use when determining the priority of resources to sync. Supported values are dfs (depth-first search), round-robin, shuffle and shuffle-queue.
For more information about this, see performance tuning.

account #

This is used to specify one or more accounts to extract information from. Note that it should be an array of objects, each with the following fields:

name (string, required)
Account name.
api_key (string, required if not using access_token)
Datadog API key.
app_key (string, required if not using access_token)
Datadog App key.
access_token (string)
A valid OAuth access token to use, instead of api_key and app_key.

Licenses #

The following tools / packages are used in this plugin:

Name	License
github.com/DataDog/datadog-api-client-go/v2	Apache-2.0
github.com/DataDog/zstd	BSD-3-Clause
github.com/adrg/xdg	MIT
github.com/apache/arrow-go/v18	Apache-2.0
github.com/apache/arrow/go/v13	Apache-2.0
github.com/apapsch/go-jsonmerge/v2	MIT
github.com/aws/aws-sdk-go-v2	Apache-2.0
github.com/aws/aws-sdk-go-v2/config	Apache-2.0
github.com/aws/aws-sdk-go-v2/credentials	Apache-2.0
github.com/aws/aws-sdk-go-v2/feature/ec2/imds	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/configsources	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/ini	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/sync/singleflight	BSD-3-Clause
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/licensemanager	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/marketplacemetering	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/sso	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/ssooidc	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/sts	Apache-2.0
github.com/aws/smithy-go	Apache-2.0
github.com/aws/smithy-go/internal/sync/singleflight	BSD-3-Clause
github.com/bahlo/generic-list-go	BSD-3-Clause
github.com/buger/jsonparser	MIT
github.com/cenkalti/backoff/v5	MIT
github.com/cloudquery/cloudquery-api-go	MPL-2.0
github.com/cloudquery/codegen/jsonschema/docs	MPL-2.0
github.com/cloudquery/plugin-pb-go	MPL-2.0
github.com/cloudquery/plugin-sdk/v2/internal/glob	MIT
github.com/cloudquery/plugin-sdk/v2/schema	MIT
github.com/cloudquery/plugin-sdk/v2/types	MPL-2.0
github.com/cloudquery/plugin-sdk/v4	MPL-2.0
github.com/cloudquery/plugin-sdk/v4/glob	MIT
github.com/cloudquery/plugin-sdk/v4/scalar	MIT
github.com/davecgh/go-spew/spew	ISC
github.com/ghodss/yaml	MIT
github.com/go-logr/logr	Apache-2.0
github.com/go-logr/stdr	Apache-2.0
github.com/goccy/go-json	MIT
github.com/golang/mock/gomock	Apache-2.0
github.com/google/flatbuffers/go	Apache-2.0
github.com/google/uuid	BSD-3-Clause
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors	Apache-2.0
github.com/grpc-ecosystem/grpc-gateway/v2	BSD-3-Clause
github.com/hashicorp/go-cleanhttp	MPL-2.0
github.com/hashicorp/go-retryablehttp	MPL-2.0
github.com/invopop/jsonschema	MIT
github.com/klauspost/compress	Apache-2.0
github.com/klauspost/compress/internal/snapref	BSD-3-Clause
github.com/klauspost/compress/zstd/internal/xxhash	MIT
github.com/mailru/easyjson	MIT
github.com/mattn/go-colorable	MIT
github.com/mattn/go-isatty	MIT
github.com/oapi-codegen/runtime	Apache-2.0
github.com/pierrec/lz4/v4	BSD-3-Clause
github.com/pmezard/go-difflib/difflib	BSD-3-Clause
github.com/rs/zerolog	MIT
github.com/samber/lo	MIT
github.com/santhosh-tekuri/jsonschema/v6	Apache-2.0
github.com/spf13/cobra	Apache-2.0
github.com/spf13/pflag	BSD-3-Clause
github.com/stretchr/testify	MIT
github.com/thoas/go-funk	MIT
github.com/wk8/go-ordered-map/v2	Apache-2.0
github.com/zeebo/xxh3	BSD-2-Clause
go.opentelemetry.io/auto/sdk	Apache-2.0
go.opentelemetry.io/otel	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp	Apache-2.0
go.opentelemetry.io/otel/log	Apache-2.0
go.opentelemetry.io/otel/metric	Apache-2.0
go.opentelemetry.io/otel/sdk	Apache-2.0
go.opentelemetry.io/otel/sdk/log	Apache-2.0
go.opentelemetry.io/otel/sdk/metric	Apache-2.0
go.opentelemetry.io/otel/trace	Apache-2.0
go.opentelemetry.io/proto/otlp	Apache-2.0
golang.org/x/exp	BSD-3-Clause
golang.org/x/net	BSD-3-Clause
golang.org/x/oauth2	BSD-3-Clause
golang.org/x/sync	BSD-3-Clause
golang.org/x/sys	BSD-3-Clause
golang.org/x/text	BSD-3-Clause
golang.org/x/xerrors	BSD-3-Clause
google.golang.org/genproto/googleapis/api/httpbody	Apache-2.0
google.golang.org/genproto/googleapis/rpc/status	Apache-2.0
google.golang.org/grpc	Apache-2.0
google.golang.org/protobuf	BSD-3-Clause
gopkg.in/yaml.v2	Apache-2.0
gopkg.in/yaml.v3	MIT

Loading plugin documentation