Join our Webinar: Best Security Management Practices for Multi-Cloud Infrastructure Sign up ❯
Back to source list
buildkite
Official
Premium

Buildkite

This plugin is in preview.

Sync from Buildkite to any destination

Publisher

cloudquery

Latest version

v1.0.0

Type

Source

Platforms
Date Published

Overview #

CloudQuery Buildkite plugin reads information from Buildkite and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).

Authentication #

The Buildkite plugin requires an API access token to authenticate. API Access tokens are scoped to a single organization, in order to sync data from multiple organizations at once, multiple tokens can be provided in the access_tokens configuration. You can generate the API Access Token by following the Buildkite documentation. This integration syncs data using Buildkite's REST API, GraphQL API access should not be required.
Once you have an access token, you can export it as an environment variable:
export BUILDKITE_ACCESS_TOKEN=<your_api_token>
(MacOS / Linux) or
set BUILDKITE_ACCESS_TOKEN=<your_api_token>
(Windows)

Example Config #

This example syncs from Buildkite to a Postgres destination. The (top level) source spec section is described in the Source Spec Reference.
kind: source
# Common source-plugin configuration
spec:
  name: buildkite
  path: cloudquery/buildkite
  version: "v1.0.0"
  tables: ["*"]
  destinations: ["postgresql"]
  # buildkite specific configuration
  spec:
    access_tokens: ["${BUILDKITE_ACCESS_TOKEN}"]

    # optional parameters
    # base_url: "https://api.buildkite.com/"
    # concurrency: 100
    # scheduler: "dfs"
    # requests_per_minute: 160

Configuration #

This is the (nested) spec used by the Buildkite source plugin.
  • access_tokens ([]string) (required)
    Buildkite API Acces token. Can be obtained by generating an access token.
  • base_url (string) (optional)
    The base URL to use for the Buildkite API. Defaults to https://api.buildkite.com/
  • concurrency (integer) (optional) (default: 100)
    A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.
  • scheduler (string, optional, default: dfs)
    The scheduler to use when determining the priority of resources to sync. Supported values are dfs (depth-first search), round-robin, shuffle and shuffle-queue.
    For more information about this, see performance tuning.
  • requests_per_minute (integer) (optional)
    The maximum number of requests per minute that can be made to the Buildkite API. BuildKite API rate limit default is 200 requests per minute for each organization. CloudQuery by default uses 80% of the rate limit, to allow for other requests to be made to the API.
  • table_options (map) (default: null)
    A set of options to override the defaults for certain tables. The full list of supported options are documented under the Table Options section of each table
    The format of the table_options object is as follows:
    table_options:
      <table_name>:
        <input_object>
    table_options:
      buildkite_agents:
        include_access_token: true # default: false
      buildkite_pipelines:
        include_env_vars: true # default: false
      buildkite_builds:
        ListBuilds:
          - include_env_vars:  true # default: false
            created_from: "2 days ago"
            created_to: "1 day ago"
          - include_env_vars: false
            branch: ["dev", "feature1"]
Time in table options
Our Time type allows for defining timestamps in both absolute and relative formats.
Absolute timestamps must be RFC3339 formatted. Example: 2024-01-01T12:00:00+00:00.
Relative timestamps can take this format:
  • now
  • x seconds [ago|from now]
  • x minutes [ago|from now]
  • x hours [ago|from now]
  • x days [ago|from now]
Here are a few valid values for the until field:
  • until: now
  • until: 2 days ago
  • until: 10 months 3 days 4h20m from now
  • until: 2024-01-01T12:00:00+00:00


Licenses #

The following tools / packages are used in this plugin:
NameLicense
github.com/adrg/xdgMIT
github.com/apache/arrow-go/v18Apache-2.0
github.com/apache/arrow/go/v13Apache-2.0
github.com/apapsch/go-jsonmerge/v2MIT
github.com/aws/aws-sdk-go-v2Apache-2.0
github.com/aws/aws-sdk-go-v2/configApache-2.0
github.com/aws/aws-sdk-go-v2/credentialsApache-2.0
github.com/aws/aws-sdk-go-v2/feature/ec2/imdsApache-2.0
github.com/aws/aws-sdk-go-v2/internal/configsourcesApache-2.0
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/iniApache-2.0
github.com/aws/aws-sdk-go-v2/internal/sync/singleflightBSD-3-Clause
github.com/aws/aws-sdk-go-v2/service/internal/accept-encodingApache-2.0
github.com/aws/aws-sdk-go-v2/service/internal/presigned-urlApache-2.0
github.com/aws/aws-sdk-go-v2/service/licensemanagerApache-2.0
github.com/aws/aws-sdk-go-v2/service/marketplacemeteringApache-2.0
github.com/aws/aws-sdk-go-v2/service/ssoApache-2.0
github.com/aws/aws-sdk-go-v2/service/ssooidcApache-2.0
github.com/aws/aws-sdk-go-v2/service/stsApache-2.0
github.com/aws/smithy-goApache-2.0
github.com/aws/smithy-go/internal/sync/singleflightBSD-3-Clause
github.com/bahlo/generic-list-goBSD-3-Clause
github.com/buger/jsonparserMIT
github.com/buildkite/go-buildkite/v4BSD-2-Clause
github.com/cenkalti/backoffMIT
github.com/cenkalti/backoff/v4MIT
github.com/cloudquery/cloudquery-api-goMPL-2.0
github.com/cloudquery/codegen/jsonschema/docsMPL-2.0
github.com/cloudquery/plugin-pb-goMPL-2.0
github.com/cloudquery/plugin-sdk/v2/internal/globMIT
github.com/cloudquery/plugin-sdk/v2/schemaMIT
github.com/cloudquery/plugin-sdk/v2/typesMPL-2.0
github.com/cloudquery/plugin-sdk/v4MPL-2.0
github.com/cloudquery/plugin-sdk/v4/globMIT
github.com/cloudquery/plugin-sdk/v4/scalarMIT
github.com/davecgh/go-spew/spewISC
github.com/ghodss/yamlMIT
github.com/go-logr/logrApache-2.0
github.com/go-logr/stdrApache-2.0
github.com/goccy/go-jsonMIT
github.com/golang/mock/gomockApache-2.0
github.com/google/flatbuffers/goApache-2.0
github.com/google/go-querystring/queryBSD-3-Clause
github.com/google/uuidBSD-3-Clause
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptorsApache-2.0
github.com/grpc-ecosystem/grpc-gateway/v2BSD-3-Clause
github.com/hashicorp/go-cleanhttpMPL-2.0
github.com/hashicorp/go-retryablehttpMPL-2.0
github.com/invopop/jsonschemaMIT
github.com/klauspost/compressApache-2.0
github.com/klauspost/compress/internal/snaprefBSD-3-Clause
github.com/klauspost/compress/zstd/internal/xxhashMIT
github.com/mailru/easyjsonMIT
github.com/mattn/go-colorableMIT
github.com/mattn/go-isattyMIT
github.com/mitchellh/hashstructure/v2MIT
github.com/oapi-codegen/runtimeApache-2.0
github.com/pierrec/lz4/v4BSD-3-Clause
github.com/pmezard/go-difflib/difflibBSD-3-Clause
github.com/rs/zerologMIT
github.com/samber/loMIT
github.com/santhosh-tekuri/jsonschema/v6Apache-2.0
github.com/spf13/cobraApache-2.0
github.com/spf13/pflagBSD-3-Clause
github.com/stretchr/testifyMIT
github.com/thoas/go-funkMIT
github.com/wk8/go-ordered-map/v2Apache-2.0
github.com/zeebo/xxh3BSD-2-Clause
go.opentelemetry.io/auto/sdkApache-2.0
go.opentelemetry.io/otelApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttpApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttpApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptraceApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttpApache-2.0
go.opentelemetry.io/otel/logApache-2.0
go.opentelemetry.io/otel/metricApache-2.0
go.opentelemetry.io/otel/sdkApache-2.0
go.opentelemetry.io/otel/sdk/logApache-2.0
go.opentelemetry.io/otel/sdk/metricApache-2.0
go.opentelemetry.io/otel/traceApache-2.0
go.opentelemetry.io/proto/otlpApache-2.0
golang.org/x/expBSD-3-Clause
golang.org/x/netBSD-3-Clause
golang.org/x/syncBSD-3-Clause
golang.org/x/sysBSD-3-Clause
golang.org/x/textBSD-3-Clause
golang.org/x/time/rateBSD-3-Clause
golang.org/x/xerrorsBSD-3-Clause
google.golang.org/genproto/googleapis/api/httpbodyApache-2.0
google.golang.org/genproto/googleapis/rpc/statusApache-2.0
google.golang.org/grpcApache-2.0
google.golang.org/protobufBSD-3-Clause
gopkg.in/yaml.v2Apache-2.0
gopkg.in/yaml.v3MIT


Join our mailing list

Subscribe to our newsletter to make sure you don't miss any updates.

Legal

© 2025 CloudQuery, Inc. All rights reserved.

We use tracking cookies to understand how you use the product and help us improve it. Please accept cookies to help us improve. You can always opt out later via the link in the footer.