Official

Premium

Heroku source integration documentation

The CloudQuery Heroku plugin extracts your Heroku data and loads it into any supported CloudQuery destination

Publisher

cloudquery

Latest version

v5.12.22

Type

Source

Platforms

Date Published

Start Syncing

Documentation Tables Changelog Destinations

Overview Licenses

Overview #

The CloudQuery Heroku plugin extracts your Heroku data and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).

Example Configuration #

kind: source
spec:
  name: heroku
  path: cloudquery/heroku
  registry: cloudquery
  version: "v5.12.22"
  tables: ["*"]
  destinations: ["postgresql"]

  # Learn more about the configuration options at https://cql.ink/heroku_source
  spec:
    # required
    token: "${HEROKU_TOKEN}"

Authentication #

The CloudQuery Heroku plugin requires an OAuth token. After creating it, you will need to copy the token into your heroku.yml file.

The value can also be exported as an environment variable (e.g. HEROKU_TOKEN), and used like this:

   ...
   token: ${HEROKU_TOKEN}

Option 1: Generate a token with the Heroku CLI #

A token can be generated using the Heroku CLI.

Install the Heroku CLI: Follow the official instructions to install the Heroku CLI.
Generate an OAuth token: With the Heroku CLI installed, use your terminal to run:

heroku authorizations:create --short --description="CloudQuery token" --scope="read,identity"

(For additional options for this command, such as expiry, see the Heroku CLI commands documentation)

Option 2: Generate a token with the Heroku API #

It is also possible to manage OAuth tokens directly through the Heroku API. For more information, see the Heroku documentation.

A Note about OAuth Scopes #

CloudQuery needs to be authenticated with your Heroku account's token in order to fetch information about your Heroku resources. CloudQuery requires only read permissions (we will never make any changes to your Heroku account or apps). Following the principle of least privilege, it is recommended to grant it read-only permissions. The --scope="read,identity" parameter suggested above achieves this.

However, certain Heroku resources require a global scope, even for reading. At the time of writing, these resources are:

app_webhook_deliveries
app_webhook_events
app_webhooks
credits
invoices
keys
oauth_authorizations
oauth_clients
permission_entities
team_features
team_invitations
team_invoices
team_members
team_spaces

If you are interested in fetching any of these resources, a global scope will be necessary. See the Heroku documentation for more information about OAuth scopes on Heroku.

Configuration Reference #

This is the (nested) spec used by the Heroku source plugin.

token (string) (required)

Heroku API token. See the Authentication section on how to generate it.

concurrency (integer) (optional) (default: 10000)
A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.
scheduler (string) (optional) (default: dfs) The scheduler to use when determining the priority of resources to sync. Supported values are dfs (depth-first search), round-robin, shuffle and shuffle-queue.
For more information about this, see performance tuning.

Licenses #

The following tools / packages are used in this plugin:

Name	License
cloud.google.com/go/auth	Apache-2.0
cloud.google.com/go/auth/oauth2adapt	Apache-2.0
cloud.google.com/go/compute/metadata	Apache-2.0
github.com/adrg/xdg	MIT
github.com/apache/arrow-go/v18	Apache-2.0
github.com/apache/arrow/go/v13	Apache-2.0
github.com/apapsch/go-jsonmerge/v2	MIT
github.com/aws/aws-sdk-go-v2	Apache-2.0
github.com/aws/aws-sdk-go-v2/config	Apache-2.0
github.com/aws/aws-sdk-go-v2/credentials	Apache-2.0
github.com/aws/aws-sdk-go-v2/feature/ec2/imds	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/configsources	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/ini	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/sync/singleflight	BSD-3-Clause
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/licensemanager	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/marketplacemetering	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/sso	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/ssooidc	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/sts	Apache-2.0
github.com/aws/smithy-go	Apache-2.0
github.com/aws/smithy-go/internal/sync/singleflight	BSD-3-Clause
github.com/bahlo/generic-list-go	BSD-3-Clause
github.com/buger/jsonparser	MIT
github.com/cenkalti/backoff	MIT
github.com/cenkalti/backoff/v4	MIT
github.com/cloudquery/cloudquery-api-go	MPL-2.0
github.com/cloudquery/plugin-pb-go	MPL-2.0
github.com/cloudquery/plugin-sdk/v2/internal/glob	MIT
github.com/cloudquery/plugin-sdk/v2/schema	MIT
github.com/cloudquery/plugin-sdk/v2/types	MPL-2.0
github.com/cloudquery/plugin-sdk/v4	MPL-2.0
github.com/cloudquery/plugin-sdk/v4/glob	MIT
github.com/cloudquery/plugin-sdk/v4/scalar	MIT
github.com/davecgh/go-spew/spew	ISC
github.com/ghodss/yaml	MIT
github.com/go-logr/logr	Apache-2.0
github.com/go-logr/stdr	Apache-2.0
github.com/goccy/go-json	MIT
github.com/google/flatbuffers/go	Apache-2.0
github.com/google/go-querystring/query	BSD-3-Clause
github.com/google/s2a-go	Apache-2.0
github.com/google/uuid	BSD-3-Clause
github.com/googleapis/enterprise-certificate-proxy/client	Apache-2.0
github.com/googleapis/gax-go/v2	BSD-3-Clause
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors	Apache-2.0
github.com/grpc-ecosystem/grpc-gateway/v2	BSD-3-Clause
github.com/hashicorp/go-cleanhttp	MPL-2.0
github.com/hashicorp/go-retryablehttp	MPL-2.0
github.com/heroku/heroku-go/v5	MIT
github.com/invopop/jsonschema	MIT
github.com/klauspost/compress	Apache-2.0
github.com/klauspost/compress/internal/snapref	BSD-3-Clause
github.com/klauspost/compress/zstd/internal/xxhash	MIT
github.com/mailru/easyjson	MIT
github.com/mattn/go-colorable	MIT
github.com/mattn/go-isatty	MIT
github.com/oapi-codegen/runtime	Apache-2.0
github.com/pborman/uuid	BSD-3-Clause
github.com/pierrec/lz4/v4	BSD-3-Clause
github.com/pkg/errors	BSD-2-Clause
github.com/pmezard/go-difflib/difflib	BSD-3-Clause
github.com/rs/zerolog	MIT
github.com/samber/lo	MIT
github.com/santhosh-tekuri/jsonschema/v6	Apache-2.0
github.com/spf13/cobra	Apache-2.0
github.com/spf13/pflag	BSD-3-Clause
github.com/stretchr/testify	MIT
github.com/thoas/go-funk	MIT
github.com/wk8/go-ordered-map/v2	Apache-2.0
github.com/zeebo/xxh3	BSD-2-Clause
go.opentelemetry.io/auto/sdk	Apache-2.0
go.opentelemetry.io/otel	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp	Apache-2.0
go.opentelemetry.io/otel/log	Apache-2.0
go.opentelemetry.io/otel/metric	Apache-2.0
go.opentelemetry.io/otel/sdk	Apache-2.0
go.opentelemetry.io/otel/sdk/log	Apache-2.0
go.opentelemetry.io/otel/sdk/metric	Apache-2.0
go.opentelemetry.io/otel/trace	Apache-2.0
go.opentelemetry.io/proto/otlp	Apache-2.0
golang.org/x/crypto	BSD-3-Clause
golang.org/x/exp	BSD-3-Clause
golang.org/x/net	BSD-3-Clause
golang.org/x/oauth2	BSD-3-Clause
golang.org/x/sync	BSD-3-Clause
golang.org/x/sys	BSD-3-Clause
golang.org/x/text	BSD-3-Clause
golang.org/x/xerrors	BSD-3-Clause
google.golang.org/api	BSD-3-Clause
google.golang.org/api/internal/third_party/uritemplates	BSD-3-Clause
google.golang.org/genproto/googleapis/api/httpbody	Apache-2.0
google.golang.org/genproto/googleapis/rpc	Apache-2.0
google.golang.org/grpc	Apache-2.0
google.golang.org/protobuf	BSD-3-Clause
gopkg.in/yaml.v2	Apache-2.0
gopkg.in/yaml.v3	MIT

Loading plugin documentation

Overview #

The CloudQuery Heroku plugin extracts your Heroku data and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).

Example Configuration #

kind: source
spec:
  name: heroku
  path: cloudquery/heroku
  registry: cloudquery
  version: "v5.12.22"
  tables: ["*"]
  destinations: ["postgresql"]

  # Learn more about the configuration options at https://cql.ink/heroku_source
  spec:
    # required
    token: "${HEROKU_TOKEN}"

Authentication #

The CloudQuery Heroku plugin requires an OAuth token. After creating it, you will need to copy the token into your heroku.yml file.

The value can also be exported as an environment variable (e.g. HEROKU_TOKEN), and used like this:

   ...
   token: ${HEROKU_TOKEN}

Option 1: Generate a token with the Heroku CLI #

A token can be generated using the Heroku CLI.

Install the Heroku CLI: Follow the official instructions to install the Heroku CLI.
Generate an OAuth token: With the Heroku CLI installed, use your terminal to run:

heroku authorizations:create --short --description="CloudQuery token" --scope="read,identity"

(For additional options for this command, such as expiry, see the Heroku CLI commands documentation)

Option 2: Generate a token with the Heroku API #

It is also possible to manage OAuth tokens directly through the Heroku API. For more information, see the Heroku documentation.

A Note about OAuth Scopes #

However, certain Heroku resources require a global scope, even for reading. At the time of writing, these resources are:

app_webhook_deliveries
app_webhook_events
app_webhooks
credits
invoices
keys
oauth_authorizations
oauth_clients
permission_entities
team_features
team_invitations
team_invoices
team_members
team_spaces

If you are interested in fetching any of these resources, a global scope will be necessary. See the Heroku documentation for more information about OAuth scopes on Heroku.

Configuration Reference #

This is the (nested) spec used by the Heroku source plugin.

token (string) (required)

Heroku API token. See the Authentication section on how to generate it.

concurrency (integer) (optional) (default: 10000)
A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.
scheduler (string) (optional) (default: dfs) The scheduler to use when determining the priority of resources to sync. Supported values are dfs (depth-first search), round-robin, shuffle and shuffle-queue.
For more information about this, see performance tuning.

Name

License

cloud.google.com/go/auth

Apache-2.0

cloud.google.com/go/auth/oauth2adapt

Apache-2.0

cloud.google.com/go/compute/metadata

Apache-2.0

github.com/adrg/xdg

MIT

github.com/apache/arrow-go/v18

Apache-2.0

github.com/apache/arrow/go/v13

Apache-2.0

github.com/apapsch/go-jsonmerge/v2

MIT

github.com/aws/aws-sdk-go-v2

Apache-2.0

github.com/aws/aws-sdk-go-v2/config

Apache-2.0

github.com/aws/aws-sdk-go-v2/credentials

Apache-2.0

github.com/aws/aws-sdk-go-v2/feature/ec2/imds

Apache-2.0

github.com/aws/aws-sdk-go-v2/internal/configsources

Apache-2.0

github.com/aws/aws-sdk-go-v2/internal/endpoints/v2

Apache-2.0

github.com/aws/aws-sdk-go-v2/internal/ini

Apache-2.0

github.com/aws/aws-sdk-go-v2/internal/sync/singleflight

BSD-3-Clause

github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding

Apache-2.0

github.com/aws/aws-sdk-go-v2/service/internal/presigned-url

Apache-2.0

github.com/aws/aws-sdk-go-v2/service/licensemanager

Apache-2.0

github.com/aws/aws-sdk-go-v2/service/marketplacemetering

Apache-2.0

github.com/aws/aws-sdk-go-v2/service/sso

Apache-2.0

github.com/aws/aws-sdk-go-v2/service/ssooidc

Apache-2.0

github.com/aws/aws-sdk-go-v2/service/sts

Apache-2.0

github.com/aws/smithy-go

Apache-2.0

github.com/aws/smithy-go/internal/sync/singleflight

BSD-3-Clause

github.com/bahlo/generic-list-go

BSD-3-Clause

github.com/buger/jsonparser

MIT

github.com/cenkalti/backoff

MIT

github.com/cenkalti/backoff/v4

MIT

github.com/cloudquery/cloudquery-api-go

MPL-2.0

github.com/cloudquery/plugin-pb-go

MPL-2.0

github.com/cloudquery/plugin-sdk/v2/internal/glob

MIT

github.com/cloudquery/plugin-sdk/v2/schema

MIT

github.com/cloudquery/plugin-sdk/v2/types

MPL-2.0

github.com/cloudquery/plugin-sdk/v4

MPL-2.0

github.com/cloudquery/plugin-sdk/v4/glob

MIT

github.com/cloudquery/plugin-sdk/v4/scalar

MIT

github.com/davecgh/go-spew/spew

ISC

github.com/ghodss/yaml

MIT

github.com/go-logr/logr

Apache-2.0

github.com/go-logr/stdr

Apache-2.0

github.com/goccy/go-json

MIT

github.com/google/flatbuffers/go

Apache-2.0

github.com/google/go-querystring/query

BSD-3-Clause

github.com/google/s2a-go

Apache-2.0

github.com/google/uuid

BSD-3-Clause

github.com/googleapis/enterprise-certificate-proxy/client

Apache-2.0

github.com/googleapis/gax-go/v2

BSD-3-Clause

github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors

Apache-2.0

github.com/grpc-ecosystem/grpc-gateway/v2

BSD-3-Clause

github.com/hashicorp/go-cleanhttp

MPL-2.0

github.com/hashicorp/go-retryablehttp

MPL-2.0

github.com/heroku/heroku-go/v5

MIT

github.com/invopop/jsonschema

MIT

github.com/klauspost/compress

Apache-2.0

github.com/klauspost/compress/internal/snapref

BSD-3-Clause

github.com/klauspost/compress/zstd/internal/xxhash

MIT

github.com/mailru/easyjson

MIT

github.com/mattn/go-colorable

MIT

github.com/mattn/go-isatty

MIT

github.com/oapi-codegen/runtime

Apache-2.0

github.com/pborman/uuid

BSD-3-Clause

github.com/pierrec/lz4/v4

BSD-3-Clause

github.com/pkg/errors

BSD-2-Clause

github.com/pmezard/go-difflib/difflib

BSD-3-Clause

github.com/rs/zerolog

MIT

github.com/samber/lo

MIT

github.com/santhosh-tekuri/jsonschema/v6

Apache-2.0

github.com/spf13/cobra

Apache-2.0

github.com/spf13/pflag

BSD-3-Clause

github.com/stretchr/testify

MIT

github.com/thoas/go-funk

MIT

github.com/wk8/go-ordered-map/v2

Apache-2.0

github.com/zeebo/xxh3

BSD-2-Clause

go.opentelemetry.io/auto/sdk

Apache-2.0

go.opentelemetry.io/otel

Apache-2.0

go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp

Apache-2.0

go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp

Apache-2.0

go.opentelemetry.io/otel/exporters/otlp/otlptrace

Apache-2.0

go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp

Apache-2.0

go.opentelemetry.io/otel/log

Apache-2.0

go.opentelemetry.io/otel/metric

Apache-2.0

go.opentelemetry.io/otel/sdk

Apache-2.0

go.opentelemetry.io/otel/sdk/log

Apache-2.0

go.opentelemetry.io/otel/sdk/metric

Apache-2.0

go.opentelemetry.io/otel/trace

Apache-2.0

go.opentelemetry.io/proto/otlp

Apache-2.0

golang.org/x/crypto

BSD-3-Clause

golang.org/x/exp

BSD-3-Clause

golang.org/x/net

BSD-3-Clause

golang.org/x/oauth2

BSD-3-Clause

golang.org/x/sync

BSD-3-Clause

golang.org/x/sys

BSD-3-Clause

golang.org/x/text

BSD-3-Clause

golang.org/x/xerrors

BSD-3-Clause

google.golang.org/api

BSD-3-Clause

google.golang.org/api/internal/third_party/uritemplates

BSD-3-Clause

google.golang.org/genproto/googleapis/api/httpbody

Apache-2.0

google.golang.org/genproto/googleapis/rpc

Apache-2.0

google.golang.org/grpc

Apache-2.0

google.golang.org/protobuf

BSD-3-Clause

gopkg.in/yaml.v2

Apache-2.0

gopkg.in/yaml.v3

MIT