New
Join our webinar! Building a customizable and extensible cloud asset inventory at scale
Back to source list
fastly
Official
Premium

Fastly

The CloudQuery Fastly plugin reads information from your Fastly account(s) and loads it into any supported CloudQuery destination

Publisher

cloudquery

Latest version

v4.8.4

Type

Source

Platforms
Date Published

Price per 1M rows

Starting from $17

monthly free quota

1M rows

Overview #

The CloudQuery Fastly plugin reads information from your Fastly account(s) and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).

Example Config #

kind: source
spec:
  name: fastly
  path: cloudquery/fastly
  registry: cloudquery
  version: "v4.8.4"
  tables: ["*"]
  destinations: ["postgresql"]
  # Fastly specific configuration
  # Learn more about the configuration options at https://cql.ink/fastly_source
  spec:
    fastly_api_key: ${FASTLY_API_TOKEN}
    # services: ["1234567890abcdefghijkl", "0987654321mnopqrstuvwx"]
    # concurrency: 10000

Authentication #

The Fastly plugin requires an API key to authenticate with Fastly. You can generate an API key in the Fastly UI by following the instructions in the Fastly documentation.
Once you have an API key, export it as an environment variable:
export FASTLY_API_TOKEN=<your_api_key>
(MacOS / Linux) or
set FASTLY_API_TOKEN=<your_api_key>
(Windows)

Configuration #

This is the (nested) spec used by the Fastly source plugin.
  • fastly_api_key (string) (required)
    An API token to access Fastly resources. This can be obtained by creating a Fastly API token. It should be a User token with the Global:read scope. Automation tokens do not allow the listing of service versions.
  • services ([]string) (optional)
    A list of Fastly service IDs to sync. If not specified, all services will be used.
  • concurrency (integer) (optional) (default: 10000)
    A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.
  • scheduler (string, optional, default: dfs)
    The scheduler to use when determining the priority of resources to sync. Supported values are dfs (depth-first search), round-robin, shuffle and shuffle-queue.
    For more information about this, see performance tuning.
  • table_options (object) (optional):
    Options to apply to specific tables. Use it to select the sampling rate for the fastly_stats_services table:
    table_options:
        fastly_stats_services:
          sampling_rate: day

Example Queries #

List all services and their active versions #

select name, id, type, active_version from fastly_services;
+----------------------+------------------------+------+----------------+
| name                 | id                     | type | active_version |
|----------------------+------------------------+------+----------------|
| My Test Service      | 1234567890abcdefghijkl | vcl  | 6              |
| Another Service      | 0987654321mnopqrstuvwx | vcl  | 7              |
+----------------------+------------------------+------+----------------+

List domains attached to active versions of all services #

SELECT s.id   AS service_id,
       s.name AS service_name,
       s.active_version,
       d.name AS domain_name
FROM fastly_service_domains d
   JOIN fastly_services s
     ON d.service_id = s.id
     AND d.service_version = s.active_version
ORDER BY service_name;
+------------------------+----------------------+----------------+--------------------------------------+
| service_id             | service_name         | active_version | domain_name                          |
|------------------------+----------------------+----------------+--------------------------------------|
| 1234567890abcdefghijkl | My Service           | 7              | www.my-service-domain.com            |
| 1234567890abcdefghijkl | My Service           | 7              | my-service-domain.com                |
| 0987654321mnopqrstuvwx | Test Service         | 6              | my-test-service-domain.com           |
+------------------------+----------------------+----------------+--------------------------------------+

Discover how health check definitions changed between service versions #

SELECT host,
       path,
       method,
       threshold,
       service_version,
       check_interval
FROM   fastly_service_health_checks
WHERE  service_id = '1234567890abcdefghijkl'
ORDER  BY service_version DESC;
+-----------------------+--------+--------+-----------+-----------------+----------------+
| host                  | path   | method | threshold | service_version | check_interval |
|-----------------------+--------+--------+-----------+-----------------+----------------|
| my-service-domain.com | /blog  | HEAD   | 1         | 4               | 10000          |
| my-service-domain.com | /blog  | HEAD   | 2         | 3               | 20000          |
| my-service-domain.com | /blog  | GET    | 3         | 2               | 30000          |
| my-service-domain.com | /blog  | GET    | 10        | 1               | 40000          |
+-----------------------+--------+--------+-----------+-----------------+----------------+

Retrieve stats for a service for a specific time period #

SELECT   to_char(Date_trunc('month', start_time), 'Month')             AS month,
         sum(requests)                                                 AS requests,
         pg_size_pretty(Sum(resp_body_bytes) + sum(resp_header_bytes)) AS resp_bytes,
         sum(status_ 2xx)                                              AS status_2xx,
         sum(status_4xx)                                               AS status_4xx,
         sum(status_5xx)                                               AS status_5xx
FROM     fastly_stats_services
WHERE    service_id = '1234567890abcdefghijkl'
AND      start_time >= date '2022-01-01'
AND      s tart_time < date '2023-01-01'
GROUP BY date_trunc('month', start_time)
ORDER BY date_trunc('month', start_time) ASC
+-----------+----------+------------+------------+------------+------------+
| month     | requests | resp_bytes | status_2xx | status_4xx | status_5xx |
|-----------+----------+------------+------------+------------+------------|
| January   | 24274    | 225 MB     | 17526      | 1937       | 43         |
| February  | 26584    | 251 MB     | 17817      | 4232       | 14         |
| March     | 24508    | 240 MB     | 18416      | 1788       | 18         |
| April     | 25098    | 243 MB     | 17892      | 3066       | 142        |
| May       | 25865    | 254 MB     | 18647      | 2849       | 18         |
| June      | 18001    | 181 MB     | 12487      | 2711       | 5          |
| July      | 22005    | 206 MB     | 14759      | 3414       | 30         |
| August    | 19737    | 186 MB     | 12824      | 3344       | 14         |
| September | 24001    | 235 MB     | 15944      | 4483       | 5          |
| October   | 23244    | 240 MB     | 16180      | 3099       | 8          |
| November  | 22119    | 201 MB     | 15237      | 3832       | 2          |
| December  | 18767    | 180 MB     | 13414      | 2423       | 18         |
+-----------+----------+------------+------------+------------+------------+

Select users that don't have 2FA enabled #

SELECT name,
       login,
       two_factor_auth_enabled
FROM   fastly_account_users
WHERE  two_factor_auth_enabled IS FALSE
+---------------+---------------------------+-------------------------+
| name          | login                     | two_factor_auth_enabled |
|---------------+---------------------------+-------------------------|
| Rudolph       | [email protected]         | False                   |
| Santa         | [email protected] | False                   |
+---------------+---------------------------+-------------------------+

List all API tokens and their expiry dates #

SELECT NAME,
       scope,
       created_at,
       last_used_at,
       expires_at
FROM   fastly_auth_tokens
+-----------------------------------+-------------+---------------------+---------------------+---------------------+
| name                              | scope       | created_at          | last_used_at        | expires_at          |
|-----------------------------------+-------------+---------------------+---------------------+---------------------|
| manage.fastly.com browser session | global      | 2022-12-26 12:24:01 | <null>              | <null>              |
| Robot Santa Claus                 | global:read | 2022-11-25 00:00:00 | <null>              | 2024-12-25 00:00:00 |
+-----------------------------------+-------------+---------------------+---------------------+---------------------+


Licenses #

The following tools / packages are used in this plugin:
NameLicense
github.com/adrg/xdgMIT
github.com/apache/arrow/go/v13Apache-2.0
github.com/apache/arrow-go/v18Apache-2.0
github.com/apapsch/go-jsonmerge/v2MIT
github.com/aws/aws-sdk-go-v2Apache-2.0
github.com/aws/aws-sdk-go-v2/configApache-2.0
github.com/aws/aws-sdk-go-v2/credentialsApache-2.0
github.com/aws/aws-sdk-go-v2/feature/ec2/imdsApache-2.0
github.com/aws/aws-sdk-go-v2/internal/configsourcesApache-2.0
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/iniApache-2.0
github.com/aws/aws-sdk-go-v2/internal/sync/singleflightBSD-3-Clause
github.com/aws/aws-sdk-go-v2/service/internal/accept-encodingApache-2.0
github.com/aws/aws-sdk-go-v2/service/internal/presigned-urlApache-2.0
github.com/aws/aws-sdk-go-v2/service/licensemanagerApache-2.0
github.com/aws/aws-sdk-go-v2/service/marketplacemeteringApache-2.0
github.com/aws/aws-sdk-go-v2/service/ssoApache-2.0
github.com/aws/aws-sdk-go-v2/service/ssooidcApache-2.0
github.com/aws/aws-sdk-go-v2/service/stsApache-2.0
github.com/aws/smithy-goApache-2.0
github.com/aws/smithy-go/internal/sync/singleflightBSD-3-Clause
github.com/bahlo/generic-list-goBSD-3-Clause
github.com/buger/jsonparserMIT
github.com/cenkalti/backoff/v4MIT
github.com/cloudquery/cloudquery-api-goMPL-2.0
github.com/cloudquery/plugin-pb-goMPL-2.0
github.com/cloudquery/plugin-sdk/v2/internal/globMIT
github.com/cloudquery/plugin-sdk/v2/schemaMIT
github.com/cloudquery/plugin-sdk/v2/typesMPL-2.0
github.com/cloudquery/plugin-sdk/v4MPL-2.0
github.com/cloudquery/plugin-sdk/v4/globMIT
github.com/cloudquery/plugin-sdk/v4/scalarMIT
github.com/davecgh/go-spew/spewISC
github.com/fastly/go-fastly/v9/fastlyApache-2.0
github.com/ghodss/yamlMIT
github.com/go-logr/logrApache-2.0
github.com/go-logr/stdrApache-2.0
github.com/goccy/go-jsonMIT
github.com/golang/mock/gomockApache-2.0
github.com/google/flatbuffers/goApache-2.0
github.com/google/go-querystring/queryBSD-3-Clause
github.com/google/jsonapiMIT
github.com/google/uuidBSD-3-Clause
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptorsApache-2.0
github.com/grpc-ecosystem/grpc-gateway/v2BSD-3-Clause
github.com/hashicorp/go-cleanhttpMPL-2.0
github.com/hashicorp/go-retryablehttpMPL-2.0
github.com/invopop/jsonschemaMIT
github.com/klauspost/compressApache-2.0
github.com/klauspost/compress/internal/snaprefBSD-3-Clause
github.com/klauspost/compress/zstd/internal/xxhashMIT
github.com/mailru/easyjsonMIT
github.com/mattn/go-colorableMIT
github.com/mattn/go-isattyMIT
github.com/mitchellh/mapstructureMIT
github.com/oapi-codegen/runtimeApache-2.0
github.com/peterhellberg/linkMIT
github.com/pierrec/lz4/v4BSD-3-Clause
github.com/pmezard/go-difflib/difflibBSD-3-Clause
github.com/rs/zerologMIT
github.com/samber/loMIT
github.com/santhosh-tekuri/jsonschema/v6Apache-2.0
github.com/spf13/cobraApache-2.0
github.com/spf13/pflagBSD-3-Clause
github.com/stretchr/testifyMIT
github.com/thoas/go-funkMIT
github.com/wk8/go-ordered-map/v2Apache-2.0
github.com/zeebo/xxh3BSD-2-Clause
go.opentelemetry.io/otelApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttpApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttpApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptraceApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttpApache-2.0
go.opentelemetry.io/otel/logApache-2.0
go.opentelemetry.io/otel/metricApache-2.0
go.opentelemetry.io/otel/sdkApache-2.0
go.opentelemetry.io/otel/sdk/logApache-2.0
go.opentelemetry.io/otel/sdk/metricApache-2.0
go.opentelemetry.io/otel/traceApache-2.0
go.opentelemetry.io/proto/otlpApache-2.0
golang.org/x/cryptoBSD-3-Clause
golang.org/x/expBSD-3-Clause
golang.org/x/netBSD-3-Clause
golang.org/x/syncBSD-3-Clause
golang.org/x/sysBSD-3-Clause
golang.org/x/textBSD-3-Clause
golang.org/x/xerrorsBSD-3-Clause
google.golang.org/genproto/googleapis/api/httpbodyApache-2.0
google.golang.org/genproto/googleapis/rpc/statusApache-2.0
google.golang.org/grpcApache-2.0
google.golang.org/protobufBSD-3-Clause
gopkg.in/yaml.v2Apache-2.0
gopkg.in/yaml.v3MIT


Join our mailing list

Subscribe to our newsletter to make sure you don't miss any updates.

Legal

© 2024 CloudQuery, Inc. All rights reserved.

We use tracking cookies to understand how you use the product and help us improve it. Please accept cookies to help us improve. You can always opt out later via the link in the footer.