Official

Premium

Fastly

The CloudQuery Fastly plugin reads information from your Fastly account(s) and loads it into any supported CloudQuery destination

Publisher

cloudquery

Latest version

v4.7.0

Type

Source

Platforms

Date Published

October 22, 2024

Price per 1M rows

Starting from $15

monthly free quota

1M rows

Documentation Tables Changelog Destinations

Overview

Set up process #

brew install cloudquery/tap/cloudquery

1. Download CLI and login

See installation options

2. Create source and destination configs

Plugin configuration

cloudquery sync fastly.yml postgresql.yml

3. Run the sync

CloudQuery sync

Overview #

The CloudQuery Fastly plugin reads information from your Fastly account(s) and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).

Example Config #

kind: source
spec:
  name: fastly
  path: cloudquery/fastly
  registry: cloudquery
  version: "v4.7.0"
  tables: ["*"]
  destinations: ["postgresql"]
  # Fastly specific configuration
  # Learn more about the configuration options at https://cql.ink/fastly_source
  spec:
    fastly_api_key: ${FASTLY_API_TOKEN}
    # services: ["1234567890abcdefghijkl", "0987654321mnopqrstuvwx"]
    # concurrency: 10000

Authentication #

The Fastly plugin requires an API key to authenticate with Fastly. You can generate an API key in the Fastly UI by following the instructions in the Fastly documentation.

Once you have an API key, export it as an environment variable:

export FASTLY_API_TOKEN=<your_api_key>

(MacOS / Linux) or

set FASTLY_API_TOKEN=<your_api_key>

(Windows)

Configuration #

This is the (nested) spec used by the Fastly source plugin.

fastly_api_key (string) (required)
An API token to access Fastly resources. This can be obtained by creating a Fastly API token. It should be a User token with the Global:read scope. Automation tokens do not allow the listing of service versions.
services ([]string) (optional)
A list of Fastly service IDs to sync. If not specified, all services will be used.
concurrency (integer) (optional) (default: 10000)
A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.
scheduler (string, optional, default: dfs)
The scheduler to use when determining the priority of resources to sync. Supported values are dfs (depth-first search), round-robin, shuffle and shuffle-queue.
For more information about this, see performance tuning.
table_options (object) (optional):
Options to apply to specific tables. Use it to select the sampling rate for the fastly_stats_services table:
```
table_options:
    fastly_stats_services:
      sampling_rate: day
```

Example Queries #

List all services and their active versions #

select name, id, type, active_version from fastly_services;

+----------------------+------------------------+------+----------------+
| name                 | id                     | type | active_version |
|----------------------+------------------------+------+----------------|
| My Test Service      | 1234567890abcdefghijkl | vcl  | 6              |
| Another Service      | 0987654321mnopqrstuvwx | vcl  | 7              |
+----------------------+------------------------+------+----------------+

List domains attached to active versions of all services #

SELECT s.id   AS service_id,
       s.name AS service_name,
       s.active_version,
       d.name AS domain_name
FROM fastly_service_domains d
   JOIN fastly_services s
     ON d.service_id = s.id
     AND d.service_version = s.active_version
ORDER BY service_name;

+------------------------+----------------------+----------------+--------------------------------------+
| service_id             | service_name         | active_version | domain_name                          |
|------------------------+----------------------+----------------+--------------------------------------|
| 1234567890abcdefghijkl | My Service           | 7              | www.my-service-domain.com            |
| 1234567890abcdefghijkl | My Service           | 7              | my-service-domain.com                |
| 0987654321mnopqrstuvwx | Test Service         | 6              | my-test-service-domain.com           |
+------------------------+----------------------+----------------+--------------------------------------+

Discover how health check definitions changed between service versions #

SELECT host,
       path,
       method,
       threshold,
       service_version,
       check_interval
FROM   fastly_service_health_checks
WHERE  service_id = '1234567890abcdefghijkl'
ORDER  BY service_version DESC;

+-----------------------+--------+--------+-----------+-----------------+----------------+
| host                  | path   | method | threshold | service_version | check_interval |
|-----------------------+--------+--------+-----------+-----------------+----------------|
| my-service-domain.com | /blog  | HEAD   | 1         | 4               | 10000          |
| my-service-domain.com | /blog  | HEAD   | 2         | 3               | 20000          |
| my-service-domain.com | /blog  | GET    | 3         | 2               | 30000          |
| my-service-domain.com | /blog  | GET    | 10        | 1               | 40000          |
+-----------------------+--------+--------+-----------+-----------------+----------------+

Retrieve stats for a service for a specific time period #

SELECT   to_char(Date_trunc('month', start_time), 'Month')             AS month,
         sum(requests)                                                 AS requests,
         pg_size_pretty(Sum(resp_body_bytes) + sum(resp_header_bytes)) AS resp_bytes,
         sum(status_ 2xx)                                              AS status_2xx,
         sum(status_4xx)                                               AS status_4xx,
         sum(status_5xx)                                               AS status_5xx
FROM     fastly_stats_services
WHERE    service_id = '1234567890abcdefghijkl'
AND      start_time >= date '2022-01-01'
AND      s tart_time < date '2023-01-01'
GROUP BY date_trunc('month', start_time)
ORDER BY date_trunc('month', start_time) ASC

+-----------+----------+------------+------------+------------+------------+
| month     | requests | resp_bytes | status_2xx | status_4xx | status_5xx |
|-----------+----------+------------+------------+------------+------------|
| January   | 24274    | 225 MB     | 17526      | 1937       | 43         |
| February  | 26584    | 251 MB     | 17817      | 4232       | 14         |
| March     | 24508    | 240 MB     | 18416      | 1788       | 18         |
| April     | 25098    | 243 MB     | 17892      | 3066       | 142        |
| May       | 25865    | 254 MB     | 18647      | 2849       | 18         |
| June      | 18001    | 181 MB     | 12487      | 2711       | 5          |
| July      | 22005    | 206 MB     | 14759      | 3414       | 30         |
| August    | 19737    | 186 MB     | 12824      | 3344       | 14         |
| September | 24001    | 235 MB     | 15944      | 4483       | 5          |
| October   | 23244    | 240 MB     | 16180      | 3099       | 8          |
| November  | 22119    | 201 MB     | 15237      | 3832       | 2          |
| December  | 18767    | 180 MB     | 13414      | 2423       | 18         |
+-----------+----------+------------+------------+------------+------------+

Select users that don't have 2FA enabled #

SELECT name,
       login,
       two_factor_auth_enabled
FROM   fastly_account_users
WHERE  two_factor_auth_enabled IS FALSE

+---------------+---------------------------+-------------------------+
| name          | login                     | two_factor_auth_enabled |
|---------------+---------------------------+-------------------------|
| Rudolph       | [email protected]         | False                   |
| Santa         | [email protected] | False                   |
+---------------+---------------------------+-------------------------+

List all API tokens and their expiry dates #

SELECT NAME,
       scope,
       created_at,
       last_used_at,
       expires_at
FROM   fastly_auth_tokens

+-----------------------------------+-------------+---------------------+---------------------+---------------------+
| name                              | scope       | created_at          | last_used_at        | expires_at          |
|-----------------------------------+-------------+---------------------+---------------------+---------------------|
| manage.fastly.com browser session | global      | 2022-12-26 12:24:01 | <null>              | <null>              |
| Robot Santa Claus                 | global:read | 2022-11-25 00:00:00 | <null>              | 2024-12-25 00:00:00 |
+-----------------------------------+-------------+---------------------+---------------------+---------------------+