Set up process #

Authentication #

OAuth 2.0 #

• Using an existing access token

  This token should be authorized for https://www.googleapis.com/auth/analytics.readonly scope (e.g. by using OAuth 2.0 Playground).
• Using OAuth client ID & client secret

  You can get your own OAuth credentials using this guide.

Application Default Credentials #

Local Environment

gcloud auth application-default login \
  --scopes=https://www.googleapis.com/auth/analytics.readonly \
  --client-id-file=[PATH/TO/credentials.json]

Google Cloud cloud-based development environment

Google Cloud containerized environment

Google Cloud services that support attaching a service account

On-premises or another cloud provider

kind: source
# Common source-plugin configuration
spec:
  name: googleanalytics
  path: cloudquery/googleanalytics
  registry: cloudquery
  version: "v4.5.0"
  tables: ["*"]
  destinations: ["postgresql"]
  backend_options:
    table_name: "cq_state_googleanalytics"
    connection: "@@plugins.postgresql.connection"

  # Google Analytics specific configuration
  # Learn more about the configuration options at https://cql.ink/googleanalytics_source
  spec:
    property_id: "<YOUR_PROPERTY_ID_HERE>"
    oauth:
      access_token: "<YOUR_OAUTH_ACCESS_TOKEN>"
    reports:
      - name: example
        dimensions:
          - date
          - language
          - country
          - city
          - browser
          - operatingSystem
          - year
          - month
          - hour
        metrics:
          - name: totalUsers
          - name: new_users
            expression: newUsers
          - name: new_users2
            expression: "newUsers + totalUsers"
            invisible: true
        keep_empty_rows: true

Configuration #

• property_id (string) (required)

  A Google Analytics GA4 property identifier whose events are tracked. To learn more, see where to find your Property ID.

  Supported formats:

  • A plain property ID (example: 1234)
  • Prefixed with properties/ (example: properties/1234)
• reports ([]report) (required)

  Reports to be fetched from Google Analytics.
• start_date (string) (optional) (default: date 7 days prior to the sync start)

  A date in YYYY-MM-DD format (example: 2023-05-15). If not specified, the start date will be the one that is 7 days prior to the sync start date.
• oauth (OAuth spec) (optional)

  OAuth spec for authorization in Google Analytics.
• concurrency (integer) (optional) (default: 10000)

  A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.
• scheduler (string) (optional) (default: ds) The scheduler to use when determining the priority of resources to sync. Supported values are dfs (depth-first search), round-robin, shuffle and shuffle-queue.

  For more information about this, see performance tuning.

Google Analytics OAuth spec #

• access_token (string) (optional)

  An access token that you generated authorizing for https://www.googleapis.com/auth/analytics.readonly scope (e.g., by using OAuth 2.0 Playground).
• client_id (string) (optional)

  OAuth 2.0 Client ID. Required if access_token is empty.
• client_secret (string) (optional)

  OAuth 2.0 Client secret. Required if access_token is empty.

Google Analytics Report spec #

• name (string) (required)

  Name of the report. It will be translated into a table name as ga_ prefix followed by report name in snake case.
• dimensions ([]string) (optional)

  A list of Google Analytics Data API v1 dimensions. At most 9 dimensions can be specified per report.
• metrics ([]metric) (required)

  A list of Google Analytics Data API v1 metrics. Expressions are supported, too.
• keep_empty_rows (boolean) (optional)

  Whether empty rows should be captured, too.

Google Analytics metric spec

• name (string) (required)

  A name or alias (if expression is specified) of the requested metric.
• expression (string) (optional)

  A mathematical expression for derived metrics.
• invisible (boolean) (optional)

  Indicates if a metric is invisible in the report response. This allows creating more complex requests, while also not saving the intermediate results.