CloudQuery

Authentication #

Incremental Syncing #

Example Configuration #

kind: source
# Common source-plugin configuration
spec:
  name: shopify
  path: cloudquery/shopify
  registry: cloudquery
  version: "v8.4.7"
  tables: ["*"]
  destinations: ["postgresql"]
  backend_options:
    table_name: "cq_state_shopify"
    connection: "@@plugins.postgresql.connection"
  # Shopify specific configuration
  # Learn more about the configuration options at https://cql.ink/shopify_source
  spec:
    # required, or alternatively use access_token
    api_key: "${SHOPIFY_API_KEY}"
    # required, or alternatively use access_token
    api_secret: "${SHOPIFY_API_SECRET}"
    # required, e.g. https://mystore.myshopify.com
    shop_url: "${SHOPIFY_SHOP_URL}"

Configuration Reference #

• api_key (string) (required if access_token isn't used)

  The API Key for your custom app in your store.
• api_secret (string) (required if access_token isn't used)

  The API Secret for your custom app in your store.
• access_token (string) (required if api_key & api_secret aren't used)

  An access token for your Shopify custom app. This is an alternative way of authenticating, use either this or the ones above.
• shop_url (string) (required)

  The URL of your Shopify store, e.g. https://mystore.myshopify.com.
• api_version (string) (optional) (default: 2023-10)

  The Shopify Admin API version to use. See here for more information.
• timeout_secs (integer) (optional) (default: 10)

  Timeout (in seconds) for requests against the Shopify Admin API.
• max_retries (integer) (optional) (default: 30)

  Number of retries if a request was rate limited.
• page_size (integer) (optional) (default: 50)

  Maximum number of items queried each request. Find an optimum value to balance amount of data fetched and requests timing out. Maximum value 250.
• max_pages (integer) (optional)

  If set, stop after fetching this many pages for each resource. Useful for debugging.
• concurrency (integer) (optional) (default: 1000)

  A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.
• scheduler (string) (optional) (default: dfs) The scheduler to use when determining the priority of resources to sync. Supported values are dfs (depth-first search), round-robin, shuffle and shuffle-queue.

  For more information about this, see performance tuning.

Query Examples

Get all your active products with a specific tag #

SELECT * FROM shopify_products WHERE status='active' AND 'your-tag' = ANY(tags);