We use tracking cookies to understand how you use the product and help us improve it. Please accept cookies to help us improve. You can always opt out later via the link in the footer.
private_key
as plaintext #|
to pass the multi-line private key as plaintext.- org: cloudquery
private_key: |
-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEA3eVv6PCn9P8zO+EP8K7pLMfxcA2uVrSZ2f+H3GgYIavDxWtO
vM9tE3jAA8mOjZpdLaG5yy4QfV1LQ3R7kO49JCB6VbClwN2lNvd8Iw49JCBDid7D
...
-----END RSA PRIVATE KEY-----
app_id: your_app_id
private_key
as environment variable #private_key
as a string from environment variables, you will need to replace all the new lines in your PEM file with \n
otherwise the new line and indent will prevent CloudQuery from reading the variable correctly.- org: cloudquery
private_key: "${GITHUB_PRI_KEY}"
app_id: your_app_id
...
GITHUB_PRI_KEY="-----BEGIN RSA PRIVATE KEY-----\nMIIEpQIBAAKCAQEA3eVv6PCn9P8zO+EP8K7pLMfxcA2uVrSZ2f+H3GgYIavDxWtO\n...vM9tE3jAA8mOjZpdLaG5yy4QfV1LQ3R7kO49JCB6VbClwN2lNvd8Iw==\n-----END RSA PRIVATE KEY-----"
.yml
file in your CloudQuery configuration directory.cloudquery/cloudquery
repository:kind: source
spec:
# Source spec section
name: github
path: cloudquery/github
registry: cloudquery
version: "v14.0.1"
tables: ["github_issues"]
destinations: ["postgresql"]
# Learn more about the configuration options at https://cql.ink/github_source
spec:
access_token: "${GITHUB_PERSONAL_ACCESS_TOKEN}" # Personal Access Token, required if not using App Authentication.
# # App Authentication (one per org):
# app_auth:
# - org: cloudquery
# private_key: <PRIVATE_KEY> # Private key as a string
# private_key_path: <PATH_TO_PRIVATE_KEY> # Path to private key file
# app_id: <YOUR_APP_ID> # App ID, required for App Authentication.
# installation_id: <ORG_INSTALLATION_ID> # Installation ID for this org
# # List of organizations to sync from. You must specify either orgs or repos in the configuration.
# orgs: []
# # List of repositories to sync from. The format is `owner/repo` (e.g. `cloudquery/cloudquery`). You must specify either `orgs` or `repos` in the configuration.
# repos: ["cloudquery/cloudquery"]
# # GitHub Enterprise
# # In order to enable GHE you have to provide two urls, the base url of the server and the upload url.
# # Quote from GitHub's client:
# # If the base URL does not have the suffix "/api/v3/", it will be added automatically. If the upload URL does not have the suffix "/api/uploads", it will be added automatically.
# # Another important thing is that by default, the GitHub Enterprise URL format should be http(s)://[hostname]/api/v3/ or you will always receive the 406 status code. The upload URL format should be http(s)://[hostname]/api/uploads/"
# # If you are not configuring against an enterprise server please omit the enterprise configuration bellow
# enterprise:
# base_url: "http(s)://[your-ghe-hostname]/api/v3/"
# upload_url: "http(s)://[your-ghe-hostname]/api/uploads/"
# # Optional parameters
# concurrency: 1500 # Optional. The best effort maximum number of Go routines to use. Lower this number to reduce memory usage or to avoid hitting GitHub API rate limits. Default 1500.
# discovery_concurrency: 1 # Optional. Number of concurrent requests to GitHub API during discovery phase. Default 1.
# include_archived_repos: false # Optional. Include archived repositories in the sync. Default false.
# local_cache_path: "" # Optional. Path to a local directory that will hold the cache. If set, the plugin will cache the GitHub API responses in this directory. Defaults to an empty string (no cache).
# table_options:
# github_workflow_runs:
# created_since: "" # e.g. "7 days ago", defaults to all workflow runs
# github_issues:
# state: "" # e.g. "open, all, closed", defaults to `all`
orgs
or repos
in the configuration. If a repository is specified in both orgs
and repos
, it will be extracted only once, and other repositories from that organization will be ignored.private_key
or private_key_path
in the configuration, but not both.GITHUB_ACCESS_TOKEN
:spec:
access_token: ${GITHUB_ACCESS_TOKEN}
repos
([]string
, optional. Default: empty)
List of repositories to sync from. The format is owner/repo
(e.g. cloudquery/cloudquery
). You must specify either orgs
or repos
in the configuration.orgs
([]string
, optional. Default: empty):
List of organizations to sync from. You must specify either orgs
or repos
in the configuration.concurrency
(integer
, optional, default: 1500
)
A best effort maximum number of Go routines to use. Lower this number to reduce memory usage or to avoid hitting GitHub API rate limits.discovery_concurrency
(integer
, optional, default: 1
)orgs
, to be used later on during the sync process.
By default the plugin discovers repositories one organization at a time. You can increase discovery_concurrency
to discover multiple organizations in parallel, or use a negative value to discover all organizations in parallel.
Please note that it's possible to hit GitHub API rate limits when using a high value for discovery_concurrency
.scheduler
(string
, optional, default: dfs
)
The scheduler to use when determining the priority of resources to sync.
Supported values are dfs
(depth-first search), round-robin
, shuffle
and shuffle-queue
.include_archived_repos
(bool
) (default: false
)include_archived_repos
to true
.local_cache_path
(string
, optional, default: empty)
Path to a local directory that will hold the cache. If set, the plugin will cache the GitHub API responses in this directory. Defaults to an empty string (no cache).
By using a cache, the plugin can use conditional requests when appropriate, and help avoid hitting GitHub API rate limits.table_options
(Table Options spec) (optional)github_workflow_runs
created_since
(string
in natural date format) (optional)7 days ago
, last month
(see more here)github_issues
state
(string
) (optional)open
, closed
, all
. Defaults to all
.labels
([]string
) (optional)since
(string
in natural date format) (optional)7 days ago
, last month
(see more here)Assignee
(string
) (optional)Creator
(string
) (optional)Mentioned
(string
) (optional)github_repositories
updated_after
(Time
) (optional)github_copilot_org_metrics
since
(Time
) (optional)github_organization_dependabot_alerts
updated_after
(Time
) (optional)github_repository_dependabot_alerts
updated_after
(Time
) (optional)github_code_scanning_alerts
updated_after
(Time
) (optional)github_secret_scanning_alerts
updated_after
(Time
) (optional)github_pull_requests
updated_after
(Time
) (optional)github_pull_request_closing_issues_references
updated_after
(Time
) (optional)2025-01-01T12:00:00+00:00
.
Relative timestamps can take this format:now
x seconds [ago|from now]
x minutes [ago|from now]
x hours [ago|from now]
x days [ago|from now]
kind: source
spec:
name: gihtub
path: cloudquery/github
registry: cloudquery
version: "v14.0.1-fips"
...