Back to plugin list
k8s
Official

Kubernetes

The K8s Source plugin for CloudQuery extracts configuration from a variety of K8s APIs

Publisher

cloudquery

Repositorygithub.com
Latest version

v6.0.4

Type

Source

Platforms
Date Published

Nov 16, 2023

Price

Free

Set up process


brew install cloudquery/tap/cloudquery

1. Download CLI and login

See installation options

2. Create source and destination configs

Plugin configuration

cloudquery sync k8s.yml postgresql.yml

3. Run the sync

CloudQuery sync

Overview

Kubernetes Source Plugin

The K8s Source plugin for CloudQuery extracts configuration from a variety of K8s APIs.

Libraries in Use

Authentication

Similar to how kubectl works, cloudquery depends on a Kubernetes configuration file to connect to a Kubernetes cluster and sync its information. By default, cloudquery uses the default Kubernetes configuration file (~/.kube/config). You can also specify a different configuration by setting the KUBECONFIG environment variable before running cloudquery sync.
export KUBECONFIG="<PATH_TO_YOUR_CONFIG_FILE>"

Kubernetes Service Account

If cloudquery is running in a pod of the Kubernetes cluster, the Kubernetes Service Account can be used for direct authentication. To use the Kubernetes Service Account for direct authentication, a cluster role with all get and list privileges will need to be used.
The below command creates a new cluster role with get and list privileges.
kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind:       ClusterRole
metadata:
  name: cloudquery-cluster-read
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - get
  - list
- nonResourceURLs:
  - '*'
  verbs:
  - get
  - list
EOF
Next, the cluster role and service account will need to be linked via a cluster role binding. The following creates a cluster role binding for the role we created above and the service account for the cloudquery pod.
kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind:       ClusterRoleBinding
metadata:
  name: cloudquery-cluster-read-binding
subjects:
- kind: ServiceAccount
  name: cloudquery-sa
roleRef:
  kind: ClusterRole
  name: cloudquery-cluster-read
EOF


Configuration

K8s Source Plugin Configuration Reference

The K8s source plugin connects to a Kubernetes cluster, fetches resources and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).

Example

This example connects a single k8s context to a Postgres destination. The (top level) source spec section is described in the Source Spec Reference.
kind: source
spec:
  # Source spec section
  name: k8s
  path: cloudquery/k8s
  registry: cloudquery
  version: "v5.2.2"
  tables: ["*"]
  destinations: ["postgresql"]

  spec:
    contexts: ["context"]

K8s Spec

This is the (nested) spec used by K8s Source Plugin
  • contexts ([]string) (optional) (default: empty. Will use the default context from K8s's config file)
    Specify K8s contexts to connect to. Specifying * will connect to all contexts available in the K8s config file (usually ~/.kube/config).
  • concurrency (integer) (optional) (default: 50000):
    The best effort maximum number of Go routines to use. Lower this number to reduce memory usage.


Subscribe to product updates

Be the first to know about new features.