Back to source list
Official
Premium
Kubernetes
The K8s Source plugin for CloudQuery extracts configuration from a variety of K8s APIs
Publisher
cloudquery
Latest version
v7.4.2
Type
Source
Platforms
Date Published
Overview #
The K8s Source plugin for CloudQuery extracts configuration from a variety of K8s APIs.
Libraries in Use #
Authentication #
Similar to how
kubectl works, cloudquery depends on a Kubernetes configuration file to connect to a Kubernetes cluster and sync its information.
By default, cloudquery uses the default Kubernetes configuration file (~/.kube/config).
You can also specify a different configuration by setting the KUBECONFIG environment variable before running cloudquery sync.export KUBECONFIG="<PATH_TO_YOUR_CONFIG_FILE>"Kubernetes Service Account #
If
cloudquery is running in a pod of the Kubernetes cluster, the Kubernetes Service Account can be used for direct authentication. To use the Kubernetes Service Account for direct authentication, a cluster role with all get and list privileges will need to be used.The below command creates a new cluster role with
get and list privileges.kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cloudquery-cluster-read
rules:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- get
- list
- nonResourceURLs:
- '*'
verbs:
- get
- list
EOFNext, the cluster role and service account will need to be linked via a cluster role binding.
The following creates a cluster role binding for the role we created above and the service account for the
cloudquery pod.kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cloudquery-cluster-read-binding
subjects:
- kind: ServiceAccount
name: cloudquery-sa
roleRef:
kind: ClusterRole
name: cloudquery-cluster-read
EOFConfiguration #
K8s Source Plugin Configuration Reference
The K8s source plugin connects to a Kubernetes cluster, fetches resources and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).
Example #
This example connects a single k8s context to a Postgres destination. The (top level) source spec section is described in the Source Spec Reference.
This example connects a single k8s context to a Postgres destination. The (top level) source spec section is described in the Source Spec Reference.
kind: source
spec:
# Source spec section
name: k8s
path: cloudquery/k8s
registry: cloudquery
version: "v7.4.2"
tables: ["*"]
destinations: ["postgresql"]
# Learn more about the configuration options at https://cql.ink/k8s_source
spec:
contexts: ["context"]This example connects to an EKS cluster using an IAM role.
Provider spec is used to generate a kube config file for the provider.
The (top level) source spec section is described in the Source Spec Reference.
kind: source
spec:
name: k8s
path: cloudquery/k8s
version: v7.3.8
tables:
- "*"
spec:
providers:
- type: aws
cluster: eks-cluster-name
aws:
region: us-east-1
role_arn: arn:aws:iam::111111111111:role/cross-account-readonly-roleK8s Spec #
This is the (nested) spec used by K8s Source Plugin
contexts([]string) (optional) (default: empty. Will use the default context from K8s's config file)Specify K8s contexts to connect to. Specifying*will connect to all contexts available in the K8s config file (usually~/.kube/config).concurrency(integer) (optional) (default:5000)A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.scheduler(string) (optional) (default:dfs) The scheduler to use when determining the priority of resources to sync. Supported values aredfs(depth-first search),round-robin,shuffleandshuffle-queue.For more information about this, see performance tuning.providers([]Provider) (optional) (default: empty.)List of providers to connect to. This is used to generate a kube config file for the provider. Each entry in the list represents a context in the K8s config file and first entry is the default context. Cluster name is the name of the context in the K8s config file also.
Provider Spec #
cluster(string) (required)Name of the cluster.type(string) (required)Type of the provider (e.g. "aws", "gcp", "azure") Currently onlyawsis supported.aws(AWSSpec) (optional)AWS specific configuration for EKS cluster access
AWSSpec #
region(string) (required)Region of the EKS cluster.role_arn(string) (optional)IAM Role ARN to assume to access the EKS cluster.
Example #
This example connects a single k8s context to a Postgres destination. The (top level) source spec section is described in the Source Spec Reference.
kind: source
spec:
# Source spec section
name: k8s
path: cloudquery/k8s
registry: cloudquery
version: "v7.4.2"
tables: ["*"]
destinations: ["postgresql"]
# Learn more about the configuration options at https://cql.ink/k8s_source
spec:
contexts: ["context"]This example connects to an EKS cluster using an IAM role.
Provider spec is used to generate a kube config file for the provider.
The (top level) source spec section is described in the Source Spec Reference.
kind: source
spec:
name: k8s
path: cloudquery/k8s
version: v7.3.8
tables:
- "*"
spec:
providers:
- type: aws
cluster: eks-cluster-name
aws:
region: us-east-1
role_arn: arn:aws:iam::111111111111:role/cross-account-readonly-role