Back to destination list
Official
Elasticsearch
The Elasticsearch plugin syncs data from any CloudQuery source plugin(s) to an Elasticsearch cluster
Loading plugin documentation
The Elasticsearch plugin syncs data from any CloudQuery source plugin(s) to an Elasticsearch cluster
Loading plugin documentation
We use tracking cookies to understand how you use the product and help us improve it. Please accept cookies to help us improve. You can always opt out later via the link in the footer.
localhost:9200:kind: destination
spec:
name: elasticsearch
path: cloudquery/elasticsearch
registry: cloudquery
version: "v3.5.24"
write_mode: "overwrite-delete-stale"
spec:
# Elastic Cloud configuration parameters
cloud_id: "${ELASTICSEARCH_CLOUD_ID}"
api_key: "${ELASTICSEARCH_API_KEY}"
# Self-hosted Elasticsearch configuration parameters
# addresses: ["http://localhost:9200"]
# username: ""
# password: ""
# service_token: ""
# certificate_fingerprint: ""
# ca_cert: ""
# Optional parameters
# concurrency: 5 # default: number of CPUs
# batch_size: 1000
# batch_size_bytes: 5242880 # 5 MiB
batch_size and batch_size_bytes.append, overwrite and overwrite-delete-stale write modes. The default write mode is overwrite-delete-stale.addresses ([]string) (optional) (default: ["http://localhost:9200"])cloud_id.username (string) (optional)password (string) (optional)cloud_id (string) (optional) (example: MyDeployment:abcdefgh)addresses.api_key (string) (optional)service_token (string) (optional)certificate_fingerprint (string) (optional)ca_cert (string) (optional)concurrency (string) (optional) (default: number of CPUs)batch_size (integer) (optional) (default: 1000)batch_size_bytes (integer) (optional) (default: 5242880 (5 MiB))--no-migrate option when running cloudquery sync.append: indexes will be named using the format <table_name>-<YYYY-MM-DD>. In other words, a new index will be created every day the table is synced. Entries will never be overwritten.overwrite: indexes will be named using the format <table_name>. Objects with duplicate primary keys will be overwritten.overwrite-delete-stale: indexes will be named using the format <table_name>. Objects with duplicate primary keys will be overwritten, and any objects that are not present in the current sync will be deleted.<table_name>-*. For example, if you have a table named aws_ec2_instances, you should create a data view with index pattern named aws_ec2_instances-*. One useful feature of Elasticsearch and Kibana, however, is the ability to query across all data. To do this for the aws source plugin, for example, you may use an index pattern named aws_*. This will then allow queries across all tables synced by the aws source plugin.v2.0.0 and later) supports most Apache Arrow types. The following table shows the supported types and how they are mapped to Elasticsearch field data types.