New
Join our webinar! Building a customizable and extensible cloud asset inventory at scale
Back to destination list
splunk
Official

Splunk

This plugin is in preview.

The Splunk destination plugin syncs data from any CloudQuery source plugin(s) to Splunk

Publisher

cloudquery

Latest version

v1.2.4

Type

Destination

Platforms
Date Published

Price

Free

Overview #

Splunk Destination Plugin

The Splunk plugin syncs data from any CloudQuery source plugin(s) to Splunk.

Example config #

kind: destination
spec:
  name: splunk
  path: cloudquery/splunk
  registry: cloudquery
  version: "v1.2.4"
  write_mode: "append"
  spec:
    # Splunk Cloud configuration parameters
    host: "${SPLUNK_HOST}"
    port: 8088
    hec_token: "${SPLUNK_HEC_TOKEN}"
    
    # Optional parameters
    # protocol: https
    # rest_port: 8089
    # session_key: "${SPLUNK_SESSION_KEY}
    # index: main
    # event_source: cloudquery
    # event_sourcetype: _json
    # concurrency: 16
    # batch_size: 1000
    # batch_size_bytes: 5242880
The Splunk destination utilizes batching, and supports batch_size and batch_size_bytes.
It supports append mode only for write_mode.

Splunk Spec #

This is the spec used by the Splunk destination plugin.
  • host (string) (optional) (default: "localhost")
    Host of the Splunk instance. Can be local or remote.
  • port (integer) (optional) (default: 8088)
    Port of the Splunk instance.
  • hec_token (string) (required)
    Splunk HEC token. Go to Settings -> Data Inputs -> HTTP Event Collector -> New Token to create a new token. https://docs.splunk.com/Documentation/SplunkCloud/latest/Data/UsetheHTTPEventCollector
  • protocol (string) (optional) (default: https)
    Protocol for connecting to Splunk. Can be also specified in host.
  • rest_port (integer) (optional) (default: 8089)
    REST API access port of the Splunk instance. Not used besides tests.
  • session_key (string) (optional)
    Splunk session key used for rest API access. Not used besides test. Can be acquired for example by logging in using /services/auth/login Splunk endpoint
  • index (string) (optional) (default: main)
    Splunk index to use for writing the events
  • event_source (string) (optional) (default: cloudquery)
    The source field of events sent to Splunk will have this value
  • event_sourcetype (string) (optional) (default: _json)
    Sourcetype field of events sent to Splunk will have this value
  • batch_size (integer) (optional) (default: 3000)
    Maximum number of items in a batch that may be received by the plugin at a time.
  • batch_size_bytes (integer) (optional) (default: 15728640 (15 MiB))
    Maximum size of items in a batch that may be received by the plugin at a time.
  • max_concurrent_requests (integer) (optional) (default: 3)
    Maximum number of concurrent requests to the Splunk instance. Consider keeping the ratio batch_size / max_concurrent_requests around 1000, while keeping max_concurrent_requests as low as possible. Doing otherwise may result in increasingly high response times from the Splunk instance.


Licenses #

The following tools / packages are used in this plugin:
NameLicense
github.com/adrg/xdgMIT
github.com/apache/arrow/go/v13Apache-2.0
github.com/apache/arrow-go/v18Apache-2.0
github.com/apapsch/go-jsonmerge/v2MIT
github.com/aws/aws-sdk-go-v2Apache-2.0
github.com/aws/aws-sdk-go-v2/configApache-2.0
github.com/aws/aws-sdk-go-v2/credentialsApache-2.0
github.com/aws/aws-sdk-go-v2/feature/ec2/imdsApache-2.0
github.com/aws/aws-sdk-go-v2/internal/configsourcesApache-2.0
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/iniApache-2.0
github.com/aws/aws-sdk-go-v2/internal/sync/singleflightBSD-3-Clause
github.com/aws/aws-sdk-go-v2/service/internal/accept-encodingApache-2.0
github.com/aws/aws-sdk-go-v2/service/internal/presigned-urlApache-2.0
github.com/aws/aws-sdk-go-v2/service/licensemanagerApache-2.0
github.com/aws/aws-sdk-go-v2/service/marketplacemeteringApache-2.0
github.com/aws/aws-sdk-go-v2/service/ssoApache-2.0
github.com/aws/aws-sdk-go-v2/service/ssooidcApache-2.0
github.com/aws/aws-sdk-go-v2/service/stsApache-2.0
github.com/aws/smithy-goApache-2.0
github.com/aws/smithy-go/internal/sync/singleflightBSD-3-Clause
github.com/cenkalti/backoff/v4MIT
github.com/cloudquery/cloudquery-api-goMPL-2.0
github.com/cloudquery/plugin-pb-goMPL-2.0
github.com/cloudquery/plugin-sdk/v2/internal/globMIT
github.com/cloudquery/plugin-sdk/v2/schemaMIT
github.com/cloudquery/plugin-sdk/v2/typesMPL-2.0
github.com/cloudquery/plugin-sdk/v4MPL-2.0
github.com/cloudquery/plugin-sdk/v4/globMIT
github.com/cloudquery/plugin-sdk/v4/scalarMIT
github.com/davecgh/go-spew/spewISC
github.com/ghodss/yamlMIT
github.com/go-logr/logrApache-2.0
github.com/go-logr/stdrApache-2.0
github.com/goccy/go-jsonMIT
github.com/google/flatbuffers/goApache-2.0
github.com/google/uuidBSD-3-Clause
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptorsApache-2.0
github.com/grpc-ecosystem/grpc-gateway/v2BSD-3-Clause
github.com/hashicorp/go-cleanhttpMPL-2.0
github.com/hashicorp/go-retryablehttpMPL-2.0
github.com/huandu/xstringsMIT
github.com/klauspost/compressApache-2.0
github.com/klauspost/compress/internal/snaprefBSD-3-Clause
github.com/klauspost/compress/zstd/internal/xxhashMIT
github.com/mattn/go-colorableMIT
github.com/mattn/go-isattyMIT
github.com/oapi-codegen/runtimeApache-2.0
github.com/pierrec/lz4/v4BSD-3-Clause
github.com/pmezard/go-difflib/difflibBSD-3-Clause
github.com/rs/zerologMIT
github.com/santhosh-tekuri/jsonschema/v6Apache-2.0
github.com/spf13/cobraApache-2.0
github.com/spf13/pflagBSD-3-Clause
github.com/stretchr/testifyMIT
github.com/thoas/go-funkMIT
github.com/zeebo/xxh3BSD-2-Clause
go.opentelemetry.io/otelApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttpApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttpApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptraceApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttpApache-2.0
go.opentelemetry.io/otel/logApache-2.0
go.opentelemetry.io/otel/metricApache-2.0
go.opentelemetry.io/otel/sdkApache-2.0
go.opentelemetry.io/otel/sdk/logApache-2.0
go.opentelemetry.io/otel/sdk/metricApache-2.0
go.opentelemetry.io/otel/traceApache-2.0
go.opentelemetry.io/proto/otlpApache-2.0
golang.org/x/expBSD-3-Clause
golang.org/x/netBSD-3-Clause
golang.org/x/sync/errgroupBSD-3-Clause
golang.org/x/sysBSD-3-Clause
golang.org/x/textBSD-3-Clause
golang.org/x/xerrorsBSD-3-Clause
google.golang.org/genproto/googleapis/api/httpbodyApache-2.0
google.golang.org/genproto/googleapis/rpc/statusApache-2.0
google.golang.org/grpcApache-2.0
google.golang.org/protobufBSD-3-Clause
gopkg.in/yaml.v2Apache-2.0
gopkg.in/yaml.v3MIT


Join our mailing list

Subscribe to our newsletter to make sure you don't miss any updates.

Legal

© 2024 CloudQuery, Inc. All rights reserved.

We use tracking cookies to understand how you use the product and help us improve it. Please accept cookies to help us improve. You can always opt out later via the link in the footer.