Official

Neo4j destination integration documentation

This destination plugin lets you sync data from a CloudQuery source to a Neo4j database

Publisher

cloudquery

Repository

github.com

Latest version

v5.4.26

Type

Destination

Platforms

Date Published

Documentation Changelog

Overview Types Licenses

Overview #

Neo4j Destination Plugin

This destination plugin lets you sync data from a CloudQuery source to a Neo4j database.

Supported database (tested) versions (We use the official Neo4j Go driver):

Neo4j >= 4.4

As a side note graph databases can be quite useful for various networking use-cases, visualization, for read-teams, blue-teams and more.

Configuration #

Example #

This example configures a Neo4j destination, located at bolt://localhost:7687. The username and password are stored in environment variables.

kind: destination
spec:
  name: "neo4j"
  path: "cloudquery/neo4j"
  registry: "cloudquery"
  version: "v5.4.26"
  # Learn more about the configuration options at https://cql.ink/neo4j_destination
  spec:
    connection_string: "${NEO4J_CONNECTION_STRING}"
    username: "${NEO4J_USERNAME}"
    password: "${NEO4J_PASSWORD}"
    # Optional parameters:
    # batch_size: 1000 # 1K entries
    # batch_size_bytes: 4194304 # 4 MiB

The (top level) spec section is described in the Destination Spec Reference.

Make sure you use environment variable expansion in production instead of committing the credentials to the configuration file directly.

The Neo4j destination utilizes batching, and supports batch_size and batch_size_bytes.

Plugin Spec #

This is the (nested) spec used by the Neo4j destination Plugin.

connection_string (string) (required)
Connection string to connect to the database. This can be a URL or a DSN, as per official neo4j docs.
- "bolt://localhost:7687"
- "neo4j://localhost:7687"
username (string) (required)
Username to connect to the database.
password (string) (required)
Password to connect to the database.
batch_size (integer) (optional) (default: 1000)
Number of records to batch together before sending to the database.
batch_size_bytes (integer) (optional) (default: 4194304 (= 4 MiB))
Number of bytes (as Arrow buffer size) to batch together before sending to the database.

Types #

Neo4j Types

The Neo4j destination (v3.0.0 and later) supports most Apache Arrow types. The following table shows the supported types and how they are mapped to Neo4j data types.

Arrow Column Type	Supported?	Neo4j Type
Binary	✅ Yes	`Bytes`
Boolean	✅ Yes	`Boolean`
Date32	✅ Yes	`String`
Date64	✅ Yes	`String`
Decimal	✅ Yes	`String`
Dense Union	✅ Yes	`String`
Dictionary	✅ Yes	`String`
Duration[ms]	✅ Yes	`String`
Duration[ns]	✅ Yes	`String`
Duration[s]	✅ Yes	`String`
Duration[us]	✅ Yes	`String`
Fixed Size List	✅ Yes	`String`
Float16	✅ Yes	`String`
Float32	✅ Yes	`Float`
Float64	✅ Yes	`Float`
Inet	✅ Yes	`String`
Int8	✅ Yes	`BigInt`
Int16	✅ Yes	`BigInt`
Int32	✅ Yes	`BigInt`
Int64	✅ Yes	`BigInt`
Interval[DayTime]	✅ Yes	`String`
Interval[MonthDayNano]	✅ Yes	`String`
Interval[Month]	✅ Yes	`String`
JSON	✅ Yes	`String`
Large Binary	✅ Yes	`Bytes`
Large List	✅ Yes	`List`
Large String	✅ Yes	`String`
List	✅ Yes	`List`
MAC	✅ Yes	`String`
Map	✅ Yes	`String`
String	✅ Yes	`String`
Struct	✅ Yes	`String`
Timestamp[ms]	✅ Yes	`DateTime`
Timestamp[ns]	✅ Yes	`DateTime`
Timestamp[s]	✅ Yes	`DateTime`
Timestamp[us]	✅ Yes	`DateTime`
UUID	✅ Yes	`String`
Uint8	✅ Yes	`BigInt`
Uint16	✅ Yes	`BigInt`
Uint32	✅ Yes	`BigInt`
Uint64	✅ Yes	`BigInt`
Union	✅ Yes	`String`

String-persisted data types are encoded according to the Arrow String Representation specification.

Licenses #

The following tools / packages are used in this plugin:

Name	License
github.com/adrg/xdg	MIT
github.com/apache/arrow/go/v13	Apache-2.0
github.com/apache/arrow-go/v18	Apache-2.0
github.com/apapsch/go-jsonmerge/v2	MIT
github.com/aws/aws-sdk-go-v2	Apache-2.0
github.com/aws/aws-sdk-go-v2/config	Apache-2.0
github.com/aws/aws-sdk-go-v2/credentials	Apache-2.0
github.com/aws/aws-sdk-go-v2/feature/ec2/imds	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/configsources	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/ini	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/sync/singleflight	BSD-3-Clause
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/licensemanager	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/marketplacemetering	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/sso	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/ssooidc	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/sts	Apache-2.0
github.com/aws/smithy-go	Apache-2.0
github.com/aws/smithy-go/internal/sync/singleflight	BSD-3-Clause
github.com/cenkalti/backoff/v4	MIT
github.com/cloudquery/cloudquery-api-go	MPL-2.0
github.com/cloudquery/plugin-pb-go	MPL-2.0
github.com/cloudquery/plugin-sdk/v2/internal/glob	MIT
github.com/cloudquery/plugin-sdk/v2/schema	MIT
github.com/cloudquery/plugin-sdk/v2/types	MPL-2.0
github.com/cloudquery/plugin-sdk/v4	MPL-2.0
github.com/cloudquery/plugin-sdk/v4/glob	MIT
github.com/cloudquery/plugin-sdk/v4/scalar	MIT
github.com/davecgh/go-spew/spew	ISC
github.com/ghodss/yaml	MIT
github.com/go-logr/logr	Apache-2.0
github.com/go-logr/stdr	Apache-2.0
github.com/goccy/go-json	MIT
github.com/google/flatbuffers/go	Apache-2.0
github.com/google/uuid	BSD-3-Clause
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors	Apache-2.0
github.com/grpc-ecosystem/grpc-gateway/v2	BSD-3-Clause
github.com/hashicorp/go-cleanhttp	MPL-2.0
github.com/hashicorp/go-retryablehttp	MPL-2.0
github.com/huandu/xstrings	MIT
github.com/klauspost/compress	Apache-2.0
github.com/klauspost/compress/internal/snapref	BSD-3-Clause
github.com/klauspost/compress/zstd/internal/xxhash	MIT
github.com/mattn/go-colorable	MIT
github.com/mattn/go-isatty	MIT
github.com/neo4j/neo4j-go-driver/v5/neo4j	Apache-2.0
github.com/oapi-codegen/runtime	Apache-2.0
github.com/pierrec/lz4/v4	BSD-3-Clause
github.com/pmezard/go-difflib/difflib	BSD-3-Clause
github.com/rs/zerolog	MIT
github.com/santhosh-tekuri/jsonschema/v6	Apache-2.0
github.com/spf13/cobra	Apache-2.0
github.com/spf13/pflag	BSD-3-Clause
github.com/stretchr/testify	MIT
github.com/thoas/go-funk	MIT
github.com/zeebo/xxh3	BSD-2-Clause
go.opentelemetry.io/otel	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp	Apache-2.0
go.opentelemetry.io/otel/log	Apache-2.0
go.opentelemetry.io/otel/metric	Apache-2.0
go.opentelemetry.io/otel/sdk	Apache-2.0
go.opentelemetry.io/otel/sdk/log	Apache-2.0
go.opentelemetry.io/otel/sdk/metric	Apache-2.0
go.opentelemetry.io/otel/trace	Apache-2.0
go.opentelemetry.io/proto/otlp	Apache-2.0
golang.org/x/exp	BSD-3-Clause
golang.org/x/net	BSD-3-Clause
golang.org/x/sync/errgroup	BSD-3-Clause
golang.org/x/sys	BSD-3-Clause
golang.org/x/text	BSD-3-Clause
golang.org/x/xerrors	BSD-3-Clause
google.golang.org/genproto/googleapis/api/httpbody	Apache-2.0
google.golang.org/genproto/googleapis/rpc/status	Apache-2.0
google.golang.org/grpc	Apache-2.0
google.golang.org/protobuf	BSD-3-Clause
gopkg.in/yaml.v2	Apache-2.0
gopkg.in/yaml.v3	MIT

Loading plugin documentation