Back to destination list
Official
Neo4j
This destination plugin lets you sync data from a CloudQuery source to a Neo4j database
Price
Free
Overview #
Neo4j Destination Plugin
This destination plugin lets you sync data from a CloudQuery source to a Neo4j database.
Supported database (tested) versions (We use the official Neo4j Go driver):
- Neo4j >= 4.4
As a side note graph databases can be quite useful for various networking use-cases, visualization, for read-teams, blue-teams and more.
Configuration #
Example #
This example configures a Neo4j destination, located at
bolt://localhost:7687
. The username and password are stored in environment variables.kind: destination
spec:
name: "neo4j"
path: "cloudquery/neo4j"
registry: "cloudquery"
version: "v5.4.1"
# Learn more about the configuration options at https://cql.ink/neo4j_destination
spec:
connection_string: "${NEO4J_CONNECTION_STRING}"
username: "${NEO4J_USERNAME}"
password: "${NEO4J_PASSWORD}"
# Optional parameters:
# batch_size: 1000 # 1K entries
# batch_size_bytes: 4194304 # 4 MiB
The (top level) spec section is described in the Destination Spec Reference.
The Neo4j destination utilizes batching, and supports
batch_size
and batch_size_bytes
.Plugin Spec #
This is the (nested) spec used by the Neo4j destination Plugin.
connection_string
(string
) (required)Connection string to connect to the database. This can be a URL or a DSN, as per official neo4j docs."bolt://localhost:7687"
"neo4j://localhost:7687"
username
(string
) (required)Username to connect to the database.password
(string
) (required)Password to connect to the database.batch_size
(integer
) (optional) (default:1000
)Number of records to batch together before sending to the database.batch_size_bytes
(integer
) (optional) (default:4194304
(= 4 MiB))Number of bytes (as Arrow buffer size) to batch together before sending to the database.
Types #
Neo4j Types
The Neo4j destination (
v3.0.0
and later) supports most Apache Arrow types. The following table shows the supported types and how they are mapped to Neo4j data types.Arrow Column Type | Supported? | Neo4j Type |
---|---|---|
Binary | ✅ Yes | Bytes |
Boolean | ✅ Yes | Boolean |
Date32 | ✅ Yes | String |
Date64 | ✅ Yes | String |
Decimal | ✅ Yes | String |
Dense Union | ✅ Yes | String |
Dictionary | ✅ Yes | String |
Duration[ms] | ✅ Yes | String |
Duration[ns] | ✅ Yes | String |
Duration[s] | ✅ Yes | String |
Duration[us] | ✅ Yes | String |
Fixed Size List | ✅ Yes | String |
Float16 | ✅ Yes | String |
Float32 | ✅ Yes | Float |
Float64 | ✅ Yes | Float |
Inet | ✅ Yes | String |
Int8 | ✅ Yes | BigInt |
Int16 | ✅ Yes | BigInt |
Int32 | ✅ Yes | BigInt |
Int64 | ✅ Yes | BigInt |
Interval[DayTime] | ✅ Yes | String |
Interval[MonthDayNano] | ✅ Yes | String |
Interval[Month] | ✅ Yes | String |
JSON | ✅ Yes | String |
Large Binary | ✅ Yes | Bytes |
Large List | ✅ Yes | List |
Large String | ✅ Yes | String |
List | ✅ Yes | List |
MAC | ✅ Yes | String |
Map | ✅ Yes | String |
String | ✅ Yes | String |
Struct | ✅ Yes | String |
Timestamp[ms] | ✅ Yes | DateTime |
Timestamp[ns] | ✅ Yes | DateTime |
Timestamp[s] | ✅ Yes | DateTime |
Timestamp[us] | ✅ Yes | DateTime |
UUID | ✅ Yes | String |
Uint8 | ✅ Yes | BigInt |
Uint16 | ✅ Yes | BigInt |
Uint32 | ✅ Yes | BigInt |
Uint64 | ✅ Yes | BigInt |
Union | ✅ Yes | String |
String-persisted data types are encoded according to the Arrow String Representation specification.
Licenses #
The following tools / packages are used in this plugin:
Name | License |
---|---|
github.com/adrg/xdg | MIT |
github.com/apache/arrow/go/v13 | Apache-2.0 |
github.com/apache/arrow/go/v17 | Apache-2.0 |
github.com/apapsch/go-jsonmerge/v2 | MIT |
github.com/aws/aws-sdk-go-v2 | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/config | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/credentials | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/feature/ec2/imds | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/configsources | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/ini | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/sync/singleflight | BSD-3-Clause |
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/licensemanager | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/marketplacemetering | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/sso | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/ssooidc | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/sts | Apache-2.0 |
github.com/aws/smithy-go | Apache-2.0 |
github.com/aws/smithy-go/internal/sync/singleflight | BSD-3-Clause |
github.com/cenkalti/backoff/v4 | MIT |
github.com/cloudquery/cloudquery-api-go | MPL-2.0 |
github.com/cloudquery/plugin-pb-go | MPL-2.0 |
github.com/cloudquery/plugin-sdk/v2/internal/glob | MIT |
github.com/cloudquery/plugin-sdk/v2/schema | MIT |
github.com/cloudquery/plugin-sdk/v2/types | MPL-2.0 |
github.com/cloudquery/plugin-sdk/v4 | MPL-2.0 |
github.com/cloudquery/plugin-sdk/v4/glob | MIT |
github.com/cloudquery/plugin-sdk/v4/scalar | MIT |
github.com/davecgh/go-spew/spew | ISC |
github.com/ghodss/yaml | MIT |
github.com/go-logr/logr | Apache-2.0 |
github.com/go-logr/stdr | Apache-2.0 |
github.com/goccy/go-json | MIT |
github.com/google/flatbuffers/go | Apache-2.0 |
github.com/google/uuid | BSD-3-Clause |
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors | Apache-2.0 |
github.com/grpc-ecosystem/grpc-gateway/v2 | BSD-3-Clause |
github.com/hashicorp/go-cleanhttp | MPL-2.0 |
github.com/hashicorp/go-retryablehttp | MPL-2.0 |
github.com/huandu/xstrings | MIT |
github.com/klauspost/compress | Apache-2.0 |
github.com/klauspost/compress/internal/snapref | BSD-3-Clause |
github.com/klauspost/compress/zstd/internal/xxhash | MIT |
github.com/mattn/go-colorable | MIT |
github.com/mattn/go-isatty | MIT |
github.com/neo4j/neo4j-go-driver/v5/neo4j | Apache-2.0 |
github.com/oapi-codegen/runtime | Apache-2.0 |
github.com/pierrec/lz4/v4 | BSD-3-Clause |
github.com/pmezard/go-difflib/difflib | BSD-3-Clause |
github.com/rs/zerolog | MIT |
github.com/santhosh-tekuri/jsonschema/v6 | Apache-2.0 |
github.com/spf13/cobra | Apache-2.0 |
github.com/spf13/pflag | BSD-3-Clause |
github.com/stretchr/testify | MIT |
github.com/thoas/go-funk | MIT |
github.com/zeebo/xxh3 | BSD-2-Clause |
go.opentelemetry.io/otel | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlptrace | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp | Apache-2.0 |
go.opentelemetry.io/otel/log | Apache-2.0 |
go.opentelemetry.io/otel/metric | Apache-2.0 |
go.opentelemetry.io/otel/sdk | Apache-2.0 |
go.opentelemetry.io/otel/sdk/log | Apache-2.0 |
go.opentelemetry.io/otel/sdk/metric | Apache-2.0 |
go.opentelemetry.io/otel/trace | Apache-2.0 |
go.opentelemetry.io/proto/otlp | Apache-2.0 |
golang.org/x/exp | BSD-3-Clause |
golang.org/x/net | BSD-3-Clause |
golang.org/x/sync/errgroup | BSD-3-Clause |
golang.org/x/sys | BSD-3-Clause |
golang.org/x/text | BSD-3-Clause |
golang.org/x/xerrors | BSD-3-Clause |
google.golang.org/genproto/googleapis/api/httpbody | Apache-2.0 |
google.golang.org/genproto/googleapis/rpc/status | Apache-2.0 |
google.golang.org/grpc | Apache-2.0 |
google.golang.org/protobuf | BSD-3-Clause |
gopkg.in/yaml.v2 | Apache-2.0 |
gopkg.in/yaml.v3 | MIT |