Back to transformer list
J
Official
JSON Flattener
This plugin is in preview.
The JSON Flattener CloudQuery transformer plugin provides basic JSON flattening capabilities
Overview #
This CloudQuery transformer plugin provides basic JSON flattening capabilities:
- JSON fields on source table will be flattened into separate columns in the destination, while still preserving the original JSON fields.
- Only JSON fields with schema metadata will be flattened. This metadata is present for selected CloudQuery sources.
- Only single-level JSON object fields will be flattened. JSON arrays will not be flattened.
- Multi-level JSON objects will not be flattened recursively.
Example transformation:
If before applying the transformer your sync is producing this table in the destination:
After applying the transformer, you will get this table (
details column content is omitted for brevity):Extra fields are typed based on the
TypeSchema metadata:field_1is typed astimestampfield_2is typed asstringfield_3is typed asbooleanfield_4is typed asint64
Configuration #
First, add the transformer to your destination. For example, this will add a jsonflattener transformer to a PostgreSQL destination:
kind: destination
spec:
name: "postgresql"
path: "cloudquery/postgresql"
registry: "cloudquery"
version: "v8.7.11"
write_mode: "overwrite-delete-stale"
migrate_mode: forced # optional
transformers:
- "jsonflattener"
spec:
connection_string: "postgresql://your.user:your.password@localhost:5432/db_name"The
migrate_mode: forced setting might make sense if you plan on modifying the schema from a previous sync.Then, add your transformer spec:
kind: transformer
spec:
name: "jsonflattener"
path: "cloudquery/jsonflattener"
version: v2.0.4
spec:
tables: ["aws_ec2_instances"]