Back to plugin list
Official
Premium
MySQL
The CloudQuery MySQL plugin syncs your MySQL database to any of the supported CloudQuery destinations
Publisher
cloudquery
Latest version
v4.4.0
Type
Source
Platforms
Date Published
Price per 1M rows
Starting from $15
monthly free quota
1M rows
Set up process #
brew install cloudquery/tap/cloudquery1. Download CLI and login
2. Create source and destination configs
Plugin configurationOverview #
The CloudQuery MySQL plugin syncs your MySQL database to any of the supported CloudQuery destinations (e.g. PostgreSQL, BigQuery, Snowflake, and more).
Supported database versions are >= 8.0.
If you need support for older versions, please contact us
Example Configuration #
kind: source
spec:
name: mysql
path: cloudquery/mysql
registry: cloudquery
version: "v4.4.0"
tables: ["*"]
destinations: ["postgresql"]
# Learn more about the configuration options at https://cql.ink/mysql_source
spec:
connection_string: "user:password@/dbname"
# Optional parameters
# rows_per_record: 500Configuration Reference #
This is the (nested) spec used by the MySQL destination plugin.
connection_string(string) (required)Connection string to connect to the database. See the Go driver documentation for more details."user:password@tcp(127.0.0.1:3306)/dbname"connect with TCP"user:[email protected]:3306/dbname?charset=utf8mb4\u0026parseTime=True\u0026loc=Local"connect and set character set, time parsing, and location"user:password@localhost:3306/dbname?timeout=30s\u0026readTimeout=1s\u0026writeTimeout=1s"connect and set various timeouts"user:password@/dbname?loc=UTC\u0026allowNativePasswords=true\u0026tls=preferred"connect and set location and native password allowance, and prefer TLS
rows_per_record(integer) (optional) (default:500)Amount of rows to be packed into a single Apache Arrow record to be sent over the wire during sync.
Table Options Spec #
Enables setting table options for specific tables. Map key is the name of the table in untemplated form.
incremental_column(string) (optional)
Name of the incremental column in the table. If empty, no incremental column will be used. See incremental-examples for more information
Incremental-Examples #
Configuration Examples for Incremental Sync
To sync tables incrementally each table needs to have an
incremental_column specified in the table_options section of the configuration. The tables without an incremental_column will be synced fully.Sync a Table Incrementally #
To sync a table incrementally, you need to specify the
incremental_column in the table_options section of the configuration. Here's an example source spec to sync a list of tables incrementally:kind: source
spec:
name: mysql
path: cloudquery/mysql
registry: cloudquery
version: "v4.4.0"
tables:
- "my_table"
- "another_table"
- "yet_another_table"
destinations: ["postgresql"]
backend_options:
table_name: "cq_state_bq"
connection: "@@plugins.postgresql.connection"
spec:
connection_string: "user:password@/dbname"
# Optional parameters
# rows_per_record: 500
table_options:
my_table:
incremental_column: updated_at
another_table:
incremental_column: idIn the example above, the
my_table table will be synced incrementally based on the updated_at column. another_table will be synced incrementally based on the id column. yet_another_table will be synced fully.The incremental column will be used to fetch only the new or updated rows from the source table and the state will be stored in the
cq_state_bq table in the PostgreSQL destination.