Sync data from AWS to BigQuery
CloudQuery is the simple, fast data integration platform that can fetch your data from AWS APIs and load it into BigQuery
Trusted by
Get a personalized demo
Book a demo and see how quickly CloudQuery delivers new and ongoing insights on your multi-cloud environments.
Enterprise Ready
Customize & Extend
Query Assets with SQL
Non-invasive account access for better security and efficiency.
Import data with CloudQuery SDKs and build your own plugins.
Query cloud assets and security with a simple SQL-based UI.
Step by step guide for how to export data from AWS to BigQuery
Table of Contents
MacOS Setup
Step 1: Install CloudQuery
To install CloudQuery, run the following command in your terminal:
brew install cloudquery/tap/cloudqueryStep 2: Create a Configuration File
Next, run the following command to initialize a sync configuration file for AWS to BigQuery:
cloudquery init --source=aws --destination=bigqueryThis will generate a config file named aws_to_bigquery.yaml. Follow the instructions to fill out the necessary fields to authenticate against your own environment.
Step 3: Log in to CloudQuery CLI
Next, log in to the CloudQuery CLI. If you have't already, you can sign up for a free account as part of this step:
cloudquery loginStep 4: Run a Sync
cloudquery sync aws_to_bigquery.yamlThis will start syncing data from the AWS API to your BigQuery database! 🚀
See the CloudQuery documentation portal for more deployment guides, options and further tips.