We just raised $3.5M and we are hiring!

Azure Provider

Azure Provider extends CloudQuery with ability to fetch information on Azure cloud resources and store it in PostgreSQL database.

$ cloudquery init azure
Category
Public Cloud
Version
v0.10.3
License
MPL-2.0
Published at
Tue May 17 2022

The CloudQuery Azure provider pulls configuration out of Azure resources, normalizes them and stores them in PostgreSQL database.

Install

cloudquery init azure

Authentication

CloudQuery needs to be authenticated with your Azure account in order to fetch information about your cloud setup.

You can either authenticate with az login (when running cloudquery locally), or by using a "service principal" and exporting environment variables (appropriate for automated deployments).

You can find out more about authentication with Azure at Azure's documentation for the golang sdk.

Authentication with az login

First, install the Azure CLI (az). Then, login with the Azure CLI:

az login

You are now authenticated with cloudquery!

Authentication with Environment Variables

You will need to create a service principal for CloudQuery to use:

Creating a service principal

First, install the Azure CLI (az).

Then, login with the Azure CLI:

az login

Then, create the service principal cloudquery will use to access your cloud deployment. WARNING: The output of az ad sp create-for-rbac contains credentials that you must protect - Make sure to handle with appropriate care. This example uses bash - The commands for CMD and PowerShell are similar.

export SUBSCRIPTION_ID=<YOUR_SUBSCRIPTION_ID> az account set --subscription $SUBSCRIPTION_ID az provider register --namespace 'Microsoft.Security' # Create a service-principal for cloudquery az ad sp create-for-rbac --name cloudquery-sp --scopes /subscriptions/$SUBSCRIPTION_ID --role Reader

(you can, of course, choose any name you'd like for your service-principal, cloudquery-sp is just an example. If the service principal doesn't exist it will create a new one, otherwise it will update an existing one)

The output of az ad sp create-for-rbac should look like this:

{ "appId": <YOUR AZURE_CLIENT_ID>, "displayName": "cloudquery-sp", "password": <YOUR AZURE_CLIENT_SECRET>, "tenant": <YOUR AZURE_TENANT_ID> }

Exporting environment variables

Next, you need to export the environment variables that cloudquery will use to fetch your cloud configuration. Copy them from the output of az ad sp create-for-rbac (or, take the opportunity to show off your jq-foo). The example shows how to export environment variables for linux - exporting for CMD and PowerShell is similar.

  • AZURE_TENANT_ID is tenant in the json.
  • AZURE_CLIENT_ID is appId in the json.
  • AZURE_CLIENT_SECRET is password in the json.

export AZURE_TENANT_ID=<YOUR AZURE_TENANT_ID> export AZURE_CLIENT_ID=<YOUR AZURE_CLIENT_ID> export AZURE_CLIENT_SECRET=<YOUR AZURE_CLIENT_SECRET> export AZURE_SUBSCRIPTION_ID=$SUBSCRIPTION_ID

Configuration

The following configuration section can be automatically generated by cloudquery init azure:

provider "azure" { configuration { // Optional. if you not specified, cloudquery tries to access all subscriptions available to tenant // subscriptions = ["<YOU_SUBSCRIPTION_ID_HERE>"] } resources = ["*"] }

By default cloudquery will fetch all configuration from all resources in all subscription. You can change this behaviour with the following arguments:

  • subscriptions (Optional) - Specify multiple subscriptions to fetch data from them concurrently.

Query Examples

Find all mysql servers

SELECT * FROM azure_mysql_servers;

Find storage accounts which allowing non https traffic

SELECT * from azure_storage_accounts where enable_https_traffic_only = false;

Find all expired key vault

SELECT * from azure_keyvault_vault_keys where attributes_expires >= extract(epoch from now()) * 1000;