Back to source list
Official
Premium
Microsoft Entra ID (Azure AD)
The CloudQuery Microsoft Entra ID (Azure AD) source plugin extracts your Microsoft Entra ID information and loads it into any supported CloudQuery destination
Publisher
cloudquery
Latest version
v1.8.1
Type
Source
Platforms
Date Published
Price per 1M rows
Starting from $17
monthly free quota
1M rows
Set up process #
brew install cloudquery/tap/cloudquery
1. Download CLI and login
2. Create source and destination configs
Plugin configurationOverview #
The CloudQuery Microsoft Entra ID (Azure AD) source plugin extracts your Microsoft Entra ID information and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).
Authentication #
The Microsoft Entra ID source plugin uses the Microsoft Graph API. Authentication is done using a service principal, you can create one using the Azure CLI.
First, install the Azure CLI (
az
).Then, login with the Azure CLI:
az login
Then, create the service principal the plugin will use to access Entra ID. WARNING: The output of
az ad sp create-for-rbac
contains credentials that you must protect - Make sure to handle with appropriate care.
This example uses bash - The commands for CMD and PowerShell are similar.az ad sp create-for-rbac --name CloudQuerySP
You can choose any name you'd like for your service-principal,CloudQuerySP
is an example. If the service principal doesn't exist it will create a new one, otherwise it will update the existing one
The output of az ad sp create-for-rbac should look like this:
{
"appId": "YOUR AZURE_CLIENT_ID",
"displayName": "CloudQuerySP",
"password": "YOUR AZURE_CLIENT_SECRET",
"tenant": "YOUR AZURE_TENANT_ID"
}
Exporting environment variables #
Next, you need to export the environment variables that the plugin will use to sync Entra ID resources.
Copy them from the output of
az ad sp create-for-rbac
.
The example shows how to export environment variables for Linux - exporting for CMD and PowerShell is similar.AZURE_TENANT_ID
istenant
in the JSON.AZURE_CLIENT_ID
isappId
in the JSON.AZURE_CLIENT_SECRET
ispassword
in the JSON.
export AZURE_TENANT_ID=<YOUR AZURE_TENANT_ID>
export AZURE_CLIENT_ID=<YOUR AZURE_CLIENT_ID>
export AZURE_CLIENT_SECRET=<YOUR AZURE_CLIENT_SECRET>
Adding permissions to the service principal #
The Microsoft Graph API requires the service principal to have specific permissions to access the data.
We can cover most tables in the plugin with the
Global Reader
role, but some tables require additional roles.
Below is a list of permissions required for each table (some permissions cover more than one table).Table | Required Permission |
---|---|
entraid_auditlogs_directoryaudits | AuditLog.Read.All |
entraid_auditlogs_signins | AuditLog.Read.All |
entraid_devicemanagement_configurations | DeviceManagementConfiguration.Read.All |
entraid_devicemanagement_manageddevices | DeviceManagementManagedDevices.Read.All |
entraid_group_lifecyclepolicies | Policy.Read.All |
entraid_identity_conditional_access_policies | Policy.Read.All |
entraid_identityprotection_riskdetections | IdentityRiskEvent.Read.All |
entraid_identityprotection_riskyserviceprincipals | IdentityRiskyServicePrincipal.Read.All |
entraid_identityprotection_riskyusers | IdentityRiskyUser.Read.All |
entraid_identityprotection_serviceprincipalriskdetections | IdentityRiskEvent.Read.All |
entraid_policies_identity_security_defaults_enforcement_policy | Policy.Read.All ,Organization.Read.All |
entraid_reports_user_registration_details | AuditLog.Read.All |
entraid_rolemanagement_roleassignmentscheduleinstances | RoleAssignmentSchedule.Read.Directory |
entraid_rolemanagement_roleassignmentschedulerequests | RoleAssignmentSchedule.ReadWrite.Directory |
entraid_rolemanagement_roleassignmentschedules | RoleAssignmentSchedule.Read.Directory |
To add the permissions open the Azure portal, search for
Microsoft Entra ID
and click the Microsoft Entra ID
service to open the overview page.
Then, click on Roles and administrators
as shown in the image below.
Under
Roles and administrators
, search for the Global Reader
role and click on it.
Under the
Global Reader
role, under the Manage->Assignments
sidebar location, click on Add assignments
.
Under
Add assignments
, click no members selected
to open a search box for members.
Search for the service principal you created earlier and click on it.
Approve the next screens to finalize the assignment.
To add specific permissions open
App registrations
from the Microsoft Entra ID
overview page and click on the service principal you created earlier.
Under
manage->API permissions
click on Add a permission
.
Chose
Microsoft Graph
.
Then click on
Application permissions
.
Search for the permission you need to add, for example
AuditLog.Read.All
, select it and click on Add permissions
.
Repeat the process for all permissions required by the tables you want to sync.
After adding all permissions, click on
Grant admin consent for <your tenant>
to finalize the process.
After granting the permissions, you should see a message like the one below.
That is it! You have successfully added the required permissions to the service principal.
Configuration #
This example syncs from Entra ID to a PostgreSQL database destination. The (top level) source spec section is described in the Source Spec Reference.
kind: source
# Common source-plugin configuration
spec:
name: entraid
path: cloudquery/entraid
registry: cloudquery
version: "v1.8.1"
tables: ["*"]
destinations: ["postgresql"]
# Entra ID specific configuration
# Learn more about the configuration options at https://cql.ink/entraid_source
spec:
# Optional parameters
# concurrency: 50000
Entra ID Spec #
concurrency
(integer
) (optional) (default:50000
)A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.scheduler
(string
) (optional) (default:dfs
)The scheduler to use when determining the priority of resources to sync. Supported values aredfs
(depth-first search),round-robin
,shuffle
andshuffle-queue
.For more information about this, see performance tuning.
Incremental syncing #
The Entra ID plugin support incremental syncing via MS Graph API delta queries.
The following tables support incremental syncing:
entraid_applications
using https://learn.microsoft.com/en-us/graph/api/application-deltaentraid_directoryroles
using https://learn.microsoft.com/en-us/graph/api/directoryrole-deltaentraid_groups
using https://learn.microsoft.com/en-us/graph/api/group-deltaentraid_serviceprincipals
using https://learn.microsoft.com/en-us/graph/api/serviceprincipal-deltaentraid_users
using https://learn.microsoft.com/en-us/graph/api/user-delta
To enable incremental syncing for the above tables, set the
backend_options
configuration as shown in the example below. This is documented in the Managing Incremental Tables section.kind: source
spec:
name: "entraid"
path: "cloudquery/entraid"
version: "v1.8.1"
destinations: ["postgresql"]
tables: ["entraid_applications", "entraid_directoryroles", "entraid_groups", "entraid_serviceprincipals", "entraid_users"]
backend_options:
table_name: "cq_state_entraid"
connection: "@@plugins.postgresql.connection"
---
kind: destination
spec:
name: "postgresql"
path: "cloudquery/postgresql"
version: "v8.7.2"
spec:
connection_string: "postgresql://postgres:pass@localhost:5432/postgres?sslmode=disable"
Licenses #
The following tools / packages are used in this plugin:
Name | License |
---|---|
github.com/Azure/azure-sdk-for-go/sdk/azcore | MIT |
github.com/Azure/azure-sdk-for-go/sdk/azidentity | MIT |
github.com/Azure/azure-sdk-for-go/sdk/internal | MIT |
github.com/AzureAD/microsoft-authentication-library-for-go/apps | MIT |
github.com/adrg/xdg | MIT |
github.com/apache/arrow/go/v13 | Apache-2.0 |
github.com/apache/arrow/go/v17 | Apache-2.0 |
github.com/apapsch/go-jsonmerge/v2 | MIT |
github.com/aws/aws-sdk-go-v2 | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/config | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/credentials | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/feature/ec2/imds | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/configsources | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/ini | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/sync/singleflight | BSD-3-Clause |
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/licensemanager | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/marketplacemetering | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/sso | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/ssooidc | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/sts | Apache-2.0 |
github.com/aws/smithy-go | Apache-2.0 |
github.com/aws/smithy-go/internal/sync/singleflight | BSD-3-Clause |
github.com/bahlo/generic-list-go | BSD-3-Clause |
github.com/buger/jsonparser | MIT |
github.com/cenkalti/backoff/v4 | MIT |
github.com/cjlapao/common-go/duration | MIT |
github.com/cloudquery/cloudquery-api-go | MPL-2.0 |
github.com/cloudquery/plugin-pb-go | MPL-2.0 |
github.com/cloudquery/plugin-sdk/v2/internal/glob | MIT |
github.com/cloudquery/plugin-sdk/v2/schema | MIT |
github.com/cloudquery/plugin-sdk/v2/types | MPL-2.0 |
github.com/cloudquery/plugin-sdk/v4 | MPL-2.0 |
github.com/cloudquery/plugin-sdk/v4/glob | MIT |
github.com/cloudquery/plugin-sdk/v4/scalar | MIT |
github.com/davecgh/go-spew/spew | ISC |
github.com/ghodss/yaml | MIT |
github.com/go-logr/logr | Apache-2.0 |
github.com/go-logr/stdr | Apache-2.0 |
github.com/goccy/go-json | MIT |
github.com/golang-jwt/jwt/v5 | MIT |
github.com/google/flatbuffers/go | Apache-2.0 |
github.com/google/uuid | BSD-3-Clause |
github.com/gorilla/mux | BSD-3-Clause |
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors | Apache-2.0 |
github.com/grpc-ecosystem/grpc-gateway/v2 | BSD-3-Clause |
github.com/hashicorp/go-cleanhttp | MPL-2.0 |
github.com/hashicorp/go-retryablehttp | MPL-2.0 |
github.com/invopop/jsonschema | MIT |
github.com/klauspost/compress | Apache-2.0 |
github.com/klauspost/compress/internal/snapref | BSD-3-Clause |
github.com/klauspost/compress/zstd/internal/xxhash | MIT |
github.com/kylelemons/godebug | Apache-2.0 |
github.com/mailru/easyjson | MIT |
github.com/mattn/go-colorable | MIT |
github.com/mattn/go-isatty | MIT |
github.com/microsoft/kiota-abstractions-go | MIT |
github.com/microsoft/kiota-authentication-azure-go | MIT |
github.com/microsoft/kiota-http-go | MIT |
github.com/microsoft/kiota-serialization-form-go | MIT |
github.com/microsoft/kiota-serialization-json-go | MIT |
github.com/microsoft/kiota-serialization-multipart-go | MIT |
github.com/microsoft/kiota-serialization-text-go | MIT |
github.com/microsoftgraph/msgraph-beta-sdk-go | MIT |
github.com/microsoftgraph/msgraph-sdk-go | MIT |
github.com/microsoftgraph/msgraph-sdk-go-core | MIT |
github.com/oapi-codegen/runtime | Apache-2.0 |
github.com/pierrec/lz4/v4 | BSD-3-Clause |
github.com/pkg/browser | BSD-2-Clause |
github.com/pmezard/go-difflib/difflib | BSD-3-Clause |
github.com/rs/zerolog | MIT |
github.com/samber/lo | MIT |
github.com/santhosh-tekuri/jsonschema/v6 | Apache-2.0 |
github.com/spf13/cobra | Apache-2.0 |
github.com/spf13/pflag | BSD-3-Clause |
github.com/std-uritemplate/std-uritemplate/go | Apache-2.0 |
github.com/stretchr/testify | MIT |
github.com/thoas/go-funk | MIT |
github.com/wk8/go-ordered-map/v2 | Apache-2.0 |
github.com/zeebo/xxh3 | BSD-2-Clause |
go.opentelemetry.io/otel | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlptrace | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp | Apache-2.0 |
go.opentelemetry.io/otel/log | Apache-2.0 |
go.opentelemetry.io/otel/metric | Apache-2.0 |
go.opentelemetry.io/otel/sdk | Apache-2.0 |
go.opentelemetry.io/otel/sdk/log | Apache-2.0 |
go.opentelemetry.io/otel/sdk/metric | Apache-2.0 |
go.opentelemetry.io/otel/trace | Apache-2.0 |
go.opentelemetry.io/proto/otlp | Apache-2.0 |
golang.org/x/crypto/pkcs12 | BSD-3-Clause |
golang.org/x/exp | BSD-3-Clause |
golang.org/x/net | BSD-3-Clause |
golang.org/x/sync | BSD-3-Clause |
golang.org/x/sys | BSD-3-Clause |
golang.org/x/text | BSD-3-Clause |
golang.org/x/xerrors | BSD-3-Clause |
google.golang.org/genproto/googleapis/api/httpbody | Apache-2.0 |
google.golang.org/genproto/googleapis/rpc/status | Apache-2.0 |
google.golang.org/grpc | Apache-2.0 |
google.golang.org/protobuf | BSD-3-Clause |
gopkg.in/yaml.v2 | Apache-2.0 |
gopkg.in/yaml.v3 | MIT |