New
Join our webinar! Building a customizable and extensible cloud asset inventory at scale
Report an issue
Back to destination list
mssql
Official

Microsoft SQL Server

This destination plugin lets you sync data from a CloudQuery source to a Microsoft SQL Server compatible database. This includes both Microsoft SQL Server and Azure SQL Server.

Publisher

cloudquery

Repositorygithub.com
Latest version

v5.1.4

Type

Destination

Platforms
Date Published

Price

Free

Overview #

Microsoft SQL Server destination plugin

This destination plugin lets you sync data from a CloudQuery source to a Microsoft SQL Server compatible database. This includes both Microsoft SQL Server and Azure SQL Server.
Supported database versions:
  • Microsoft SQL Server >= 2019
  • Azure SQL Database >= 2019


Configuration #

Microsoft SQL Server destination plugin configuration reference

Example Configuration #

kind: destination
spec:
  name: "mssql"
  path: "cloudquery/mssql"
  registry: "cloudquery"
  version: "v5.1.4"
  spec:
    # Connection string in the format `server=localhost;user id=SA;password=yourStrongP@ssword;port=1433;database=cloudquery;`
    connection_string: "${MSSQL_CONNECTION_STRING}"
    # Optional parameters:
    # auth_mode: ms
    # schema: dbo
    # batch_size: 1000 # 1K entries
    # batch_size_bytes: 5242880 # 5 MiB
    # batch_timeout: 20s
The Microsoft SQL Server destination utilizes batching, and supports batch_size and batch_size_bytes.

Microsoft SQL Server spec #

This is the (nested) spec used by the Microsoft SQL Server destination plugin.
  • connection_string (string) (required)
    Connection string to connect to the database. See SDK documentation for details.
    Example connection strings:
    • "sqlserver://username:password@hostname/instance" basic connection using a named instance
    • "sqlserver://username:password@localhost?database=master&connection+timeout=30" select "master" database and set connection timeout (default instance)
 
  • auth_mode (string) (optional) (default: ms)
If you need to authenticate via Azure Active Directory ensure you specify azure value. See SDK documentation for more information. Supported values:
- `ms` _connect to Microsoft SQL Server instance_
- `azure` _connect to Azure SQL Server instance_
  • schema (string) (optional) (default: dbo)
    Schema name to be used. By default, Microsoft SQL Server destination plugin will use the default schema named dbo.
  • batch_size (integer) (optional) (default: 1000)
    Maximum amount of items that may be grouped together to be written in a single write.
  • batch_size_bytes (integer) (optional) (default: 5242880 (= 5 MiB))
    Maximum size of items that may be grouped together to be written in a single write.
  • batch_timeout (duration) (optional) (default: 20s (= 20 seconds))
    Maximum interval between batch writes.

Verbose logging for debug #

The Microsoft SQL Server destination can be run in debug mode. To achieve this pass the log option to connection_string. See SDK documentation for more details.
Note: This will use SDK built-in logging and might output data and sensitive information to logs. Make sure not to use it in production environment.
kind: destination
spec:
  name:     "mssql"
  path:     "cloudquery/mssql"
  registry:   "cloudquery"
  version:  "v5.1.4"

  spec:
    connection_string: "${MSSQL_CONNECTION_STRING};log=255"


Example #

Microsoft SQL Server destination plugin example

In this article we will show you a simple example of configuring Microsoft SQL Server destination plugin.

Prerequisites #

In order to be able to sync to Microsoft SQL Server you will need a running installation. We will be using the quickstart guide for running Microsoft SQL Server locally using Docker.

Create admin password for Microsoft SQL Server #

Microsoft SQL Server enforces password complexity. In order to successfully run the database you must specify a password that adheres the policy described here.
For this example we will be using yourStrongP@ssword as a password.

Start Microsoft SQL Server locally #

docker run \
  -e "ACCEPT_EULA=Y" \
  -e "MSSQL_SA_PASSWORD=yourStrongP@ssword" \
  -p 1433:1433 \
  -d \
  mcr.microsoft.com/mssql/server:2022-latest

Create database to sync to #

We will be using cloudquery for database name in this example.
docker exec $(docker ps -alq) \
  /opt/mssql-tools/bin/sqlcmd \
  -U "SA" \
  -P "yourStrongP@ssword" \
  -Q "CREATE DATABASE cloudquery;"
Note: docker ps -alq returns container ID for the latest started container. You can use container ID discovered manually via docker ps output instead.

Configure Microsoft SQL Server destination plugin #

Once you've completed the steps from Prerequisites section you should be able to connect to the local cloudquery Microsoft SQL Server database via the following connection string:
server=localhost;user id=SA;password=yourStrongP@ssword;port=1433;database=cloudquery;
The (top level) spec section is described in the Destination Spec Reference. The full configuration for Microsoft SQL Server destination plugin will look like this:
kind: destination
spec:
  name: "mssql"
  path: "cloudquery/mssql"
  registry: "cloudquery"
  version: "v5.1.4"

  spec:
    connection_string: "server=localhost;user id=SA;password=yourStrongP@ssword;port=1433;database=cloudquery;"


Licenses #

The following tools / packages are used in this plugin:
NameLicense
github.com/Azure/azure-sdk-for-go/sdk/azcoreMIT
github.com/Azure/azure-sdk-for-go/sdk/azidentityMIT
github.com/Azure/azure-sdk-for-go/sdk/internalMIT
github.com/AzureAD/microsoft-authentication-library-for-go/appsMIT
github.com/adrg/xdgMIT
github.com/apache/arrow/go/v13Apache-2.0
github.com/apache/arrow-go/v18Apache-2.0
github.com/apapsch/go-jsonmerge/v2MIT
github.com/aws/aws-sdk-go-v2Apache-2.0
github.com/aws/aws-sdk-go-v2/configApache-2.0
github.com/aws/aws-sdk-go-v2/credentialsApache-2.0
github.com/aws/aws-sdk-go-v2/feature/ec2/imdsApache-2.0
github.com/aws/aws-sdk-go-v2/internal/configsourcesApache-2.0
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/iniApache-2.0
github.com/aws/aws-sdk-go-v2/internal/sync/singleflightBSD-3-Clause
github.com/aws/aws-sdk-go-v2/service/internal/accept-encodingApache-2.0
github.com/aws/aws-sdk-go-v2/service/internal/presigned-urlApache-2.0
github.com/aws/aws-sdk-go-v2/service/licensemanagerApache-2.0
github.com/aws/aws-sdk-go-v2/service/marketplacemeteringApache-2.0
github.com/aws/aws-sdk-go-v2/service/ssoApache-2.0
github.com/aws/aws-sdk-go-v2/service/ssooidcApache-2.0
github.com/aws/aws-sdk-go-v2/service/stsApache-2.0
github.com/aws/smithy-goApache-2.0
github.com/aws/smithy-go/internal/sync/singleflightBSD-3-Clause
github.com/bahlo/generic-list-goBSD-3-Clause
github.com/buger/jsonparserMIT
github.com/cenkalti/backoff/v4MIT
github.com/cloudquery/cloudquery-api-goMPL-2.0
github.com/cloudquery/plugin-pb-goMPL-2.0
github.com/cloudquery/plugin-sdk/v2/internal/globMIT
github.com/cloudquery/plugin-sdk/v2/schemaMIT
github.com/cloudquery/plugin-sdk/v2/typesMPL-2.0
github.com/cloudquery/plugin-sdk/v4MPL-2.0
github.com/cloudquery/plugin-sdk/v4/globMIT
github.com/cloudquery/plugin-sdk/v4/scalarMIT
github.com/davecgh/go-spew/spewISC
github.com/ghodss/yamlMIT
github.com/go-logr/logrApache-2.0
github.com/go-logr/stdrApache-2.0
github.com/goccy/go-jsonMIT
github.com/golang-jwt/jwt/v5MIT
github.com/golang-sql/civilApache-2.0
github.com/golang-sql/sqlexpBSD-3-Clause
github.com/google/flatbuffers/goApache-2.0
github.com/google/uuidBSD-3-Clause
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptorsApache-2.0
github.com/grpc-ecosystem/grpc-gateway/v2BSD-3-Clause
github.com/hashicorp/go-cleanhttpMPL-2.0
github.com/hashicorp/go-retryablehttpMPL-2.0
github.com/huandu/xstringsMIT
github.com/invopop/jsonschemaMIT
github.com/klauspost/compressApache-2.0
github.com/klauspost/compress/internal/snaprefBSD-3-Clause
github.com/klauspost/compress/zstd/internal/xxhashMIT
github.com/kylelemons/godebugApache-2.0
github.com/mailru/easyjsonMIT
github.com/mattn/go-colorableMIT
github.com/mattn/go-isattyMIT
github.com/microsoft/go-mssqldbBSD-3-Clause
github.com/oapi-codegen/runtimeApache-2.0
github.com/pierrec/lz4/v4BSD-3-Clause
github.com/pkg/browserBSD-2-Clause
github.com/pmezard/go-difflib/difflibBSD-3-Clause
github.com/rs/zerologMIT
github.com/santhosh-tekuri/jsonschema/v6Apache-2.0
github.com/spf13/cobraApache-2.0
github.com/spf13/pflagBSD-3-Clause
github.com/stretchr/testifyMIT
github.com/thoas/go-funkMIT
github.com/wk8/go-ordered-map/v2Apache-2.0
github.com/zeebo/xxh3BSD-2-Clause
go.opentelemetry.io/otelApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttpApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttpApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptraceApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttpApache-2.0
go.opentelemetry.io/otel/logApache-2.0
go.opentelemetry.io/otel/metricApache-2.0
go.opentelemetry.io/otel/sdkApache-2.0
go.opentelemetry.io/otel/sdk/logApache-2.0
go.opentelemetry.io/otel/sdk/metricApache-2.0
go.opentelemetry.io/otel/traceApache-2.0
go.opentelemetry.io/proto/otlpApache-2.0
golang.org/x/cryptoBSD-3-Clause
golang.org/x/expBSD-3-Clause
golang.org/x/netBSD-3-Clause
golang.org/x/sync/errgroupBSD-3-Clause
golang.org/x/sysBSD-3-Clause
golang.org/x/textBSD-3-Clause
golang.org/x/xerrorsBSD-3-Clause
google.golang.org/genproto/googleapis/api/httpbodyApache-2.0
google.golang.org/genproto/googleapis/rpc/statusApache-2.0
google.golang.org/grpcApache-2.0
google.golang.org/protobufBSD-3-Clause
gopkg.in/yaml.v2Apache-2.0
gopkg.in/yaml.v3MIT


Join our mailing list

Subscribe to our newsletter to make sure you don't miss any updates.

Legal

© 2024 CloudQuery, Inc. All rights reserved.

We use tracking cookies to understand how you use the product and help us improve it. Please accept cookies to help us improve. You can always opt out later via the link in the footer.