Back to destination list
Official
Microsoft SQL Server
This destination plugin lets you sync data from a CloudQuery source to a Microsoft SQL Server compatible database. This includes both Microsoft SQL Server and Azure SQL Server.
Price
Free
Overview #
Microsoft SQL Server destination plugin
This destination plugin lets you sync data from a CloudQuery source to a Microsoft SQL Server compatible database.
This includes both Microsoft SQL Server and Azure SQL Server.
Supported database versions:
- Microsoft SQL Server >= 2019
- Azure SQL Database >= 2019
Configuration #
Microsoft SQL Server destination plugin configuration reference
Example Configuration #
kind: destination
spec:
name: "mssql"
path: "cloudquery/mssql"
registry: "cloudquery"
version: "v5.1.4"
spec:
# Connection string in the format `server=localhost;user id=SA;password=yourStrongP@ssword;port=1433;database=cloudquery;`
connection_string: "${MSSQL_CONNECTION_STRING}"
# Optional parameters:
# auth_mode: ms
# schema: dbo
# batch_size: 1000 # 1K entries
# batch_size_bytes: 5242880 # 5 MiB
# batch_timeout: 20s
The Microsoft SQL Server destination utilizes batching, and supports
batch_size
and batch_size_bytes
.Microsoft SQL Server spec #
This is the (nested) spec used by the Microsoft SQL Server destination plugin.
connection_string
(string
) (required)Connection string to connect to the database. See SDK documentation for details.Example connection strings:"sqlserver://username:password@hostname/instance"
basic connection using a named instance"sqlserver://username:password@localhost?database=master&connection+timeout=30"
select "master" database and set connection timeout (default instance)
auth_mode
(string
) (optional) (default:ms
)
If you need to authenticate via Azure Active Directory ensure you specify
azure
value.
See SDK documentation for more information.
Supported values:- `ms` _connect to Microsoft SQL Server instance_
- `azure` _connect to Azure SQL Server instance_
schema
(string
) (optional) (default:dbo
)Schema name to be used. By default, Microsoft SQL Server destination plugin will use the default schema nameddbo
.batch_size
(integer
) (optional) (default:1000
)Maximum amount of items that may be grouped together to be written in a single write.batch_size_bytes
(integer
) (optional) (default:5242880
(= 5 MiB))Maximum size of items that may be grouped together to be written in a single write.batch_timeout
(duration
) (optional) (default:20s
(= 20 seconds))Maximum interval between batch writes.
Verbose logging for debug #
The Microsoft SQL Server destination can be run in debug mode.
To achieve this pass the
log
option to connection_string
.
See SDK documentation for more details.Note: This will use SDK built-in logging
and might output data and sensitive information to logs.
Make sure not to use it in production environment.
kind: destination
spec:
name: "mssql"
path: "cloudquery/mssql"
registry: "cloudquery"
version: "v5.1.4"
spec:
connection_string: "${MSSQL_CONNECTION_STRING};log=255"
Example #
Microsoft SQL Server destination plugin example
In this article we will show you a simple example of configuring Microsoft SQL Server destination plugin.
Prerequisites #
In order to be able to sync to Microsoft SQL Server you will need a running installation.
We will be using the quickstart guide
for running Microsoft SQL Server locally using Docker.
Create admin password for Microsoft SQL Server #
Microsoft SQL Server enforces password complexity.
In order to successfully run the database you must specify a password that adheres the policy described
here.
For this example we will be using
yourStrongP@ssword
as a password.Start Microsoft SQL Server locally #
docker run \
-e "ACCEPT_EULA=Y" \
-e "MSSQL_SA_PASSWORD=yourStrongP@ssword" \
-p 1433:1433 \
-d \
mcr.microsoft.com/mssql/server:2022-latest
Create database to sync to #
We will be using
cloudquery
for database name in this example.docker exec $(docker ps -alq) \
/opt/mssql-tools/bin/sqlcmd \
-U "SA" \
-P "yourStrongP@ssword" \
-Q "CREATE DATABASE cloudquery;"
Note:
docker ps -alq
returns container ID for the latest started container.
You can use container ID discovered manually via docker ps
output instead.Configure Microsoft SQL Server destination plugin #
Once you've completed the steps from Prerequisites section
you should be able to connect to the local
cloudquery
Microsoft SQL Server database
via the following connection string:server=localhost;user id=SA;password=yourStrongP@ssword;port=1433;database=cloudquery;
The (top level) spec section is described in the Destination Spec Reference.
The full configuration for Microsoft SQL Server destination plugin will look like this:
kind: destination
spec:
name: "mssql"
path: "cloudquery/mssql"
registry: "cloudquery"
version: "v5.1.4"
spec:
connection_string: "server=localhost;user id=SA;password=yourStrongP@ssword;port=1433;database=cloudquery;"
Licenses #
The following tools / packages are used in this plugin:
Name | License |
---|---|
github.com/Azure/azure-sdk-for-go/sdk/azcore | MIT |
github.com/Azure/azure-sdk-for-go/sdk/azidentity | MIT |
github.com/Azure/azure-sdk-for-go/sdk/internal | MIT |
github.com/AzureAD/microsoft-authentication-library-for-go/apps | MIT |
github.com/adrg/xdg | MIT |
github.com/apache/arrow/go/v13 | Apache-2.0 |
github.com/apache/arrow-go/v18 | Apache-2.0 |
github.com/apapsch/go-jsonmerge/v2 | MIT |
github.com/aws/aws-sdk-go-v2 | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/config | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/credentials | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/feature/ec2/imds | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/configsources | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/ini | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/sync/singleflight | BSD-3-Clause |
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/licensemanager | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/marketplacemetering | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/sso | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/ssooidc | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/sts | Apache-2.0 |
github.com/aws/smithy-go | Apache-2.0 |
github.com/aws/smithy-go/internal/sync/singleflight | BSD-3-Clause |
github.com/bahlo/generic-list-go | BSD-3-Clause |
github.com/buger/jsonparser | MIT |
github.com/cenkalti/backoff/v4 | MIT |
github.com/cloudquery/cloudquery-api-go | MPL-2.0 |
github.com/cloudquery/plugin-pb-go | MPL-2.0 |
github.com/cloudquery/plugin-sdk/v2/internal/glob | MIT |
github.com/cloudquery/plugin-sdk/v2/schema | MIT |
github.com/cloudquery/plugin-sdk/v2/types | MPL-2.0 |
github.com/cloudquery/plugin-sdk/v4 | MPL-2.0 |
github.com/cloudquery/plugin-sdk/v4/glob | MIT |
github.com/cloudquery/plugin-sdk/v4/scalar | MIT |
github.com/davecgh/go-spew/spew | ISC |
github.com/ghodss/yaml | MIT |
github.com/go-logr/logr | Apache-2.0 |
github.com/go-logr/stdr | Apache-2.0 |
github.com/goccy/go-json | MIT |
github.com/golang-jwt/jwt/v5 | MIT |
github.com/golang-sql/civil | Apache-2.0 |
github.com/golang-sql/sqlexp | BSD-3-Clause |
github.com/google/flatbuffers/go | Apache-2.0 |
github.com/google/uuid | BSD-3-Clause |
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors | Apache-2.0 |
github.com/grpc-ecosystem/grpc-gateway/v2 | BSD-3-Clause |
github.com/hashicorp/go-cleanhttp | MPL-2.0 |
github.com/hashicorp/go-retryablehttp | MPL-2.0 |
github.com/huandu/xstrings | MIT |
github.com/invopop/jsonschema | MIT |
github.com/klauspost/compress | Apache-2.0 |
github.com/klauspost/compress/internal/snapref | BSD-3-Clause |
github.com/klauspost/compress/zstd/internal/xxhash | MIT |
github.com/kylelemons/godebug | Apache-2.0 |
github.com/mailru/easyjson | MIT |
github.com/mattn/go-colorable | MIT |
github.com/mattn/go-isatty | MIT |
github.com/microsoft/go-mssqldb | BSD-3-Clause |
github.com/oapi-codegen/runtime | Apache-2.0 |
github.com/pierrec/lz4/v4 | BSD-3-Clause |
github.com/pkg/browser | BSD-2-Clause |
github.com/pmezard/go-difflib/difflib | BSD-3-Clause |
github.com/rs/zerolog | MIT |
github.com/santhosh-tekuri/jsonschema/v6 | Apache-2.0 |
github.com/spf13/cobra | Apache-2.0 |
github.com/spf13/pflag | BSD-3-Clause |
github.com/stretchr/testify | MIT |
github.com/thoas/go-funk | MIT |
github.com/wk8/go-ordered-map/v2 | Apache-2.0 |
github.com/zeebo/xxh3 | BSD-2-Clause |
go.opentelemetry.io/otel | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlptrace | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp | Apache-2.0 |
go.opentelemetry.io/otel/log | Apache-2.0 |
go.opentelemetry.io/otel/metric | Apache-2.0 |
go.opentelemetry.io/otel/sdk | Apache-2.0 |
go.opentelemetry.io/otel/sdk/log | Apache-2.0 |
go.opentelemetry.io/otel/sdk/metric | Apache-2.0 |
go.opentelemetry.io/otel/trace | Apache-2.0 |
go.opentelemetry.io/proto/otlp | Apache-2.0 |
golang.org/x/crypto | BSD-3-Clause |
golang.org/x/exp | BSD-3-Clause |
golang.org/x/net | BSD-3-Clause |
golang.org/x/sync/errgroup | BSD-3-Clause |
golang.org/x/sys | BSD-3-Clause |
golang.org/x/text | BSD-3-Clause |
golang.org/x/xerrors | BSD-3-Clause |
google.golang.org/genproto/googleapis/api/httpbody | Apache-2.0 |
google.golang.org/genproto/googleapis/rpc/status | Apache-2.0 |
google.golang.org/grpc | Apache-2.0 |
google.golang.org/protobuf | BSD-3-Clause |
gopkg.in/yaml.v2 | Apache-2.0 |
gopkg.in/yaml.v3 | MIT |