Back to source list
Official
Premium
Salesforce
The CloudQuery Salesforce plugin extracts information from your Salesforce organization(s) and loads it into any supported CloudQuery destination
Publisher
cloudquery
Latest version
v5.2.4
Type
Source
Platforms
Date Published
Price per 1M rows
Starting from $17
monthly free quota
1M rows
Overview #
The CloudQuery Salesforce plugin extracts information from your Salesforce organization(s) and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).
Authentication #
You will need to create a Salesforce Connected App using the OAuth2.0 Client Credentials flow. This is a Salesforce app that will be used to authenticate with Salesforce.
You can create a Connected App in the Salesforce Setup UI. Checkout the Salesforce documentation for more information. Or check out our latest guide to set up a connected app.
Configuration #
This example syncs from Salesforce to a Postgres destination:
kind: source
spec:
name: slack
path: cloudquery/salesforce
registry: cloudquery
version: "v5.2.4"
tables: ["*"]
destinations: ["postgresql"]
# Salesforce specific configuration
# Learn more about the configuration options at https://cql.ink/salesforce_source
spec:
domain_name: "${SF_DOMAIN_NAME}" // e.g. my-domain
client_id: "${SF_CLIENT_ID}"
client_secret: "${SF_CLIENT_SECRET}"
It assumes that the
SF_DOMAIN_NAME
, SF_CLIENT_ID
and SF_CLIENT_SECRET
environment variables are set.Configuration #
This is the (nested) spec used by the Slack source plugin.
domain_name
(string
) (required)This is the domain name of your Salesforce instance. It will be used to build the instance endpoint to which calls will be sent. For example, if your Salesforce instance ishttps://my-domain.my.salesforce.com
, then the domain name ismy-domain
.client_id
(string
) (required)This is the consumer key of the connected app. This can be obtained by Creating a Salesforce connected app.client_secret
(string
) (required)This is the consumer secreted of the connected app. This can be obtained by Creating a Salesforce connected app.include_objects
([]string
) (optional)By default the plugin will sync all Salesforce objects. If you want to limit the objects that are synced you can specify the objects you want to sync in this array.exclude_objects
([]string
) (optional)By default the plugin will sync all Salesforce objects. If you want to exclude some objects from being synced you can specify the objects you want to exclude in this array.concurrency
(integer
) (optional) (default:10000
)A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.scheduler
(string
) (optional) (default:dfs
) The scheduler to use when determining the priority of resources to sync. Supported values aredfs
(depth-first search),round-robin
,shuffle
andshuffle-queue
.For more information about this, see performance tuning.
Example Queries #
List all Salesforce objects #
select distinct(object_type) from salesforce_objects;
List all Accounts #
select * from salesforce_objects where object_type = 'Account';
Creating Connected Guide #
Full documentation is available here. This guide will walk you through the steps to create a connected app.
Pre-requisites #
- Salesforce developer account to be able to access the REST APIs
- A Salesforce user with the permissions to create connected a
Step 1: Go To Connected App #
Go to Setup Screen and search for "App Manager" and click on "New Connected App".
Step 2: Create Connected App #
Fill in the fields as the following and click
save
:Step 3: Set up an execution user for the Client Credentials Flow #
Salesforce uses the OAuth 2.0 Client Credentials Flow to authenticate the connected app. This flow requires a user to execute the flow. You can use the same user that created the connected app or create a new user (this approach is more secure as you have the possibility to create an API-only user).
Go to "Apps" > "Connected Apps" > "Manage Connected Apps" > "Edit Policies" and under the "Client Credentials flow" section pick the "Run As" user of your choice.
Step 4: Copy Consumer Key and Consumer Secret #
Now that this is configured click
Manage Consumer Details
to view your Consumer Key and Consumer Secret (those will be used as client_id
and client_secret
to configure your CloudQuery Salesforce source plugin).And copy the Consumer Key and Consumer Secret somewhere safe so you can provide them as environment variables to the Salesforce plugin.
Step 5: IP Relaxation (optional) #
Depending on where you run CloudQuery your might need to change
IP Relxation
to Relax IP restrictions
or configure the IP address of the machine you run CloudQuery on.Licenses #
The following tools / packages are used in this plugin:
Name | License |
---|---|
github.com/adrg/xdg | MIT |
github.com/apache/arrow/go/v13 | Apache-2.0 |
github.com/apache/arrow-go/v18 | Apache-2.0 |
github.com/apapsch/go-jsonmerge/v2 | MIT |
github.com/aws/aws-sdk-go-v2 | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/config | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/credentials | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/feature/ec2/imds | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/configsources | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/ini | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/internal/sync/singleflight | BSD-3-Clause |
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/licensemanager | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/marketplacemetering | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/sso | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/ssooidc | Apache-2.0 |
github.com/aws/aws-sdk-go-v2/service/sts | Apache-2.0 |
github.com/aws/smithy-go | Apache-2.0 |
github.com/aws/smithy-go/internal/sync/singleflight | BSD-3-Clause |
github.com/bahlo/generic-list-go | BSD-3-Clause |
github.com/buger/jsonparser | MIT |
github.com/cenkalti/backoff/v4 | MIT |
github.com/cloudquery/cloudquery-api-go | MPL-2.0 |
github.com/cloudquery/plugin-pb-go | MPL-2.0 |
github.com/cloudquery/plugin-sdk/v2/internal/glob | MIT |
github.com/cloudquery/plugin-sdk/v2/schema | MIT |
github.com/cloudquery/plugin-sdk/v2/types | MPL-2.0 |
github.com/cloudquery/plugin-sdk/v4 | MPL-2.0 |
github.com/cloudquery/plugin-sdk/v4/glob | MIT |
github.com/cloudquery/plugin-sdk/v4/scalar | MIT |
github.com/davecgh/go-spew/spew | ISC |
github.com/ghodss/yaml | MIT |
github.com/go-logr/logr | Apache-2.0 |
github.com/go-logr/stdr | Apache-2.0 |
github.com/goccy/go-json | MIT |
github.com/google/flatbuffers/go | Apache-2.0 |
github.com/google/uuid | BSD-3-Clause |
github.com/gorilla/mux | BSD-3-Clause |
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors | Apache-2.0 |
github.com/grpc-ecosystem/grpc-gateway/v2 | BSD-3-Clause |
github.com/hashicorp/go-cleanhttp | MPL-2.0 |
github.com/hashicorp/go-retryablehttp | MPL-2.0 |
github.com/invopop/jsonschema | MIT |
github.com/klauspost/compress | Apache-2.0 |
github.com/klauspost/compress/internal/snapref | BSD-3-Clause |
github.com/klauspost/compress/zstd/internal/xxhash | MIT |
github.com/mailru/easyjson | MIT |
github.com/mattn/go-colorable | MIT |
github.com/mattn/go-isatty | MIT |
github.com/oapi-codegen/runtime | Apache-2.0 |
github.com/pierrec/lz4/v4 | BSD-3-Clause |
github.com/pmezard/go-difflib/difflib | BSD-3-Clause |
github.com/rs/zerolog | MIT |
github.com/samber/lo | MIT |
github.com/santhosh-tekuri/jsonschema/v6 | Apache-2.0 |
github.com/spf13/cobra | Apache-2.0 |
github.com/spf13/pflag | BSD-3-Clause |
github.com/stretchr/testify | MIT |
github.com/thoas/go-funk | MIT |
github.com/wk8/go-ordered-map/v2 | Apache-2.0 |
github.com/zeebo/xxh3 | BSD-2-Clause |
go.opentelemetry.io/otel | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlptrace | Apache-2.0 |
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp | Apache-2.0 |
go.opentelemetry.io/otel/log | Apache-2.0 |
go.opentelemetry.io/otel/metric | Apache-2.0 |
go.opentelemetry.io/otel/sdk | Apache-2.0 |
go.opentelemetry.io/otel/sdk/log | Apache-2.0 |
go.opentelemetry.io/otel/sdk/metric | Apache-2.0 |
go.opentelemetry.io/otel/trace | Apache-2.0 |
go.opentelemetry.io/proto/otlp | Apache-2.0 |
golang.org/x/exp | BSD-3-Clause |
golang.org/x/net | BSD-3-Clause |
golang.org/x/oauth2 | BSD-3-Clause |
golang.org/x/sync | BSD-3-Clause |
golang.org/x/sys | BSD-3-Clause |
golang.org/x/text | BSD-3-Clause |
golang.org/x/xerrors | BSD-3-Clause |
google.golang.org/genproto/googleapis/api/httpbody | Apache-2.0 |
google.golang.org/genproto/googleapis/rpc/status | Apache-2.0 |
google.golang.org/grpc | Apache-2.0 |
google.golang.org/protobuf | BSD-3-Clause |
gopkg.in/yaml.v2 | Apache-2.0 |
gopkg.in/yaml.v3 | MIT |