New
Join our webinar! Building a customizable and extensible cloud asset inventory at scale
Back to source list
salesforce
Official
Premium

Salesforce

The CloudQuery Salesforce plugin extracts information from your Salesforce organization(s) and loads it into any supported CloudQuery destination

Publisher

cloudquery

Latest version

v5.2.4

Type

Source

Platforms
Date Published

Price per 1M rows

Starting from $17

monthly free quota

1M rows

Overview #

The CloudQuery Salesforce plugin extracts information from your Salesforce organization(s) and loads it into any supported CloudQuery destination (e.g. PostgreSQL, BigQuery, Snowflake, and more).

Authentication #

You will need to create a Salesforce Connected App using the OAuth2.0 Client Credentials flow. This is a Salesforce app that will be used to authenticate with Salesforce. You can create a Connected App in the Salesforce Setup UI. Checkout the Salesforce documentation for more information. Or check out our latest guide to set up a connected app.

Configuration #

This example syncs from Salesforce to a Postgres destination:
kind: source
spec:
  name: slack
  path: cloudquery/salesforce
  registry: cloudquery
  version: "v5.2.4"
  tables: ["*"]
  destinations: ["postgresql"]
  # Salesforce specific configuration
  # Learn more about the configuration options at https://cql.ink/salesforce_source
  spec:
    domain_name: "${SF_DOMAIN_NAME}" // e.g. my-domain
    client_id: "${SF_CLIENT_ID}"
    client_secret: "${SF_CLIENT_SECRET}"
It assumes that the SF_DOMAIN_NAME, SF_CLIENT_ID and SF_CLIENT_SECRET environment variables are set.

Configuration #

This is the (nested) spec used by the Slack source plugin.
  • domain_name (string) (required)
    This is the domain name of your Salesforce instance. It will be used to build the instance endpoint to which calls will be sent. For example, if your Salesforce instance is https://my-domain.my.salesforce.com, then the domain name is my-domain.
  • client_id (string) (required)
    This is the consumer key of the connected app. This can be obtained by Creating a Salesforce connected app.
  • client_secret (string) (required)
    This is the consumer secreted of the connected app. This can be obtained by Creating a Salesforce connected app.
  • include_objects ([]string) (optional)
    By default the plugin will sync all Salesforce objects. If you want to limit the objects that are synced you can specify the objects you want to sync in this array.
  • exclude_objects ([]string) (optional)
    By default the plugin will sync all Salesforce objects. If you want to exclude some objects from being synced you can specify the objects you want to exclude in this array.
  • concurrency (integer) (optional) (default: 10000)
    A best effort maximum number of Go routines to use. Lower this number to reduce memory usage.
  • scheduler (string) (optional) (default: dfs) The scheduler to use when determining the priority of resources to sync. Supported values are dfs (depth-first search), round-robin, shuffle and shuffle-queue.
    For more information about this, see performance tuning.

Example Queries #

List all Salesforce objects #

select distinct(object_type) from salesforce_objects;

List all Accounts #

select * from salesforce_objects where object_type = 'Account';

Creating Connected Guide #

Full documentation is available here. This guide will walk you through the steps to create a connected app.

Pre-requisites #

  • Salesforce developer account to be able to access the REST APIs
  • A Salesforce user with the permissions to create connected a

Step 1: Go To Connected App #

Go to Setup Screen and search for "App Manager" and click on "New Connected App".

Step 2: Create Connected App #

Fill in the fields as the following and click save:

Step 3: Set up an execution user for the Client Credentials Flow #

Salesforce uses the OAuth 2.0 Client Credentials Flow to authenticate the connected app. This flow requires a user to execute the flow. You can use the same user that created the connected app or create a new user (this approach is more secure as you have the possibility to create an API-only user).
Go to "Apps" > "Connected Apps" > "Manage Connected Apps" > "Edit Policies" and under the "Client Credentials flow" section pick the "Run As" user of your choice.

Step 4: Copy Consumer Key and Consumer Secret #

Now that this is configured click Manage Consumer Details to view your Consumer Key and Consumer Secret (those will be used as client_id and client_secret to configure your CloudQuery Salesforce source plugin).
And copy the Consumer Key and Consumer Secret somewhere safe so you can provide them as environment variables to the Salesforce plugin.
 

Step 5: IP Relaxation (optional) #

Depending on where you run CloudQuery your might need to change IP Relxation to Relax IP restrictions or configure the IP address of the machine you run CloudQuery on.


Licenses #

The following tools / packages are used in this plugin:
NameLicense
github.com/adrg/xdgMIT
github.com/apache/arrow/go/v13Apache-2.0
github.com/apache/arrow-go/v18Apache-2.0
github.com/apapsch/go-jsonmerge/v2MIT
github.com/aws/aws-sdk-go-v2Apache-2.0
github.com/aws/aws-sdk-go-v2/configApache-2.0
github.com/aws/aws-sdk-go-v2/credentialsApache-2.0
github.com/aws/aws-sdk-go-v2/feature/ec2/imdsApache-2.0
github.com/aws/aws-sdk-go-v2/internal/configsourcesApache-2.0
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/iniApache-2.0
github.com/aws/aws-sdk-go-v2/internal/sync/singleflightBSD-3-Clause
github.com/aws/aws-sdk-go-v2/service/internal/accept-encodingApache-2.0
github.com/aws/aws-sdk-go-v2/service/internal/presigned-urlApache-2.0
github.com/aws/aws-sdk-go-v2/service/licensemanagerApache-2.0
github.com/aws/aws-sdk-go-v2/service/marketplacemeteringApache-2.0
github.com/aws/aws-sdk-go-v2/service/ssoApache-2.0
github.com/aws/aws-sdk-go-v2/service/ssooidcApache-2.0
github.com/aws/aws-sdk-go-v2/service/stsApache-2.0
github.com/aws/smithy-goApache-2.0
github.com/aws/smithy-go/internal/sync/singleflightBSD-3-Clause
github.com/bahlo/generic-list-goBSD-3-Clause
github.com/buger/jsonparserMIT
github.com/cenkalti/backoff/v4MIT
github.com/cloudquery/cloudquery-api-goMPL-2.0
github.com/cloudquery/plugin-pb-goMPL-2.0
github.com/cloudquery/plugin-sdk/v2/internal/globMIT
github.com/cloudquery/plugin-sdk/v2/schemaMIT
github.com/cloudquery/plugin-sdk/v2/typesMPL-2.0
github.com/cloudquery/plugin-sdk/v4MPL-2.0
github.com/cloudquery/plugin-sdk/v4/globMIT
github.com/cloudquery/plugin-sdk/v4/scalarMIT
github.com/davecgh/go-spew/spewISC
github.com/ghodss/yamlMIT
github.com/go-logr/logrApache-2.0
github.com/go-logr/stdrApache-2.0
github.com/goccy/go-jsonMIT
github.com/google/flatbuffers/goApache-2.0
github.com/google/uuidBSD-3-Clause
github.com/gorilla/muxBSD-3-Clause
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptorsApache-2.0
github.com/grpc-ecosystem/grpc-gateway/v2BSD-3-Clause
github.com/hashicorp/go-cleanhttpMPL-2.0
github.com/hashicorp/go-retryablehttpMPL-2.0
github.com/invopop/jsonschemaMIT
github.com/klauspost/compressApache-2.0
github.com/klauspost/compress/internal/snaprefBSD-3-Clause
github.com/klauspost/compress/zstd/internal/xxhashMIT
github.com/mailru/easyjsonMIT
github.com/mattn/go-colorableMIT
github.com/mattn/go-isattyMIT
github.com/oapi-codegen/runtimeApache-2.0
github.com/pierrec/lz4/v4BSD-3-Clause
github.com/pmezard/go-difflib/difflibBSD-3-Clause
github.com/rs/zerologMIT
github.com/samber/loMIT
github.com/santhosh-tekuri/jsonschema/v6Apache-2.0
github.com/spf13/cobraApache-2.0
github.com/spf13/pflagBSD-3-Clause
github.com/stretchr/testifyMIT
github.com/thoas/go-funkMIT
github.com/wk8/go-ordered-map/v2Apache-2.0
github.com/zeebo/xxh3BSD-2-Clause
go.opentelemetry.io/otelApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttpApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttpApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptraceApache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttpApache-2.0
go.opentelemetry.io/otel/logApache-2.0
go.opentelemetry.io/otel/metricApache-2.0
go.opentelemetry.io/otel/sdkApache-2.0
go.opentelemetry.io/otel/sdk/logApache-2.0
go.opentelemetry.io/otel/sdk/metricApache-2.0
go.opentelemetry.io/otel/traceApache-2.0
go.opentelemetry.io/proto/otlpApache-2.0
golang.org/x/expBSD-3-Clause
golang.org/x/netBSD-3-Clause
golang.org/x/oauth2BSD-3-Clause
golang.org/x/syncBSD-3-Clause
golang.org/x/sysBSD-3-Clause
golang.org/x/textBSD-3-Clause
golang.org/x/xerrorsBSD-3-Clause
google.golang.org/genproto/googleapis/api/httpbodyApache-2.0
google.golang.org/genproto/googleapis/rpc/statusApache-2.0
google.golang.org/grpcApache-2.0
google.golang.org/protobufBSD-3-Clause
gopkg.in/yaml.v2Apache-2.0
gopkg.in/yaml.v3MIT


Join our mailing list

Subscribe to our newsletter to make sure you don't miss any updates.

Legal

© 2024 CloudQuery, Inc. All rights reserved.

We use tracking cookies to understand how you use the product and help us improve it. Please accept cookies to help us improve. You can always opt out later via the link in the footer.