DataSync for Snowflake is a single-direction (ServiceNow to Snowflake) integration that Bulk Shares and Dynamic shares to synchronize data for the collection of ServiceNow data into a Snowflake data warehouse.
WARNING:
- If you will be setting up the DataSync Agent to also connect to the same database, you will want to configure the Agent to use the same column cases. By default, the Snowflake Bulk Load Meshlet uses lowercase for the column names. Also, the QUOTED_IDENTIFIERS_IGNORE_CASE setting in Snowflake should be kept with its default value of false.
- The meshlet uses the TIMEZONE session parameter to connect to Snowflake in order to save all timestamps in UTC . By default, glide_date_time field is mapped to the TIMESTAMP_LTZ(9) column type. This allows you to query using a local timezone as needed.
Prerequisites
Request the databaseConfig.json file for your meshlet by contacting Perspectium Support.
Database, Warehouse, and Schema will not be created by the meshlet and are expected to be supplied on use.
The meshlet connects to Snowflake using a JDBC driver with a specified <user> and <password>. The user must have the role of SYSADMIN or a custom role of equivalent privileges in order to create objects (tables) and write records.
See Snowflake's list of default roles here.
Related to the meshlet's minimum database user requirements, the following Snowflake access control privileges are needed by the Snowflake role/user configured in the meshlet:
| Object | Privileges |
|---|---|
| Virtual Warehouse | USAGE |
| Database | USAGE |
| Schema* | USAGE, CREATE TABLE, CREATE FILE FORMAT, CREATE STAGE |
| Table^ | SELECT, INSERT, UPDATE, DELETE, TRUNCATE |
| Stage | WRITE |
| File Format | USAGE |
*The meshlet will executed the SHOW <objects> command. As long as the role/user has one privilege on the Schema (and it needs the ones listed above), then it will have the ability to execute this command.
^The meshlet will use the ALTER TABLE command to add new columns to tables when a new column is created in ServiceNow. The INSERT privilege allows executing this command. The meshlet uses MERGE for inserting, updating and deleting records from a table and having the above privileges allows it to run this command.
Set up a DataSync for Snowflake Integration
In the Databases page in Snowflake, click Create... to create a new database. Then, fill in the Name field and click Finish.
Create a ServiceNow bulk/dynamic share for Snowflake
To enable the sharing of ITSM data from your ServiceNow instance to your Snowflake instance, you will need to create a ServiceNow bulk share and/or dynamic share. A ServiceNow bulk share is a one-time transfer of data from your ServiceNow instance. A ServiceNow dynamic share allows for real time sharing of ServiceNow records as they are created, updated, and deleted. Creating a bulk share and/or dynamic share will share out your ServiceNow data immediately as well as share out subsequent data that is collected as ServiceNow records are created, updated, and deleted.
Prerequisites
You will first need to install the Perspectium Core update set. You can request a download link for this update set by contacting Perspectium Support.
You will also need to have a ServiceNow shared queue that points to your Snowflake instance. This queue should have been created by Perspectium Support upon initial configuration of your ServiceNow instance.
Procedure
To create a bulk share, follow these steps:
Go to your ServiceNow instance. Then, navigate to Perspectium > DataSync > Bulk Share or simply type Bulk Share in the Filter Navigator on the upper left-hand side of the screen.
Click the New button next to Bulk Shares. In the resulting form's Cipher dropdown, select an encryption type to be used. Then, follow the remaining steps to create a ServiceNow bulk share for DataSync for ServiceNow (DataSync).
To create a dynamic share, follow these steps: