Versions Compared

Old Version 8

changes.mady.by.user user-b885f

Saved on

compared with

New Version Current

changes.mady.by.user user-1bc08

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


HTML
<style>
.release-box {
	height: 30px; 
	width: 100px; 
	padding-top: 8px;
	text-align: center; 
	border-radius: 5px; 
	font-weight: bold; 
	background-color: #d4af37;  
	border-color: #FCE28A;
}

.release-box:hover {
  	cursor: hand;
    cursor: pointer;
    opacity: .9; 
}
</style>
<meta name="robots" content="noindex">

<div class="release-box">
<a href="https://docs.perspectium.com/display/gold" style="text-decoration: none; color: #FFFFFF; display: block;">
Gold
</a>
</div>



To enhance your DataSync integration for Snowflake, you can configure the Snowflake Bulk Load Meshlet to the directives listed below:

Warning
iconfalse
titleNOTE:
  • If you will be setting up the DataSync Agent to also connect to the same database, you will want to configure the Agent to use the same column cases. By default, the Snowflake Bulk Load Meshlet uses lowercase for the column names. Also, the QUOTED_IDENTIFIERS_IGNORE_CASE setting in Snowflake should be kept with its default value of false.
  • The meshlet uses the TIMEZONE session parameter to connect to Snowflake in order to save all timestamps in UTC . By default, glide_date_time field is mapped to the TIMESTAMP_LTZ(9) column type.
  • This allows you to query using a local timezone as needed. 



Key
Directive
Default ValueDescription

maxFileSize

15000

Total of individual records (per table) for each file. After x records have been received by the meshlet, a file will be created with x records and an immediate push of all records in that file to Snowflake will occur


Required Configuration. This configuration specifies the max size for temporary files created as the meshlet pushes records to Snowflake. 10000 will be used if input is over 10000 to prevent possible performance and memory issues. A suggested value is 5000.

Code Block
languageyml
perspectium:
	filesubscriber: 
		maxFileSize:
15000
5000
buildHeader


maxStringSize

false

Each CSV file will include headers of column names in the first row.
500

Specifies the maximum string size for the entire collection. The maximum value you can use is 1000, if input is greater than that value, the input will default to 1000. 

Code Block
languageyml
perspectium:
	filesubscriber:
buildHeader
maxStringSize:
false
500


customFileName

$table-$randomid

Names file with format of table - random id. File names MUST be unique.

Code Block
languageyml
perspectium:
	filesubscriber: 
		customFileName: $table-$randomid


fileDirectory

/files

Directory where the locally created files get made. (In respects to where application is running)

Code Block
languageyml
perspectium:
	filesubscriber: 
		fileDirectory: /files


postInterval

2

Minutes to check dead periods. Check every x minutes to compare if the in memory collection is the same as the last x minutes. If so, write records to file and push to Snowflake

Code Block
languageyml
perspectium:
	snowflake: 
		postInterval: 2


deleteFiles

true

Indicates whether you want to keep or delete locally created CSV files. Will not have a noticeable performance hit.

Code Block
languageyml
perspectium:
	filesubscriber: 
		deleteFiles: true


fileCleanerInterval

4

How

many hours for a job

often (in minutes) the file cleaner job will run to clean up local files

. (Like ServiceNow DataCleaner Scheduled Job)

created by the meshlet. This job is similar to the ServiceNow Data Cleaner Scheduled Job. For example, value of 4 will run the job every four minutes. The default value is 1.

Code Block
languageyml
perspectium:
	filesubscriber: 
		fileCleanerInterval: 4


fileMaxAge

1

Any csv file in the filesDirectory older than fileMaxAge in

hours

minutes, will be automatically deleted. The default value is 1.

Code Block
languageyml
perspectium:
	filesubscriber: 
		fileMaxAge: 1


legacyDbViewIdfalse

Derivative to use the legacy version for database view tables of concatenating GUID values into a sys_id field. If false, meshlet will use the pre-constructed encoded sys_id created by ServiceNow.

Code Block
languageyml
perspectium:
	filesubscriber: 
		legacyDbViewId: false


useReceiptsfalse

Enable data guarantee.

Code Block
languageyml
perspectium:
	message: 
		receipts:
			enable: false


receiptOutboundQueue

Only Required if useReceipts is true. Queue receipts are returned to, typically set to the default subscribed queue of the source ServiceNow instance sharing records.

Code Block
languageyml
perspectium:
	message: 
		receiptOutboundQueue: psp.out.servicenow.dev12345





Similar topics

Content by Label
showLabelsfalse
max5
showSpacefalse
sortmodified
cqllabel = "datasync-for-snowflake" and space = currentSpace()

Contact Perspectium Support

US: 1 888 620 8880

UK: 44 208 068 5953

support@perspectium.com