To enhance your DataSync integration for Snowflake, you can configure the Snowflake Bulk Load Meshlet to the directives listed below:
Directive | Default Value | Description |
---|---|---|
maxFileSize | 15000 | Total of individual records (per table) for each file. After x records have been received by the meshlet, a file will be created with x records and an immediate push of all records in that file to Snowflake will occur. perspectium: filesubscriber: maxFileSize: 15000 |
customFileName | $table-$randomid | Names file with format of table - random id. File names MUST be unique. perspectium: filesubscriber: customFileName: $table-$randomid |
fileDirectory | /files | Directory where the locally created files get made. (In respects to where application is running) perspectium: filesubscriber: fileDirectory: /files |
postInterval | 2 | Minutes to check dead periods. Check every x minutes to compare if the in memory collection is the same as the last x minutes. If so, write records to file and push to Snowflake perspectium: snowflake: postInterval: 2 |
deleteFiles | true | Indicates whether you want to keep or delete locally created CSV files. Will not have a noticeable performance hit. perspectium: filesubscriber: deleteFiles: true |
fileCleanerInterval | 4 | How many minutes for a job to clean up local files. (Like ServiceNow DataCleaner Scheduled Job) perspectium: filesubscriber: fileCleanerInterval: 4 |
fileMaxAge | 1 | Any csv file in the filesDirectory older than fileMaxAge in minutes, will be automatically deleted. perspectium: filesubscriber: fileMaxAge: 1 |
legacyDbViewId | false | Derivative to use the legacy version for database view tables of concatenating GUID values into a sys_id field. If false, meshlet will use the pre-constructed encoded sys_id created by ServiceNow. perspectium: filesubscriber: legacyDbViewId: false |
useReceipts | false | Enable data guarantee. perspectium: message: receipts: enable: false |
receiptOutboundQueue | Queue receipts are returned to, typically default subscribed to on a ServiceNow instance. perspectium: message: receiptOutboundQueue |
Similar topics
There is no content with the specified labels