DataSync for Snowflake Release Notes

Add batch size configuration to better avoid "String object has exceeded maximum permitted size" error in ServiceNow.

Load custom fields added to local schema files with useLocalSchema feature when the meshlet is restarted.

Optimize to use SELECT LIMIT 1 statement as part of querying if table exists in Snowflake database prior to taking action.

Updated to use CREATE TABLE to prevent overwriting tables that already exists in Snowflake database and handle response if table exists to continue processing records. Added CREATE TABLE to databaseConfig.json configuration file to allow updating of configuration outside of meshlet.

Handle messages with empty or undefined value field without looping.

Add loadSchemasOnStart configuration to create table in Snowflake database from local schemas only when a record for the table is received.

Support OAuth for accessing ServiceNow instance to get table schema.

Added additional recovery logic if Meshlet's Rabbit listener to MBS queue goes idle.

Meshlets support for Java 18

Support filtering messages to only read messages from one instance (key)

Support DB views to skip null values for the compound sys_id and order *_sys_id values in alphabetical order.

Fix for processing message and improperly pushing to error queue.

Extend column to max size if content is larger than current size. Requires the extendColumnsMaxLength configuration.

Log which column is being truncated when insert/update a record into Snowflake and the data is too large for the column. Requires the truncateColumns configuration.

Fix for Numeric value 'true' is not recognized , Numeric value 'false' is not recognized error when translating boolean values for newly adding custom columns using the translateBooleans configuration.

Fix to truncate text if its over a column's length and log if record fails being inserted/updated in Snowflake.

Fields such as password can now be encrypted for the meshlet using the meshlet install script.

Snowflake DataSync meshlet now supports the mapping of Boolean type to Number.

When quotedColumns is configured to false, adding columns in ServiceNow will now get reflected in Snowflake. 

You can now add custom columns in your Snowflake tables with configuring the databasesConfig.json file. To read more information, see add custom fields.

Snowflake meshlet now supports using table compare.

Snowflake meshlet now supports local schema files.

Snowflake meshlet now supports using Microsoft Azure as an external stage for bulk loading. 

Snowflake meshlet supports soft deletes (updating fields to indicate the record is deleted and leaving the record intact in the database table).

Meshlets now support reading multibyte_encrypted messages.

Snowflake meshlet recovers by recreating the table when the table is dropped while the Meshlet is running and processing messages.

Specifying the maximum string size for the entire collection is available through maxStringSize key. See Meshlet Configurations for Snowflake.

Adding parameters to your connectionUrl to leverage other forms of authentication is available through useConnectionProperties key. See Meshlet Configurations for Snowflake.