DataSync Agent Release Notes

Added File DataSync option for DataSync Type in the Agent installer to support manual configuration for file subscribing. 

Additional logging statements added to notify users that the truncate command executed successfully in the Agent. 

Enhanced handling on display value fields for glide_list, glide_duration and choice fields in the Table Schema (compatible with Core 6.1.0). 

Added logs to alert user when record payloads have fewer columns than database table. 

Set a dynamic file name using <file_prefix>$table_$d{yyyyMMdd}_$i</file_prefix> upon setting up DataSync Agent to share to AWS S3. 

Validate Configuration will now validate the encryption keys set for the DataSync Agent. 

Use the update_column_only tag in the IO Datetime Plugin for DataSync Agent to update the psp_per_update_dt column only when records are updated. 

Improved logging details for SQLRecoverableException errors. 

AgentConfigValidator Error in logging for DataSync Agent will no longer appear.

CSV file is no longer missing a header for CSV File Subscriber. 

Sharing all records to one file with the agent will no longer need the <separate_files> tag. 

Agent will now retry messages if statements timeout.

Fixed Perspectium Support link in Agent installer for Manual Configuration.

Fix NullPointerException that may occur when Agent is connected to Oracle for a long duration.

Upgrade to commons-text-1.10.jar to fix CVE-2022-42889.

Support for Java 18

 Agent will no longer get stuck in a loop after receiving a truncate message.

Re-queued delete messages will no longer throw decryption error. 

Enhance performance for index checking in the agent. 

IODateTimePlugin inserts will no longer have a null insert time. 

API plugins will not be disabled on failure and will retry until services are restored.

Table compare now supports the merged attachments table for fully built attachments.

IODateTimePlugin now supports the merged attachments table for fully built attachments.

You can now specify how many times an attachment will be tried to be built until it's skipped and removed from the sys_attachment table with the <attachment_retries> tag. 

Enhanced error handling for Voltage Encryption failures. 

Azure KeyVault is now accessible with custom wrapper API. 

Dynamic insert datetime columns will now have the same date/time that the record is first received by the Agent on insert and remain the same when the Agent receives an update of the same record.

Journal field entries will now be encrypted with the Voltage plugin in DataSync Agent. 

You can now choose which SQL Server schema to use in the DataSync Agent with the <database_schema> directive.

When using Voltage for encrypting your data, the host_name parameter within the plugin directive will be the name of the computer or machine that the agent is running on. 

For decrypting Edge Encrypted content, you can now access Azure KeyVault through a proxy. 

Enable Voltage encryption with the com.perspectium.replicator.sql.plugin.SQLSubscriberVoltageEncrypterPlugin plugin. 

With Voltage encryption, you can specify trace ID as a parameter to track your API traffic. 

Attachments can now be encrypted with Voltage. 

Edge Decryption plugin is now available for DataSync Agent to handle Edge Encrypted values in your ServiceNow instance. 

Attachments can now be processed by the DataSync Agent with SysAttachmentHandler. 

Use <custom_primary_key/>  to create an auto-incrementing column and set the column as the primary key. 

Use <batch_flush_interval> for the number of seconds that will trigger a batch SQL statement execution with batch processing.

Use <skip_performance_tag> to stop the Agent from reporting performance metrics (statsx) to the Integration Mesh. 

Enable truncating table in the database by the DataSync Agent using Truncate Before Share.

<use_instance_credentials/> is now available to use instead of access keys in AWS S3 Subscriber Agent to use IAM roles when uploading to the S3 buckets.

Customize your temporal database table's archive date and time stamp in the psp_valid_to column with <archive_datetime/>. 

Specify the character set for database encoding and truncate the data based on this character set by using <database_truncate_encoding>. 

Added PostgresSQL as a DataSync Type option in the Agent installer.

The DataSync Agent Installer will now use HTTPS as a protocol by default in <message_connection>. 

Snowflake DataSync Agent will automatically have parameter values to default to uppercase characters after installation. 

Choosing Manual Configurations in the DataSync Agent Installer will prompt you to contact Perspectium Support for a guided step. 

If the agent.xml file is invalid (i.e. bad file format, missing field characters, etc),  an error message will now display on the log with a description of the error.

Match text field lengths in your DataSync Agent with <match_field_lengths/>.

 Check the fields values if the updated time is the same from the previous share in your DataSync Agent with <skip_duplicates check_all_fields=”true”/> .

Using <column_sort>alpha</column_sort> in your DataSync Agent will now sort new fields. 

When a table already exists upon an integration, the new reference fields will now have length of 32.

<enable_error_log/> will now keep track of the number of invalid records and save the error count in the generated audit file. 

<close_file_interval> is added to set the amount of time the Agent will wait for tmp files to be modified. 

Enhanced error recovery when using temporal and the table does not exist in the database.

AWS S3 Subscriber Agent now supports sending receipts back to ServiceNow. 

The <region> directive in the AWS S3 Subscriber Agent is now optional and no longer required. 

AWS S3 Subscriber Agent logging extended for different levels. 

Using <update_on_bulk/> will now skip duplicates in the DataSync Agent based on the message action.

The agent will now log a warning message that a queue will be created when messages are shared if the agent gets a 404 (queue not found) status.

DataSync Agent now supports Java 14.

DataSync Agent now supports Java 15.

The DataSync Agent Installer will now allow you to enter FedRAMP URLs for your ServiceNow instance information. 

The action of processing delete messages will not alter the database tables.

Agent re-queues delete records up to 3 times when no record in database to delete.

Agents installed using the installer will create a conf/agent.xml with instances valued to 1 instead of 4 to minimize data loss.

Sharing records from a database to the Integration Mesh will now be consumed without errors. 

Set your MySQL database with utf8mb4 character set and collation to accept multibyte characters in the agent.xml with <database_table_character_set> and <database_table_collation>. 

Error when inserting/updated a record into a SAP HANA table that has 2 billion+ rows because of metadata query.

The attribute, use_basic_consume, in the agent.xml will reflect on the protocol given upon the configuration in DataSync Agent installer.

Emojis will no longer be removed in incoming messages to the DataSync Agent when at lease one custom schema feature is used (tablename, column_name, _dv suffix). 

After deleting a field from a ServiceNow table, the old value of that field will no longer be inserted into the database. 

After adding a string field with default value in a ServiceNow table, executing a bulk share will no longer generate a timestamp error. 

Table stats data will now show the correct timezone offset. 

Using skip_duplicate with check_all_fields will now compare values properly.

Removed the following DataSync Type options in the DataSync Agent Installer that are no longer being used: Rabbit MQ and Other.