The file that controls the configuration settings for Perspectium DataSync Agent is the agent.xml file. This file is automatically created within the Agent's bin directory upon installation. To make changes to the configuration settings for your DataSync Agent, open this agent.xml file in a text editing application and modify the directives described in the sections below.

What's on this page?

This is NOT a finalized list. We are currently in the process of including all of the directives in this list. 



Agent Level

The <agent> is the second highest level next to <config>. The agent level holds the job types to be executed (i.e <share> and <subscribe>), as well as optional directives for additional configurations. These directives will then contain their own configurations within. 


DirectiveDescriptionExample Value
<max_reads_per_connect>Maximum number of messages to consumer per connection interval.4000
<max_writes_per_connect>Maximum number of messages to produce per connection interval.4000

<polling_interval>

Interval for how often the agent pulls from the queue (seconds).

(info) NOTEThe value needs to be 5 seconds or greater. If the value is set to less than 5 seconds, it will default to 5 seconds.

5
<skip_message_set_processing/>

Self-closing tag that disables Message Set posting for data shared from ServiceNow.


<skip_table_stats/> Self-closing tag that disables tracking of changes to replicated tables. Without this directive, the changes would be found in psp_table_stats tables in the psp_db table on MBS. For earlier versions (Fluorine+ and below), this directive is required for Snowflake database.
<share>An Agent configured with <share> functionality (also known as a producer) monitors static data sources and places messages from those sources into queues in the Perspectium Mesh.
<subscribe>

An Agent configured with <subscribe> functionality (also known as a consumer) monitors queues in the Perspectium Mesh and processes any messages in those queues, typically by routing those messages to a static data source.

(info) NOTE: This is the more common use case for the DataSync Agent.


<skip_report/>Self-closing tag that causes the agent to skip sending reporting messages.




Subscribe & Share Level

There are two types of handlers or tasks that can be configured within the <agent> tag, the subscriber <subscribe> and the sharer <share>. The subscribe or share level will have zero or more nested <task> elements. However, the DataSync Agent will always have at least one task to perform the replication.


DirectiveDescription
<task>

Task that handles either sharing or subscribing. You can configure separate settings by defining directives within each <task> directive. You can also specify an instances attribute here and give it a value of 4 (so that 4 instances of your <share> or <subscribe> task can run concurrently, thus improving throughput for data processing.

You do not need to enter a value within the <task> directive. Nest all configuration directives noted below within your each of your <task> directives.

(info) NOTE: It is not recommended to configure multiple instances (you should leave the default value of instances="1") if you have multiple updates against the same records in this queue as this will cause data discrepancies. This is due to the multiple instances running concurrently so that updates may be applied out of order depending on which instance finishes processing first.




Task Level 

The <task> level specifies the configurations needed for connecting to the instance, server, and database along with optional directives that allows for more control and configurations. 

DirectiveDescriptionExample Value
<task_name>

Name for your <task>

(info) NOTE: Only alphanumeric characters, numbers, and underscores are permitted as values for <task_name>.

datasyncforservicenow_to_mysql_subscribe
<handler>Name of the Java handler class. In most cases, the value for this directive will be com.perspectium.replicator.sql.SQLSubscriber.com.perspectium.replicator.sql.SQLSubscriber
<encryption_key>

Encryption key used to encrypt data being shared from a static data source to an application


<decryption_key>

Decryption key used to decrypt data being shared from an application


<receipt_send_interval>

(Optional) Indicates the number of seconds in which the Agent will regularly send receipts back to the sharing/publishing instance.

(info) NOTE: This directive defaults to 1800.

1800
<ack_batch_size>

(Optional) Can be used to specify the maximum size of acknowledged receipt messages per batch. 

(info) NOTE: This directive defaults to 1000 

1000

<error_batch_size>

(Optional) Can be used to specify the maximum size of error receipt messages per batch. 

(info) NOTE: This directive defaults to 1000 

1000
<database_type>

Type of database you are syncing your data to. Supported databases and associated values for the <database_type>:

Supported databaseValue to enter for <database_type>
Oracleoracle
MS SQL Serversqlserver
MySQLmysql
SAP Hanahana
Amazon Redshiftredshift
IBM DB2db2
HP Verticavertica
PostgreSQLpostgresql

<database_server>

URL for your database. If your database is installed on your local machine, set this value to localhost.


<database_port>Port number for your database
<database_user>Username used to connect to your database
<database_password>Password used to connect to your database
<database_parms>

(Optional) Can be used to pass additional parameters to the database server during establishment of the connection

For MySQL, this directive will have a default value of autoReconnect=true & characterEncoding=UTF-8 & useSSL=false.

(info) NOTE: It is recommended that spaces are added between each parameter. Failing to do this may cause the agent to not connect to MySQL properly.


<database>Name of your database
<database_column_max_size>Maximum size of an object that can appear in your database's columns
<database_static_column_size>

Prevents altering the size of an object in your database's columns

(warning) WARNING: This directive may cause data to be truncated 


<database_table_character_set>(Optional) Sets the MySQL database character set to utf8mb4 to accept multibyte characters CHARACTER SET utf8mb4
<database_table_collation>(Optional) Sets the MySQL database collation to utf8mb4 to accept multibyte characters COLLATE utf8mb4_unicode_ci
<database_sid>

SID used for your Oracle database

(info) NOTE: This directive is required for DataSync integrations to an Oracle database only.


<byte_padding>

(Optional) Can be used for multi-byte characters by multiplying the size of the columns. Adding an integer to this directive will create columns times larger.

(info) NOTE:

3
<column_sort>

Sorts your database's columns in a specified order.

(info) NOTE:

  • Currently, this directive only supports alphabetical order.  Value should be set to alpha
  • Temporal columns will always be first. 
alpha
<sql_retries>

(Optional) Can be used to configure the number of times the DataSync agent will retry after a SQLException. 

(info) NOTE: This directive defaults to 3.

250
<sql_retries_delay>

(Optional) Can be used to configure how many milliseconds the DataSync agent will wait between retries.

(info) NOTE: This directive defaults to 250.


<skip_database_creation/>Self-closing tag that stops the agent from creating a new database. Use this only if you are replicating to an existing database.
<ensure_table_exists/>Self-closing tag that is required for replicating to Snowflake database. 
<match_field_lengths/> Self-closing tag that matches text field lengths. 
<batch_insert/>Self-closing tag that configures your Agent to batch SQL INSERT statements.
<batch_update/>Self-closing tag that configures your Agent to batch SQL UPDATE statements.
<batch_flush_interval>Number of seconds that will trigger a batch SQL statement execution. Acceptable values are integers from 1 to 30 (seconds).20
<max_batch_size>Number of SQL statements that will trigger a batch SQL statement execution.  A larger suggested value is 200. 200
<skip_performance_report>

When enabled, the directive will stop the Agent from reporting statsx messages, but reports counterin and counterout messages. 

(info) NOTE: This directive defaults to "true."

true
<enable_performance_report/>Self-closing tag that enables performance reporting on the DataSync Agent.
 <enable_snowflake/>

Self-closing tag that enables agent to share to Snowflake. 

(warning) Alternatively, DataSync for Snowflake is available and recommended for sharing data to a Snowflake database.



Directives under the Task level that contains required attributes:

DirectiveRequired attributesDescriptionExample Value
<message_connection>user = Perspectium Mesh username

Perspectium Mesh URL

(info) NOTE: Depending on which Integration Mesh you are connecting to, https or amqps may be the only connection available. Contact Perspectium Support for more information. 

https://mymesh.perspectium.net

            

password = Perspectium Mesh password

(info) NOTEYou can optionally base64 encode this password, and include the base64 encoded string after encrypted: for this attribute's value.

queue = Perspectium Mesh queue name

The use_basic_consume attribute will depend on the type of messaging protocol you will be using.  

e.g., https://mymesh.perspectium.net

ProtocolValue
HTTPSuse_basic_consume = false
AMQPSuse_basic_consume = true
<management_connection>

user = Perspectium Mesh username

(OptionalDefines the destination URI and credentials required to report usage and availability of the agent to Perspectium. In addition to the scheme, you can also override the default port associated with a given scheme as reflected in the https entry below.


https://mymesh.perspectium.net

(info) NOTE: Supported schemes are amqp, amqps, http, and https

            

password = Perspectium Mesh password
<instance_connection>

user = ServiceNow instance username

(info) This ServiceNow username must have the perspectium role assigned in order to access ServiceNow table schemas.

ServiceNow instance URL


https://myinstance.service-now.com

            

password = ServiceNow instance password

(info) NOTEYou can optionally encrypt this password by entering encrypt: followed by the password for this attribute's value. For example, "encrypt:password1". This will encrypt the password the first time the Agent is run and change the attribute's value to "encrypted:<encrypted_password>" to indicate as such.

You can use ServiceNow's OAuth implementation to connect to your ServiceNow instance. See here for further information.

<schema_connection>






user = Salesforce org username

Salesforce org URLskip_table_stats

https://login.salesforce.com/services/oauth2/token







password = Salesforce org password

(info) NOTEYou can optionally encrypt this password by entering encrypt: followed by the password for this attribute's value. For example, "encrypt:password1". This will encrypt the password the first time the Agent is run and change the attribute's value to "encrypted:<encrypted_password>" to indicate as such.

client_id = Salesforce Consumer Key for the Perspectium Replicator app
client_secret = Salesforce Consumer Secret for the Perspectium Replicator app