The file that controls the configuration settings for Perspectium DataSync Agent is the agent.xml file. This file is automatically created within the Agent's bin directory upon installation. To make changes to the configuration settings for your DataSync Agent, open this agent.xml file in a text editing application and modify the directives described in the sections below.
This is NOT a finalized list. We are currently in the process of including all of the directives in this list.
Agent Level
The <agent> is the second highest level next to <config>. The agent level holds the job types to be executed (i.e <share> and <subscribe>), as well as optional directives for additional configurations. These directives will then contain their own configurations within.
| Directive | Description | Example Value |
|---|---|---|
| <max_reads_per_connect> | Maximum number of messages to consumer per connection interval. | 4000 |
| <max_writes_per_connect> | Maximum number of messages to produce per connection interval. | 4000 |
Interval for how often the agent pulls from the queue (seconds).
| 5 | |
| <skip_message_set_processing/> | Self-closing tag that disables Message Set posting for data shared from ServiceNow. | |
| <skip_table_stats/> | Self-closing tag that disables tracking of changes to replicated tables. Without this directive, the changes would be found in psp_table_stats tables in the psp_db table on MBS. For earlier versions (Fluorine+ and below), this directive is required for Snowflake database. | |
| <share> | An Agent configured with <share> functionality (also known as a producer) monitors static data sources and places messages from those sources into queues in the Perspectium Mesh. | |
| <subscribe> | An Agent configured with <subscribe> functionality (also known as a consumer) monitors queues in the Perspectium Mesh and processes any messages in those queues, typically by routing those messages to a static data source.
| |
| <skip_report/> | Self-closing tag that causes the agent to skip sending reporting messages. |
NOTE: Subscribe & Share Level
There are two types of handlers or tasks that can be configured within the <agent> tag, the subscriber <subscribe> and the sharer <share>. The subscribe or share level will have zero or more nested <task> elements. However, the DataSync Agent will always have at least one task to perform the replication.
| Directive | Description |
|---|---|
| <task> | Task that handles either sharing or subscribing. You can configure separate settings by defining directives within each <task> directive. You can also specify an instances attribute here and give it a value of 4 (so that 4 instances of your <share> or <subscribe> task can run concurrently, thus improving throughput for data processing. You do not need to enter a value within the <task> directive. Nest all configuration directives noted below within your each of your <task> directives.
|
Task Level
The <task> level specifies the configurations needed for connecting to the instance, server, and database along with optional directives that allows for more control and configurations.
| Directive | Description | Example Value | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| <task_name> | Name for your <task>
| datasyncforservicenow_to_mysql_subscribe | ||||||||||||||||||
| <handler> | Name of the Java handler class. In most cases, the value for this directive will be com.perspectium.replicator.sql.SQLSubscriber. | com.perspectium.replicator.sql.SQLSubscriber | ||||||||||||||||||
| <encryption_key> | Encryption key used to encrypt data being shared from a static data source to an application | |||||||||||||||||||
| <decryption_key> | Decryption key used to decrypt data being shared from an application | |||||||||||||||||||
| <receipt_send_interval> | (Optional) Indicates the number of seconds in which the Agent will regularly send receipts back to the sharing/publishing instance.
| 1800 | ||||||||||||||||||
| <ack_batch_size> | (Optional) Can be used to specify the maximum size of acknowledged receipt messages per batch.
| 1000 | ||||||||||||||||||
| (Optional) Can be used to specify the maximum size of error receipt messages per batch.
| 1000 | |||||||||||||||||||
| <database_type> | Type of database you are syncing your data to. Supported databases and associated values for the <database_type>:
| |||||||||||||||||||
| <database_server> | URL for your database. If your database is installed on your local machine, set this value to localhost. | |||||||||||||||||||
| <database_port> | Port number for your database | |||||||||||||||||||
| <database_user> | Username used to connect to your database | |||||||||||||||||||
| <database_password> | Password used to connect to your database | |||||||||||||||||||
| <database_parms> | (Optional) Can be used to pass additional parameters to the database server during establishment of the connection For MySQL, this directive will have a default value of autoReconnect=true & characterEncoding=UTF-8 & useSSL=false.
| |||||||||||||||||||
| <database> | Name of your database | |||||||||||||||||||
| <database_column_max_size> | Maximum size of an object that can appear in your database's columns | |||||||||||||||||||
| <database_static_column_size> | Prevents altering the size of an object in your database's columns
| |||||||||||||||||||
| <database_table_character_set> | (Optional) Sets the MySQL database character set to utf8mb4 to accept multibyte characters | CHARACTER SET utf8mb4 | ||||||||||||||||||
| <database_table_collation> | (Optional) Sets the MySQL database collation to utf8mb4 to accept multibyte characters | COLLATE utf8mb4_unicode_ci | ||||||||||||||||||
| <database_sid> | SID used for your Oracle database
| |||||||||||||||||||
| <byte_padding> | (Optional) Can be used for multi-byte characters by multiplying the size of the columns. Adding an integer to this directive will create columns X times larger.
| 3 | ||||||||||||||||||
| <column_sort> | Sorts your database's columns in a specified order.
| alpha | ||||||||||||||||||
| <sql_retries> | (Optional) Can be used to configure the number of times the DataSync agent will retry after a SQLException.
| 250 | ||||||||||||||||||
| <sql_retries_delay> | (Optional) Can be used to configure how many milliseconds the DataSync agent will wait between retries.
| |||||||||||||||||||
| <skip_database_creation/> | Self-closing tag that stops the agent from creating a new database. Use this only if you are replicating to an existing database. | |||||||||||||||||||
| <ensure_table_exists/> | Self-closing tag that is required for replicating to Snowflake database. | |||||||||||||||||||
| <match_field_lengths/> | Self-closing tag that matches text field lengths. | |||||||||||||||||||
| <batch_insert/> | Self-closing tag that configures your Agent to batch SQL INSERT statements. | |||||||||||||||||||
| <batch_update/> | Self-closing tag that configures your Agent to batch SQL UPDATE statements. | |||||||||||||||||||
| <batch_flush_interval> | Number of seconds that will trigger a batch SQL statement execution. Acceptable values are integers from 1 to 30 (seconds). | 20 | ||||||||||||||||||
| <max_batch_size> | Number of SQL statements that will trigger a batch SQL statement execution. A larger suggested value is 200. | 200 | ||||||||||||||||||
| <skip_performance_report> | When enabled, the directive will stop the Agent from reporting statsx messages, but reports counterin and counterout messages.
| true | ||||||||||||||||||
| <enable_performance_report/> | Self-closing tag that enables performance reporting on the DataSync Agent. | |||||||||||||||||||
| <enable_snowflake/> | Self-closing tag that enables agent to share to Snowflake.
|
WARNING: This directive may cause data to be truncated Directives under the Task level that contains required attributes:
| Directive | Required attributes | Description | Example Value | |||
|---|---|---|---|---|---|---|
| <message_connection> | user = Perspectium Mesh username | Perspectium Mesh URL
| https://mymesh.perspectium.net
| |||
password = Perspectium Mesh password
| ||||||
| queue = Perspectium Mesh queue name | ||||||
The use_basic_consume attribute will depend on the type of messaging protocol you will be using. e.g., https://mymesh.perspectium.net
| ||||||
| <management_connection> | user = Perspectium Mesh username | (Optional) Defines the destination URI and credentials required to report usage and availability of the agent to Perspectium. In addition to the scheme, you can also override the default port associated with a given scheme as reflected in the https entry below. | https://mymesh.perspectium.net
| |||
| password = Perspectium Mesh password | ||||||
| <instance_connection> | user = ServiceNow instance username
| ServiceNow instance URL | https://myinstance.service-now.com
| |||
password = ServiceNow instance password
| ||||||
| You can use ServiceNow's OAuth implementation to connect to your ServiceNow instance. See here for further information. | ||||||
| user = Salesforce org username | Salesforce org URLskip_table_stats | https://login.salesforce.com/services/oauth2/token | ||||
password = Salesforce org password
| ||||||
| client_id = Salesforce Consumer Key for the Perspectium Replicator app | ||||||
| client_secret = Salesforce Consumer Secret for the Perspectium Replicator app |