The DataSync Agent can be set up to replicate table records from a Service Now instance to local file(s) on the machine where the DataSync Agent is running. This can be useful for the case where you have a separate application that is able to read files to import data.
Records can be saved in CSV, JSON, and XML formats and each record is inserted into the file (i.e. the previous version is not updated) when the Agent processes a message.
Prerequisites
First, you will need to set up one of the Perspectium DataSync Agents.
You should also stop running your DataSync Agent before making any Agent configuration changes.
Procedure
To enable file replication for the DataSync Agent, follow these steps:
Access your agent.xml configuration file
Navigate to the directory where you saved your agent.xml file when installing your DataSync Agent.
Edit the agent.xml file with the following configuration changes
Within the <task> tag, nest the following directives with your choice of how you want to save your records:
All Records in One File
If you want to save all records in one file, use the following directives:
| Directive | Example | Use | Required? | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| <handler> | <handler>com.perspectium.replicator.file.XMLFileSubscriber</handler>
| The name of the file handler class | Yes | ||||||||
| <file_name> | <file_name>records.csv</file_name> | The name of the file to which you want to save the records | Yes | ||||||||
| <files_directory> | For Linux: <files_directory>/Downloads/subscribefiles</files_directory> For Windows: <files_directory>Users\Downloads\subscribefiles</files_directory> | The directory that contains the file of the saved records | Yes | ||||||||
| <buffered_writes> | <buffered_writes>250</buffered_writes> | A number of records to buffer before writing to file (to improve performance and not write to the file upon reading each record) | No | ||||||||
| <exclude_xml_header> | <exclude_xml_header/> | For use with the XMLFileSubscriber handler, this will only output the xml header tag (i.e. <?xml version=“1.0” encoding=“UTF-8”?>) once at the top of the file. That way you can treat the entire file as one XML file with multiple elements for parsing. For example, with this configuration, the file will be: <?xml version=“1.0” encoding=“UTF-8”?> <incident></incident> <incident></incident> <cmdb_ci></cmdb_ci> versus <?xml version=“1.0” encoding=“UTF-8”?><incident></incident> <?xml version=“1.0” encoding=“UTF-8”?><incident></incident> <?xml version=“1.0” encoding=“UTF-8”?><cmdb_ci></cmdb_ci> | No |
One Record Per File
If you want to save one record per file, use the following directives:
| Directive | Example | Use | Required? | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| <handler> | <handler>com.perspectium.replicator.file.XMLFileSubscriber</handler>
| The name of the file handler class | Yes | ||||||||
| <one_record_per_file> | <one_record_per_file/> | This directive will tell the agent to save each record into its own file instead of saving all records together in a single file. | Yes | ||||||||
| <files_directory> | For Linux: <files_directory>/Downloads/subscribefiles</files_directory> For Windows: <files_directory>Users\Downloads\subscribefiles</files_directory> | The directory that contains the file of the saved records | Yes | ||||||||
| <file_prefix> | <file_prefix>record</file_prefix> | A prefix for the file name of each record. If this directive is not specified, “psp.replicator.” will be used as the prefix. | No | ||||||||
| <file_suffix> | <file_suffix>.xml</file_suffix> | A suffix for the file name of each record. If this directive is not specified, “.xml” will be used as the suffix. | No | ||||||||
| <translate_newline> | <translate_newline>nbsp</translate_newline> | This directive will replace record content newline entries with a non-breaking space. | No |
Records to Multiple Files
If you want to save your records to multiple files, use the following directives:
| Directive | Example | Use | Required? | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| <handler> | <handler>com.perspectium.replicator.file.XMLFileSubscriber</handler>
| The name of the file handler class | Yes | ||||||||
| <buffered_writes> | <buffered_writes>250</buffered_writes> | A number of records to buffer before writing to file (to improve performance and not write to the file upon reading each record) | Yes | ||||||||
| <files_directory> | For Linux: <files_directory>/Downloads/subscribefiles</files_directory> For Windows: <files_directory>Users\Downloads\subscribefiles</files_directory> | The directory that contains the file of the saved records | Yes | ||||||||
| <file_prefix> | <file_prefix>record</file_prefix>
<file_prefix>$table_$d{yyyyMMdd}_$i</file_prefix>
| A prefix for the file name of each record. If this directive is not specified, “psp.replicator.” will be used as the prefix. | No | ||||||||
| <file_suffix> | <file_suffix>.xml</file_suffix>
| A suffix for the file name of each record. If this directive is not specified, “.xml” will be used as the suffix. | No | ||||||||
| <separate_files> | <separate_files>table</separate_files> | Indicates that the files will be separated by table. | Yes | ||||||||
| <enable_audit_log/> | <enable_audit_log/> | A self-closing directive that will generate an audit file. The audit file has information about when the records are processed, name of the file, and number of records processed. | No | ||||||||
| <translate_newline> | <translate_newline>%13</translate_newline>
| This directive will replace record content newline entries with the value you input. | Varies |
Save your agent.xml
Save the changes you've made to your agent.xml and close the file.
An example agent.xml configuration for saving all records to a single file is shown below:
<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?>
<config>
<agent>
<max_reads_per_connect>10</max_reads_per_connect>
<polling_interval>20</polling_interval>
<subscribe>
<task>
<task_name>file_subscribe</task_name>
<message_connection password="password" user="user">https://<customer>.perspectium.net</message_connection>
<instance_connection password="password" user="user">https://<instance>.service-now.com</instance_connection>
<decryption_key>The cow jumped over the moon</decryption_key>
<handler>com.perspectium.replicator.file.XMLFileSubscriber</handler>
<file_name>records.xml</file_name>
<files_directory>/Users/user/Downloads</files_directory>
<exclude_xml_header/>
<buffered_writes>250</buffered_writes>
</task>
</subscribe>
</agent>
</config>
An example agent.xml configuration for saving one record per file is shown below:
<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?>
<config>
<agent>
<max_reads_per_connect>10</max_reads_per_connect>
<polling_interval>20</polling_interval>
<subscribe>
<task>
<task_name>file_subscribe</task_name>
<message_connection password="password" user="user">https://<customer>.perspectium.net</message_connection>
<instance_connection password="password" user="user">https://<instance>.service-now.com</instance_connection>
<decryption_key>The cow jumped over the moon</decryption_key>
<handler>com.perspectium.replicator.file.XMLFileSubscriber</handler>
<one_record_per_file/>
<files_directory>/tmp</files_directory>
<file_prefix>records</file_prefix>
<file_suffix>.xml</file_suffix>
</task>
</subscribe>
</agent>
</config>
An example agent.xml configuration for saving records to multiple files is shown below:
<?xml version="1.0" encoding="ISO-8859-1" ?>
<config>
<agent>
<!-- the following subscribe fragment defines subscribing class -->
<!-- and its arguments -->
<subscribe>
<task>
<task_name>test_file_subscriber</task_name>
<message_connection password="password_here" user="admin" queue="psp.in.meshlet.example">https://<customer>.perspectium.net</message_connection>
<instance_connection password="Adminadmin1" user="admin">https://<instance>.service-now.com</instance_connection>
<decryption_key>Example_decryption_key_here</decryption_key>
<handler>com.perspectium.replicator.file.JSONFileSubscriber</handler>
<buffered_writes>10</buffered_writes>
<files_directory>/Users/You/Downloads/Example</files_directory>
<file_prefix>$table_$d{yyyyMMdd}_$i</file_prefix>
<file_suffix>.json</file_suffix>
<translate_newline>%13</translate_newline>
<separate_files>table</separate_files>
<enable_audit_log/>
</task>
</subscribe>
<max_reads_per_connect>1</max_reads_per_connect>
<polling_interval>3</polling_interval>
</agent>
</config>
Restart your DataSync Agent
After configuring your agent.xml file to enable file replication, start running your DataSync Agent again.
