This guide will provide you the optimization of your DataSync integration for the best performance while avoiding issues with data updates happening incorrectly and out of order in your database.
ServiceNow Configuration
Create a shared queue
Create a shared queue using the Number of Queues option, choosing an option to match the workload expected. Sharing is best split by optimizing on the hexadecimal numbering system (16 characters, 0-9 and A-F) used for ServiceNow sys_id values. So you can optimize sharing by splitting the sharing based on the 16 characters, having Perspectium jobs run to share data with 2, 4, 8 or 16 jobs running in parallel.
NOTE: Running multiple jobs in parallel means more ServiceNow scheduled job workers are being used. So you do want to choose the option that best suits your workload and data being shared to ensure optimal performance of your instance.
Create a bulk share using the created shared queue
Agent Configuration
Create a message connection
For the shared queue you created in ServiceNow above, you will create a <message_connection> with the num_queues attribute matching what you specified in the shared queue. This will create threads to consume the child queues along with the base queue.
For example, if you specify queue=
For example, with the configuration:
<message_connection password="password" queue="psp.out.replicator.prodqueue" user="user" num_queues="4">amqp://instance.perspectium.net</message_connection>
The Agent will create 5 threads, one thread to read from the base queue psp.out.replicator.prodqueue as well as a separate thread for each of the four child queues as follows:
psp.out.replicator.prodqueue01
psp.out.replicator.prodqueue02
psp.out.replicator.prodqueue03
psp.out.replicator.prodqueue04
The base queue psp.out.replicator.prodqueue can be used as a "fast lane" for records that need to have priority for sharing into the database. By default with this optimization setup, the bulk share will use the child queues 01, 02, 03 and 04 to evenly distribute the records to be shared out of the table the bulk share is running on in ServiceNow.