Page History
HTML |
---|
<style>
.release-box {
height: 30px;
width: 100px;
padding-top: 8px;
text-align: center;
border-radius: 5px;
font-weight: bold;
background-color: #d4af37;
border-color: #FCE28A;
}
.release-box:hover {
cursor: hand;
cursor: pointer;
opacity: .9;
}
</style>
<meta name="robots" content="noindex">
<div class="release-box">
<a href="https://docs.perspectium.com/display/gold" style="text-decoration: none; color: #FFFFFF; display: block;">
Gold
</a>
</div> |
DataSync Agent Tasks
The DataSync Agent can be configured to have zero or more tasks. An agent should always have at least one task, since the tasks and their associated handler perform the actual processing.
Each task will have access to its own section of the overall configuration in addition to those directives outlined within the <agent> section of the configuration.
For example here is an agent that:
Subscribes to the queue psp.out.replicator.acme and places the records in psp_repl
Shares the table incident from Test to the queue psp.out.replicator.acme_test
Shares the table problem from psp_repl to the queue psp.out.repcliator.acme_dev
Code Block | ||
---|---|---|
| ||
<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?> <config> <agent> <max_reads_per_connect>4000</max_reads_per_connect> <!-- Subscribing Task --> <subscribe> <task> <task_name>Subscribe</task_name> <message_connection queue="psp.out.replicator.acme" password="XXXX" user="acme">https://acme.perspectium.net</message_connection> <instance_connection password="XXXX" user="acme">https://acme.service-now.com</instance_connection> <handler>com.perspectium.replicator.sql.SQLSubscriber</handler> <decryption_key>The cow jumped over the moon</decryption_key> <schedule>* * * * *</schedule> <database_type>mysql</database_type> <database_server>localhost</database_server> <database_port>3306</database_port> <database_user>root</database_user> <database_password></database_password> <database_parms></database_parms> <database>psp_repl</database> </task> </subscribe> <!-- Sharing Task --> <share> <task> <task_name>Share to Test</task_name> <handler>com.perspectium.replicator.sql.sharer.SQLSharer</handler> <message_connection queue="psp.out.replicator.acme_test" password="XXXX" user="acme">https://acme.perspectium.net</message_connection> <encryption_key>The cow jumped over the moon</encryption_key> <max_writes_per_connect>50</max_writes_per_connect> <polling_interval>90</polling_interval> <database_type>mysql</database_type> <database_server>localhost</database_server> <database_port>3306</database_port> <database_user>root</database_user> <database_password></database_password> <database>Test</database> <table_name>incident</table_name> <topic>replicator</topic> <type>agent</type> <name>.bulk</name> </task> </share> <!-- Sharing Task --> <share> <task> <task_name>Share To Dev</task_name> <handler>com.perspectium.replicator.sql.sharer.SQLSharer</handler> <message_connection queue="psp.out.replicator.acme_dev" password="XXXX" user="acme">https://acme.perspectium.net</message_connection> <encryption_key>The cow jumped over the moon</encryption_key> <max_writes_per_connect>50</max_writes_per_connect> <polling_interval>60</polling_interval> <database_type>mysql</database_type> <database_server>localhost</database_server> <database_port>3306</database_port> <database_user>root</database_user> <database_password></database_password> <database>psp_repl</database> <table_name>problem</table_name> <topic>replicator</topic> <type>agent</type> <name>.bulk</name> </task> </share> </agent> </config> |
Multiple Instances on Subscriber Task
The DataSync Agent can also be configured to have one or more instances running on a single subscriber task. By default, the agent configures each <subscriber> task to use four instances one instance during the replication process. However, you can manually configure your subscriber tasks to use multiple instances. By specifying multiple instances, the agent will run multiple parallel threads when subscribing to a queue, which allows it to replicate more data over a given period of time.
UI Text Box | ||
---|---|---|
| ||
NOTE: Because multiple instances of the task are simultaneously reading from the same queue at the same time, this approach is suggested only for when you are bulk sharing data and are not concerned with order. |
For example, here is an agent which configures a task with 4 instances and subscribes to the queue psp.out.replicator.acme.
Code Block | ||
---|---|---|
| ||
<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?> <config> <agent> <polling_interval>30</polling_interval> <!-- Subscribing Task --> <subscribe> <task instances="4"> <task_name>Subscribe</task_name> <message_connection queue="psp.out.replicator.acme" password="XXXX" user="acme">https://acme.perspectium.net</message_connection> <instance_connection password="XXXX" user="acme">https://acme.service-now.com</instance_connection> <handler>com.perspectium.replicator.sql.SQLSubscriber</handler> . . . </task> </subscribe> </agent> </config> |
When running the agent, you will see the threads being executed. Here is the sample log output of the four threads created above (Subscribe, Subscribe_2, Subscribe_3, Subscribe_4) running:
Code Block | ||
---|---|---|
| ||
INFO - PerformanceReportGenerator - PerformanceReport - Agent: <agent_name>.Subscribe_2 Processed: 128.0 Messages Errors: 0 Queue: psp.out.replicator.acme monitor 4.27 messages per second 379351 bytes processed
INFO - PerformanceReportGenerator - PerformanceReport - incident.bulk=[117, 344208]: 117 messages processed and 344208 bytes processed
INFO - PerformanceReportGenerator - PerformanceReport - change_request.bulk=[3, 12237]: 3 messages processed and 12237 bytes processed
INFO - PerformanceReportGenerator - PerformanceReport - Agent: <agent_name>.Subscribe_4 Processed: 114.0 Messages Errors: 0 Queue: psp.out.replicator.acme monitor 3.80 messages per second 333427 bytes processed
INFO - PerformanceReportGenerator - PerformanceReport - incident.bulk=[103, 303011]: 103 messages processed and 303011 bytes processed
INFO - PerformanceReportGenerator - PerformanceReport - cert_follow_on_task.bulk=[9, 24560]: 9 messages processed and 24560 bytes processed
INFO - PerformanceReportGenerator - PerformanceReport - Agent: <agent_name>.Subscribe Processed: 113.0 Messages Errors: 0 Queue: psp.out.replicator.acme monitor 3.77 messages per second 331079 bytes processed
INFO - PerformanceReportGenerator - PerformanceReport - sysapproval_group.bulk=[1, 2928]: 1 messages processed and 2928 bytes processed
INFO - PerformanceReportGenerator - PerformanceReport - incident.bulk=[106, 311844]: 106 messages processed and 311844 bytes processed
INFO - PerformanceReportGenerator - PerformanceReport - Agent: <agent_name>.Subscribe_3 Processed: 115.0 Messages Errors: 0 Queue: psp.out.replicator.acme monitor 3.83 messages per second 341240 bytes processed
INFO - PerformanceReportGenerator - PerformanceReport - sysapproval_group.bulk=[2, 6431]: 2 messages processed and 6431 bytes processed
INFO - PerformanceReportGenerator - PerformanceReport - incident.bulk=[100, 294194]: 100 messages processed and 294194 bytes |
Similar topics
Content by Label | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|