For optional advance configurations for your DataSync Agent, contact Perspectium Support if you have any questions.

Plugin Framework

The plugin framework is similar to the SQLSubscriber handler plugin framework. The major difference is that it's not dependent on a particular handler, rather it's provided by the framework and is therefore available to all handlers.

Enabling the plugin

The plugin or java class needs to be available when the DataSync agent starts. The DataSync agent installation creates a jars directory and the plugin class file must be contained within a valid java jar file which will need to be copied into that directory. Furthermore, you must explicitly configure the DataSync Agent to load and use the plugin.

Here's an example of a task called <my_task> using the plugin called TruncateMessageValue:

<task>
        <task_name>my_task</task_name>
        <task_plugins>
                <plugin max_message_value_size="1024">com.abc.replicator.plugins.TruncateMessageValue</plugin>
        </task_plugins>
 </task>

The <task_plugins> directive must be enclosed within the <task> directive as shown in the above example. A single <plugin> statement is used per plugin. Multiple <plugin> statements are executed in the order in which they are configured. This approach allows chaining of several plugins, where each performs a specific task upon the message. The attributes and associated values configured in the <plugin> tag will be made available to the plugin when its processMessage method is called. This can be useful for providing static information to the plugin.

In our example the plugin com.abc.replicator.plugins.TruncateMessageValue will be enabled and called during processing by a sharer or subscriber. The class will have a single entry in the attributes map whose key will be max_message_value_size, and the value will be 1024. The java class uses the configured attribute to ensure that the value of each message does not exceed 1024 characters by performing truncation.

Writing a plugin

The plugin framework currently requires the plugins to be written using the Java language.

Each class or plugin must implement the IPlugin interface which looks like the following:

public interface IPlugin {
        public void processMessage(Message message, Map<String, String> attributes);
}

The plugin developer creates a single class which is called for each message that is being either shared or subscribed to. The processMessage java method can act upon the message as desired. All properties of the message are available to the plugin for reading and updating. Once the method returns the next plugin will be called until all plugins have been called. In the case of a <subscribe> handler the message will then be passed to the handler for processing. If the task is a <share> type then the message will be published to the Perspectium Mesh.

↑ Go to top of page

IO Datetime Plugin

The following plugin can be activated for the DataSync Agent to populate data for the timestamps of the IO (input/output) it performs. This plugin provides you with the date and time of each case the Agent deletes, inserts, and updates a record in the database. Use of the delete column will create a column that shows the timestamp when a record is deleted instead of removing the record from the table.

This can be activated by adding in two sets of config tags into the agent.xml: one to dynamically create the columns, and one to activate them:

<!-- Create the columns -->
<dynamic_columns>     
	<dynamic_column column_type="93" column_size="32" deletes_only="true">psp_per_delete_dt</dynamic_column>
	<dynamic_column column_type="93" column_size="32" updates_only="true">psp_per_update_dt</dynamic_column>
    <dynamic_column column_type="93" column_size="32" inserts_only="true">psp_per_insert_dt</dynamic_column>
</dynamic_columns>
 
<!-- Update the columns -->
<plugins>
    <plugin insert_column="psp_per_insert_dt" update_column="psp_per_update_dt" delete_column="psp_per_delete_dt">com.perspectium.replicator.sql.plugin.SQLSubscriberIODateTimePlugin</plugin>
</plugins>

By default, the update column (psp_per_update_dt) will be updated for both inserts and updates of a record. If you want the update column to be updated only when records are updated, include the update_column_only tag:

...
<plugin insert_column="psp_per_insert_dt" update_column="psp_per_update_dt" update_column_only="true">com.perspectium.replicator.sql.plugin.SQLSubscriberIODateTimePlugin</plugin>
...

The following example will populate the three columns psp_per_delete_dt, psp_per_insert_dt, and psp_per_update_dt with their corresponding data through the plug inputted. You can name these columns whatever you wish, as long as the names match between the plugin and the dynamic column, and that you won't experience any column name conflicts with your subscribing data. 

 NOTE: The tags should be placed within the <task> tags.

<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?>
<config>
  <agent>
    <subscribe>
      <task>
        <task_name>example_subscribe</task_name>
        <message_connection queue="psp.out.replicator.example" user="acme" password="XXXXX">amqps://example.perspectium.net</message_connection>
        <instance_connection user="acme" password="XXXXX">https://example.service-now.com</instance_connection>
        <handler>com.perspectium.replicator.sql.SQLSubscriber</handler>
        <decryption_key>The cow jumped over the moon</decryption_key>
 
        <database_type>mysql</database_type>
        <database_server>localhost</database_server>
        <database_port>3306</database_port>
        <database_user>acme</database_user>
        <database_password>XXXXX</database_password>
        <database>example</database>
 
        <dynamic_columns>           
		  <dynamic_column column_type="93" column_size="32" deletes_only="true">psp_per_delete_dt</dynamic_column>
		  <dynamic_column column_type="93" column_size="32" updates_only="true">psp_per_update_dt</dynamic_column>
          <dynamic_column column_type="93" column_size="32" inserts_only="true">psp_per_insert_dt</dynamic_column>
        </dynamic_columns>
 
        <plugins>
          <plugin insert_column="psp_per_insert_dt" update_column="psp_per_update_dt" delete_column="psp_per_delete_dt">com.perspectium.replicator.sql.plugin.SQLSubscriberIODateTimePlugin</plugin>
        </plugins>
 
      </task>
    </subscribe>
 
    <max_reads_per_connect>1000</max_reads_per_connect>
    <polling_interval>5</polling_interval>
  </agent>
</config>

In Krypton 8.0.6 and newer, you can specify the timezone that the entries will be saved in by specifying the timezone attribute with a Java TimeZone ID (for example PST, Pacific/Honolulu, etc.). Doing this will populate the datetime values in the plugin's dynamic columns in the timezone specified.

 NOTE: This feature is intended for databases such as SQL Server and Oracle that do not allow you to specify a timezone in the connection from the Agent to the database. For databases such as MySQL and Postgres that allow you to specify a timezone, this will adjust the value incorrectly. For example, if in MySQL you specify the Agent's connection for UTC (i.e. you add the database_parms configuration serverTimezone=UTC) and the server is running in PST and then you set the plugin's timezone attribute as EST, this will cause the Agent to save the datetime values in the dynamic columns as 11 hours ahead.

Since in the normal scenario, the Agent would pass values in PST (since the server is running as PST) and then the Agent would translate it to UTC to be 8 hours ahead (since UTC is 8 hours ahead of PST), specifying EST causes 3 hours to be added to the values (since EST is 3 hours ahead of PST) and then the UTC conversion will add an additional 8 hours on top of this to cause the values to be 11 hours ahead.

See https://www.mssqltips.com/sqlservertip/7025/sql-server-oracle-postgresql-time-zones/ for further information.

As well, this will only update the datetime values going forward after this configuration has been added and will not update current values in the plugin's columns. That is, if the plugin had datetime values in the dynamic columns in the previous timezone (such as UTC) and then you configure it to use EST timezone, only new values in the update column will be saved as EST timezone while the insert column's values (and any previous update column values) will continue to be in UTC.

...
<plugin timezone="Pacific/Honolulu" insert_column="psp_per_insert_dt" update_column="psp_per_update_dt" update_column_only="true">com.perspectium.replicator.sql.plugin.SQLSubscriberIODateTimePlugin</plugin>
...

↑ Go to top of page