Page History

Test Connectivity (DataSync Agent) 

If you are seeing a backlog of messages in the Queue, then you’ll want to confirm whether the Agent is able to communicate with the Perspectium Mesh to retrieve the data.  The Validate Configuration Tool can be used to quickly test the connection between the Agent and the following components as defined in the agent.xml file.

• Perspectium Mesh 
• Your ServiceNow Instance
• Your Target Database

The agent will not be able to startup successfully if connection to one of these endpoints is failing.  This tool can be executed with the following commands from the Agent’s base install directory:

• Windows: bin\validateConfiguration.bat  NOTE: Double-clicking will work as well
• Linux: bash bin/validateConfiguration

The resulting output will be printed into the ConfigurationValidationReport.log file found in the bin directory/folder.  If the ConfigurationValidationReport.log shows a successful connection to all components, then there are no network related issues. 

Example output from executing validateConfiguration:

Agent Information:
Agent version: Fluorine_4.6.6
						
Agent name: 'psp_replicator_fluorine_466'
						
Configured Tasks:
        Task: psp_replicator_fluorine_466_subscribe Type: subscribe

						
               psp_replicator_fluorine_466_subscribe instances: 4

						
Attempting to connect to the Message Broker Service at: https://training.perspectium.net as user: training/training - SUCCESS
						
Attempting to connect to the database: psp_repl_466 at host: localhost port: 3306 as user: root - SUCCESS Attempting to fetch schema from: https://dev63629.service-now.com- SUCCESS
						
Validation Completed, results: SUCCESS


Report has been saved in file: ConfigurationValidationReport.log 

Typically, these connectivity issues are due to network, proxy, or firewall configuration.  Here is a list of ports that need to be open depending on the protocol used in the value of the <message_connection> directive in the agent.xml file:

_zero da_step_zero

Check java path for Agent startup issues

If the Agent is having issues starting up, check the java system command is working. Normally when Java is installed on a machine, the java system command is available to executed from any system command prompt in any directory. If you check for Java on your machine by executing the java -version and it doesn't work, then the java system command may not be set up properly.

First, verify the user you are currently logged in on the machine has access to the Java install (it may have been installed by another user and your user wasn't given access).

If the user does have access to Java, you may need to explicitly point to where the java executable is located for the Agent to run it. The executable is generally located in in the bin folder where Java is installed i.e.

/usr/local/java/jre1.8.0_181/bin/java (Linux)

C:\Program Files\Java\jre1.8.0_181\bin\java.exe (Windows)

Once you have the path of the Java executable, open up the wrapper.conf configuration file located in the conf directory of the Agent's installation directory and update the wrapper.java.command configuration near the top of the file to point to the path.

That is, change it from:

wrapper.java.command=java

To this (using the example above):

Linux

wrapper.java.command=/usr/local/java/jre1.8.0_181/bin/java

Windows

 NOTE: Use \\ for each folder in order for the path to be read correctly

wrapper.java.command=C:\\Program Files\\Java\\jre1.8.0_181\\bin\\java.exe

Anchor
da_step_one da_step_one

Test connectivity 

If you are seeing a backlog of messages in the queue, then you’ll want to confirm whether the Agent is able to communicate with the Perspectium Integration Mesh to retrieve the data. The Validate Configuration tool can be used to quickly test the connection between the Agent and the following components as defined in the agent.xml file.

• Perspectium Integration Mesh 
• Your ServiceNow instance
• Your target database

The agent will not be able to startup successfully if connection to one of these endpoints is failing. This tool can be executed with the following commands from the Agent’s base install directory:

Windows:  

bin\validateConfiguration.bat

 NOTE: Double-clicking on the .bat file will execute it as well.

Linux:

bash bin/validateConfiguration

The resulting output will be printed into the ConfigurationValidationReport.log file found in the bin directory/folder. If the ConfigurationValidationReport.log shows a successful connection to all components, then there are no network related issues. 

Example output from executing validateConfiguration:

Agent Information:
Agent version: Fluorine_4.6.6
						
Agent name: 'psp_replicator_fluorine_466'
						
Configured Tasks:
  Task: psp_replicator_fluorine_466_subscribe Type: subscribe

						
      psp_replicator_fluorine_466_subscribe instances: 4

						
Attempting to connect to the Message Broker Service at: https://training.perspectium.net as user: training/training - SUCCESS
						
Attempting to connect to the database: psp_repl_466 at host: localhost port: 3306 as user: root - SUCCESS Attempting to fetch schema from: https://dev63629.service-now.com- SUCCESS
						
Validation Completed, results: SUCCESS


Report has been saved in file: ConfigurationValidationReport.log 

Typically, these connectivity issues are due to network, proxy, or firewall configuration. Here is a list of ports that need to be open depending on the protocol used in the value of the <message_connection> directive in the agent.xml file:

Ensure that your network, firewall or proxy whitelist the following to allow your server to communicate externally:

• Perspectium Integration Mesh IP addresses 
• perspectium.net domain

 NOTE: Contact Perspectium Support for your IP addresses

Anchor
da_step_two da_step_two

 Logging level 

The Perspectium DataSync Agent has the ability to output more information to its logs in the event we are able to reproduce an issue consistently or field values are not matching up between record ServiceNow and row in the database.  

The logging level can manually be set in the log4j2.xml found in the conf folder or directory. 

By default, the logging level is set to info (this is located in line 33 of the log4j2.xml file): 

 <root level="info"> 

The parameter/tag can have the following values:

The logging level can also be set dynamically with a duration using the setLogging command: 

setLogging <logging_level> <duration>

Example

# Setting to finest logging for an hour
bin/setLogging finest
  
# Setting to finest logging for 3 hours (10800 seconds)
bin/setLogging finest 10800
  
# Setting to info logging indefinitely
bin/setLogging info -1

 NOTE: This logging level will be active until the duration expires or the Agent is restarted.

Anchor
da_step_three da_step_three

Quick analysis of the logs 

Sorting through the Agent logs will help you pinpoint issues such as:

• Connection issues to Perspectium Integration Mesh, ServiceNow or database
• SQL Response errors after failed insert/update/delete
• Failure to successfully decrypt messages
• Issues with building the SQL Statements
• Malformed JSON message published to the queue

All of these are possible problems that the DataSync Agent can encounter and provide an explanation for any discrepancy of data. 

Here are a few search terms or phrases that immediately indicate issues with the Agent:

Skipping 

This indicates 9 (3x3) failed attempts were made to insert, update, or delete the message to the database

Exception 

Generic search term that indicates error occurred during the:

• Processing of the message 
• Requests to the database or Perspectium Integration Mesh

ERROR

Generic error occurred during the processing of the message or requests to the database. You may also find the Agent may have unexpectedly shut down.  

javax.crypto.IllegalBlockSizeException: Input length must be multiple of 8 when decrypting with padded cipher

The data from the value attribute of the JSON message was not properly encrypted by ServiceNow.

A JSONObject text must begin with '{'

Invalid JSON message published to the queue.

SubscribeException: Failed to decrypt message, make sure the shared secret keys match!

The encryption key in the Perspectium properties in ServiceNow does not match the decryption key defined in the agent.xml file.

org.json.JSONException: Null key

Generic error but often times see it when the value (encrypted data containing source record being shared) of the message is empty.

failed: Connection timed out

Connection failure to queue in Perspectium Integration Mesh.

For additional errors found in the logs, see Troubleshooting Agent Installation and Configuration Errors.

Fix is in Fluorine Plus Patch 1.1 and up.

 NOTE: This is not a complete fix.  It is a workaround that skips encrypting the problematic field so our code can proceed to populate the value field of the psp_out_message record.  Investigation is ongoing.

current.setWorkflow(false) 

Impacts Perspectium Replicate business rules from a dynamic shares. This will prevent any other business rule from running on the current record.

current.autoSysFields(false)

Impacts Scheduled Sync Ups or Scheduled bulk shares. This will prevent the fields updated and updated by to change when the record itself is updated.

In regards to the .setWorkflow(false) behavioral flag, Perspectium’s GOLD Release has the option to create a Flow Designer dynamic share to get around this issue. 

 NOTE: Deletes can only be captured by a business rule. 

Inserts, updates, or deletes may not be captured on PPM Tables such as the following:

• task > planned_task
• task > planned_task > pm_project
• task > planned_task > pm_project_task

Likely due to a ServiceNow setting which prevents any business rules from executing: KB0793430: Why is my Project Task not listed in the Deleted Records?

The workaround is to configure the Query business rule such that it doesn’t run when executed by the bulk share:

Fix is in Fluorine Plus Patch 1.1 and up.