Target Audience
Customers, External
Overview
This guide provides a methodical approach to identifying causes for data to be out of sync. It requires a birds-eye understanding of the DataSync Architecture to see how data flows out of ServiceNow into the Perspectium Mesh where the data sits until the Perspectium DataSync Agent is able to consume the messages to perform the insert/update/delete into the Target Database. Here is a diagram that shows the DataSync workflow:
Listed below are the components where this document details how to troubleshoot each of them:
- ServiceNow (Source)
- Perspectium Mesh
- DataSync Agent
- Database (Target)
Before deep diving from Source to Target, it is best to perform STEP 1 to identify problematic tables where discrepancies have been identified. Prioritizing the tables then allows us to focus on specific shares and possibly time frame of an issue. If possible, it would also be helpful to find sys_id values that either exist in ServiceNow, but not in the Database or found in the Database but not in ServiceNow.
Identify tables where there is discrepancy
Create a spreadsheet that includes the following columns and their values:
- Table Name
- Count in ServiceNow
- Count in Database
- Count difference
- Date/time of when counts were recorded
- Flag indicating if there are more records in Database or ServiceNow
- Examples (if possible): sys_id values with a discrepancy
Progress through these upcoming steps as we investigate from source to target to possibly identify the root cause for each discrepancy identified or isolate where the problem may be.
ServiceNow
The Perspectium DataSync Application in the ServiceNow platform simply creates PSP Outbound Message (psp_out_message) records and publishes them to the Perspectium Mesh (MBS).
Majority of discrepancy issues from a ServiceNow perspective are due to:
- Dynamic/Bulk Share not creating psp_out_message record
- The PSP Outbound Message is not properly created (Example: empty value. See Known Issues)
- Dynamic Share does not capture final version of source record
- ‘Perspectium MultiOutput Processing’ Job(s) is inactive or platform blocks it from running
- ‘Perspectium MultiOutput Processing’ Job(s) is unable to communicate with Perspectium Mesh
Test Dynamic/Bulk Share for table(s) identified in the previous step
We want to test the Dynamic/Bulk Share for the problematic table to see if it properly creates the PSP Outbound Message and gets published to your Perspectium Mesh. If this is successful, then we can quickly eliminate the Share configuration in ServiceNow and connection to the Mesh from being an issue.
Dynamic Share
- Use Test Record feature or create/update a test record
Bulk Share
- Clone Bulk Share
- Rename Bulk Share
- Populate Limit number of records shared to a small value like 5 or set a filter to share a specific set of records
- Confirm the expected number of psp_out_message records are created with topic=replicator and are eventually Sent.
NOTE: The Attributes field of psp_out_message records contain useful information to help determine whether the expected PSP Outbound Messages are created:
| Field | Description |
|---|---|
| set_id | sys_id of Dynamic/Bulk Share that created PSP Outbound Message |
| RecordId | sys_id of Source record being replicated |
| cipher | Encryption method used by the Dynamic/Bulk Share |
| sharedQueue | sys_id of Shared Queue record |
Review the Dynamic Share/Bulk Share/Scheduled Bulk Share configuration for tables found in Step 1
Review the configuration to ensure it is configured to capture everything expected. Here a couple things to keep in mind when reviewing the share options:
Dynamic Share
- Parent/Child Hierarchy > Child Table Only (DEFAULT):
- This DEFAULT setting is enabled if the 'Share base table only' and 'Include all child tables' options are NOT selected. It will share all Child Tables 1 level below the specified Table. This may be misleading as most people expect Dynamic Shares are auto-configured to share data from the Table configured.
- Trigger Conditions TAB > Interactive Only:
- This option will only replicate data if a User executed an insert/update/delete. It will NOT share data if a Background Job or Script performed an operation against the table. Customers may not always be aware of Background Jobs or scripts performing actions on Tables they are attempting to sync to a Database.
- Filter and Enrichment TAB:
- Review the conditions and all options in this Tab to ensure nothing prevents replication from occurring for an expected record.
- Business Rule order and Business Rule when:
- There is a possibility that the ‘Perspectium Replicate’ Business Rule does not capture the final version of a record. Ensure that the Dynamic Share is configured to share the record after all Business Rules and Workflows are complete.
- There is a possibility that the ‘Perspectium Replicate’ Business Rule does not capture the final version of a record. Ensure that the Dynamic Share is configured to share the record after all Business Rules and Workflows are complete.