Page History
| HTML |
|---|
<style>
.release-box {
height: 30px;
width: 100px;
padding-top: 8px;
text-align: center;
border-radius: 5px;
font-weight: bold;
background-color: #8efeb3;
border-color: #FCE28A;
}
.release-box:hover {
cursor: hand;
cursor: pointer;
opacity: .9;
}
</style>
<meta name="robots" content="noindex">
<div class="release-box">
<a href="https://docs.perspectium.com/display/krypton" style="text-decoration: none; color: #FFFFFF; display: block;">
Krypton
</a>
</div> |
| Anchor | ||||
|---|---|---|---|---|
|
Table CompareArchive Data (found under DataSync> Tools) allows allows you to compare tables between two ServiceNow instances, or one ServiceNow instance and another integrated database. This is useful because you can see each table's record count, as well as a list of record discrepancies by sys_id between the two tables you're comparing—in other words, you can see which records exist in one table but not the other.
Explore the various contexts and instructions for using this feature.
leverage the Perspectium DataSync solution to archive data from your ServiceNow instance to an external database. Whereas ServiceNow's built-in data archiving feature limits you to running archiving each table record only once, the Archive Data feature allows you to archive a record multiple times and to an external database of your choice using the Bulk Share function.
| Panel | ||||||
|---|---|---|---|---|---|---|
| Panel | ||||||
| ||||||
|
| Info | ||
|---|---|---|
| ||
|
Below are instructions for setting up table compare in different situations:
Prerequisites
You need the following information from Perspectium Support:
- Perspectium Integration Mesh (MBS) server URL
- Integration Mesh (MBS) vhost
- Integration Mesh (MBS) username
- Integration Mesh (MBS) password
Minimum Requirements
| Perspectium Core (ServiceNow app) | Krypton 8.2.0 and newer |
| DataSync Agent | Krypton 8.0.3 and newer Krypton 8.0.10 and newer for MySQL and PostgreSQL |
Supported Databases | Oracle Microsoft SQL Server MySQL PostgreSQL
|
Installation
You will need to install the application into your ServiceNow instance by installing update sets. You have two different options to install Archive Data:
Standalone
As of Krypton 8.3.0, Archive Data is available as a standalone update set that can be installed by itself. You do not need any other update sets but the Archive Data standalone update set.
| UI Steps | ||
|---|---|---|
|
Addon
Archive Data is available as an addon to the core Perspectium application:
| UI Steps | ||||||
|---|---|---|---|---|---|---|
|
Initial Setup
Before you begin, you will need set up initial configurations in Perspectium's General Settings:
| UI Expand | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||
|
Overview
Once you navigate to DataSync > Archive Data, you will have the following options:
- Archive - Choose the tables and conditions you want to archive from your ServiceNow instance.
- Restore - Choose the previously executed archives you want to restore back into your instance.
- Dashboard - View and monitor the data you've archived using this feature on your instance.
Archive
Anchor Archive Archive
Archive allows you to use the Bulk Share feature to back up tables to a queue that the DataSync Agent can then save into an external database. The Archive option provides a simplified view of the Bulk Share feature so as to allow you to specify which table to archive. Leveraging bulk share functionality, records are sent out in the standard Perspectium message format (as found in the Messages > Outbound table).
The Archive option offers the following choices after navigating Archive Data > Archive:
Create New Archive
Create a new archive configuration to specify which table you want to archive and to what target queue the records will be shared out to. This archive configuration is created as a specialized version of a bulk share.
When creating a new archive, like a standard bulk share, you specify a unique name for it along with the table you want to archive, the encryption method and a target queue to share records out to.
You can also specify conditions for which records from the table you want to archive along with sharing the table record's related attachments, journal fields and audit records. Like standard bulk shares, you can use the Share updates since then if you want to schedule this bulk share to run at certain times and share records updated since the last time the archive ran and Enable confirmation for ServiceNow to database table compare.
| Info | ||||||
|---|---|---|---|---|---|---|
|
NOTE: Because of how ServiceNow may cache UI views, you may be shown the wrong view on occasion (i.e. shown the Restore view when you are viewing an archive). To fix this, click on the menu in the top left of the form and choose the appropriate view (Share He20 Archive view for archives and Share He20 Restore view for restores):
View Archive Configurations
View previously created archive configurations. This view will only list archive configurations and not standard bulk shares.
Scheduled Archive
To schedule created archive configurations to continuously run at a time you choose. Scheduled archive is a specialized version of the scheduled bulk share feature with the main difference being you can only choose archives and not standard bulk shares. Similarly, scheduled bulk shares only allow you to choose bulk shares and not archives.
After selecting Scheduled Archive, you have the following options:
Create New Scheduled Archive
Create a new scheduled archive to schedule when you want archive(s) to run. The options will be the same as you see on a scheduled bulk share.
NOTE: You will have to save the scheduled archive first before you are able to add any archives to the scheduled archive.
View Scheduled Archives
View previously created scheduled archives. This view will only list scheduled archives and not standard scheduled bulk shares.
From here, you can click into a scheduled archive and choose the Edit... option to add archive configurations you created to run on the scheduled archive's schedule.
NOTE: You will only be able to select archives (and not standard bulk shares) to add to your scheduled archive. Similarly, on scheduled bulk shares you can only select standard bulk shares and not archives.
Restore
| Anchor | ||||
|---|---|---|---|---|
|
Restore allows you to restore previously executed archives. Restore works by sending a command message to the DataSync Agent which will then query the external database where the records were previously saved to and send those records back into ServiceNow. The records are sent to the subscribed queue the ServiceNow instance consumes from, with records sent in the Perspectium message format (as found in the Messages > Inbound table) so they can be read in using the Perspectium Core application.
| Info |
|---|
To restore data into your instance, you must have subscribe configured for the tables you will be restoring. When Archive Data is installed, it will create a global subscribe that will allow all table records to be restored into the instance. |
NOTE: If you restore task or its child records and include journal entries (sys_journal_field records) and those journal entries appear duplicated in your task record's activity log even though only a single record exists in the sys_journal_field table, you may need to rebuild the History Set for the records you are restoring. See Step 5 on this ServiceNow Knowledge Base article for more information.
The Restore option offers the following choices after navigating Archive Data > Restore:
Create New Restore
Create a new restore (bulk share) configuration to specify which archive you want to restore from.
You can either choose an archive you previously did or manually configure the restore (you cannot do both from the same restore configuration) so as to specify your own conditions for restoring if you are trying to restore only a subset of data and know where you are restoring from (i.e. which queue your DataSync Agent is reading from).
If you choose an archive you previously did, the manual configuration restore fields (Table name, Encryption method, Target queue, etc) will be hidden since the configurations from the archive will be used.
Otherwise you can configure restoring similar to configured creating a new archive, including if you want to restore attachments, journal fields and audit records.
Clicking Execute Now will send a command message to the queue specified (whether specifying the queue manually on the restore configuration or using the queue from the archive configuration) to tell the DataSync Agent to query and return records matching the restore conditions so the records can be read back into your ServiceNow instance.
| Info |
|---|
|
NOTE: Because of how ServiceNow may cache UI views, you may be shown the wrong view on occasion (i.e. shown the Restore view when you are viewing an archive). To fix this, click on the menu in the top left of the form and choose the appropriate view (Share He20 Archive view for archives and Share He20 Restore view for restores):
View Restore Configurations
View previously created restore configurations. This view will only list restore configurations and not standard bulk shares.
Dashboard
| Anchor | ||||
|---|---|---|---|---|
|
The Dashboard provides a view of the data you've archived using the Archive Data feature.
| Info | ||
|---|---|---|
| ||
|
After navigation to Archive Data > Dashboard, you will see the Dashboard user interface that shows the following:
Recent Archives
This feature displays the most recent (up to 50) archives from you instance. You can click on any entry to access that specific record directly. This table will only list bulk shares created as archives and not standard bulk shares (those created under Perspectium Core > Shares > Bulk Shares).
Record Breakdown
The Record Breakdown graph is a visualization of data based on the calendar date selected. Toggle between different data representation (records per hour, day, month, or lifetime).
Total Daily/Monthly Records
Total Daily Records: Displays the total records archived for the current day.
Total Monthly Records: Displays the total records archived for the current month.
| Info |
|---|
Changing the date/month on the calendar does not affect this count. |
Average Records Sent + Calendar Select
Use this feature to get a breakdown of the number of records archived based on a selected date in the calendar. Information here is based on on bulk shares created as archives and executed in your instance.
Any time you access or reload the dashboard, the calendar date will reset to the current date.
You can change the date on the calendar using the available arrows or the dropdown settings.
The Records Sent/Day count reflects the number of records archived on the specific calendar date selected.
The Records Sent/Hour count is reflects the number of records archived in the the specific calendar date selected, divided by 24 hours. However, if the current date is selected, then this number is divided by the number of hours that have passed in that day so far.
Uninstall Archive Data
| Anchor | ||||
|---|---|---|---|---|
|
Located under DataSync > -- Uninstall Archive Data, this option allows you to remove Archive Data specific Perspectium components from your instance.
Once you click on the module, you will be presented with a page to confirm you want to uninstall the Archive Data feature. Once you click Confirm, this will run a job in the background to remove Archive Data specific script includes, modules, scheduled jobs and UI pages including the Dashboard mentioned page.
This will take a few minutes to complete. You may need to log out and then log back into your ServiceNow instance to ensure that all Archive Data modules are removed from your left side list display. Also, if any Archive Data modules were marked as favorites prior to the uninstall process, you can remove the modules by selecting the - icon on to the right of the module.
NOTE: If you have Archive Data installed as an addon or if you also have another standalone app installed (such as DataSync Snapshot), this will only install Archive Data functionality and not the core Perspectium application functionality.
ServiceNow to ServiceNow table compare
There are two options when it comes to comparing tables between two instances of ServiceNow:
- Audit Shares looks at the records that have been bulk shared and compare them to the records in another instance. This is useful for ensuring that bulk shares are captured in the subscribing instance properly.
- Compare Records by Table looks at records that satisfy specified conditions and compares them with the same records in another instance. This is useful if you have multiple dynamic/bulk shares on the same table and want to verify that the cumulative sharing of records is captured properly in the database.
Audit shares
The Audit Shares option compares the records shared by bulk shares with the records in another ServiceNow instance. This is useful for ensuring that bulk shares are captured in the database properly.
Follow the steps to create a bulk share or go to a bulk share that you want to compare at Perspectium > Perspectium Core > Shares > Bulk Share.On the bulk share page, switch to the Standard view. In the Miscellaneous tab, check the Advanced box, which will reveal an Advanced tab (if not already there).
In the Advanced tab, in the Runtime Settings section, check the Enable confirmation box.
Click Update, and the table compare will automatically occur once the the bulk share has been executed. Now that the bulk share is ready, you can go back into the Tools module.
Go to Perspectium > Perspectium Core > Tools, and click Table Compare: ServiceNow to ServiceNow, and then Audit Shares.
In the resulting list of all your bulk shares, select the ones that you want to use in the table compare. Make use of the Select All and Deselect All options to easily select or deselect many bulk shares at once.
Compare records by table
The Compare Records by Table option compares the records that satisfy specified conditions with the same records in your database. This is useful if you have multiple dynamic/bulk shares on the same table and want to verify that the cumulative sharing of records is captured properly in the database.
Go to Perspectium > Perspectium Core > Tools > Table Compare: ServiceNow to ServiceNow > Compare Records by Table.On the resulting page, select a table from the Choose Table dropdown.
Use the Comparing Setup dropdown to select the records that you want to compare. This is where you can specify whether you want to compare the base table records only, or include the child records of that table as well. You can find information on child tables and the general table hierarchy here.
In the Bulk share field, select a bulk share that will mimic the configurations for re-sharing missing records.
(Optional) Create a condition with the condition builder or leave it blank to select all records from the chosen table. If the bulk share selected contains conditions, the condition builder will be pre-filled with the same conditions.
Click Add to add the table and condition to the active table compare configuration.
ServiceNow to database table compare
This module allows you to compare records shared via bulk or dynamic shares with tables in another database. With bulk shares, a list of sys_ids will be sent from ServiceNow to the DataSync Agent for the Agent to check if the records with the corresponding sys_ids exist. If there are discrepancies, a message will be sent back to ServiceNow and the missing records will be re-shared to the Agent. This process will repeat until all records have made it to the target database.
Table Compare can also compare deleted records that dynamic shares share out to delete in the database. The Agent will check to make sure that the sys ids in the list do not exist in the database. The records that should have been removed but was not previously removed will be deleted immediately.
Here are the supported databases for table compare:
Database
Version*
*These versions of the associated databases have been tested at this time but in general these databases will work with Table Compare. Testing is ongoing to support more databases.
| Info | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
If using a table map with your bulk share, you will need to do the following for reshared records (records the Table Compare feature finds as missing and is resharing to the Agent to update in the database) to work properly:
to
If you do not have a databases.xml configuration file (and this configuration file is only needed if you have advanced Agent configurations like the above), you can acquire it from Perspectium Support. Once you have the file and made the above change, move databases.xml into the Agent's conf folder i.e. <Perspectium_DataSync_Agent_Installed_Directory>/conf |
There are three options when it comes to comparing tables between a ServiceNow instance and another database:
- Audit Shares looks at the records that have been bulk shared and compare them to the records in the database. This is useful for ensuring that bulk shares are captured in the subscribing instance properly.
- Compare Records by Table looks at records that satisfy specified conditions and compares them with the same records in the database. This is useful if you have multiple dynamic/bulk shares on the same table and want to verify that the cumulative sharing of records is captured properly in the database.
- Synchronize Delete Records looks at records that no longer exist in your instance and remove them from your database.
| Info | ||
|---|---|---|
| ||
|
Audit shares
The Audit Shares option compares the records bulk shared by a ServiceNow instance with the records in your database. This is useful for ensuring that bulk shares are captured in the database properly.
Follow the steps to create a bulk share or go to a bulk share that you want to compare at Perspectium > Perspectium Core > Shares > Bulk Share.On the bulk share page, switch to the Standard view. In the Miscellaneous tab, check the Advanced box, which will reveal an Advanced tab (if not already there).
In the Advanced tab, in the Runtime Settings section, check the Enable confirmation box.
Click Update, and the table compare will automatically occur once the the bulk share has been executed. Now that the bulk share is ready, you can go back into the Tools module.
Go to Perspectium > Perspectium Core > Tools, and click Table Compare: ServiceNow to Database, and then Audit Shares.
In the resulting list of all your bulk shares, select the ones that you want to use in the table compare. Make use of the Select All and Deselect All options to easily select or deselect many bulk shares at once.
Compare records by table
| Info |
|---|
If using Table Maps or Conditional Shares, the Comparing Setup will be set to Compare the table records only as otherwise you may end up with duplicate records being compared. |
The Compare Records by Table option compares the records that satisfy specified conditions with the same records in your database. This is useful if you have multiple dynamic/bulk shares on the same table and want to verify that the cumulative sharing of records is captured properly in the database.
Go to Perspectium > Perspectium Core > Tools > Table Compare: ServiceNow to Database > Compare Records by Table.On the resulting page, select a table from the Choose Table dropdown.
Use the Comparing Setup dropdown to select the records that you want to compare. This is where you can specify whether you want to compare the base table records only, or include the child records of that table as well. You can find information on child tables and the general table hierarchy here.
In the Bulk share field, select a bulk share that will mimic the configurations for re-sharing missing records.
(Optional) Create a condition with the condition builder or leave it blank to select all records from the chosen table. If the bulk share selected contains conditions, the condition builder will be pre-filled with the same conditions.
Click Add to add the table and condition to the active table compare configuration.
(Optional) A table will appear for you to edit or remove the table you just added. In this table, check the Sync Deletes checkbox to remove records that no longer exist in your instance from your agent database.
NOTE: Table compare results will be shown in the Previous Table Compare Executions table at the bottom. You can filter the results displayed in this table by the target type (Agent, Meshlet, ServiceNow, etc.). However, the target type isn't populated until the target responds back to ServiceNow with results. Prior to the results returning, the table compare result will show as Pending Response to indicate we're pending a response from the target and you can filter the results for those Pending Response as well. Further details on Previous Table Compare Executions can be found in the View and report table compare results section below.
Synchronize delete records
Enabling this feature will send a message to your agent database to start comparing which records were deleted. Once the deleted records are detected, a series of delete messages will be sent.
Here's how:
Go to Perspectium > Perspectium Core > Tools > Table Compare: ServiceNow to Database > Synchronize Delete Records.On the resulting page, use the Choose Table dropdown to select the table that you want to use.
Use the Bulk Share field (and the magnifying glass) to select the bulk share that your Synchronize Delete can mimic for configurations.
Click Add.
View and report table compare results
After running a ServiceNow to Database table compare, you can view your results, including the total records compared, records not found in the target database that were shared by the ServiceNow instance or the records not deleted in the target database that were shared out as deleted. A quick summary of previous table compares that have run will also be available with the table provided in the Previous Table Compare Executions section.
Additionally, you can run a report with your table compare results.
Here's how to view your results:
Go to Perspectium > Perspectium Core > Tools. Click Table Compare: ServiceNow to Database.Then, click the table compare that you want to view (either Audit Shares, Compare Records by Table, or Synchronize Deleted Records).
Scroll to the Previous Table Compare Executions table. This table will provide you with a quick summary of previous table compares that have run. Click the timestamp in one of the entries to view more information about that specific table compare execution such as the bulk shares or dynamic shares involved, the table and conditions use, the time frame, and the individual discrepancies in the target database.
When looking at the results, numbers (such as the Records Reshared to Target to indicate discrepancies between ServiceNow and the database that had to be reshared) will have a background color to indicate as follows:
A Status of Reshared and a Target Discrepancy Type of Not Found indicates the record was not found in the target database and has been reshared. Once the Agent has processed both the reshared message and the compare message to validate the record is in the database, this will update in here to Synced in Database. Note, for each time a record is not found and reshared (even if the same record has to be reshared multiple times because it can't process successfully), a new row will be created in the results with a Status of Reshared and a Target Discrepancy Type of Not Found.
| Info |
|---|
If using Conditional Shares, the Total Records Compared will be the total of records shared to all conditional shares. For example, if your bulk share has 2 conditional shares and one conditional share shares out 173 records and the other shares out 27 records, the Total Records Compared will show 200 records compared. |
Performance Statistics
To give you an idea of the table compare performance, tests were ran on a ServiceNow instance with two nodes, bulk sharing incidents with 4 MultiOutput processors jobs and records that averaged 5 KB in size. The total time for table compare to complete from ServiceNow to a database was as follows: