<style> .release-box { height: 30px; width: 100px; padding-top: 8px; text-align: center; border-radius: 5px; font-weight: bold; background-color: #8efeb3; border-color: #FCE28A; } .release-box:hover { cursor: hand; cursor: pointer; opacity: .9; } </style> <meta name="robots" content="noindex"> <div class="release-box"> <a href="https://docs.perspectium.com/display/krypton" style="text-decoration: none; color: #FFFFFF; display: block;"> Krypton </a> </div> |
Once you have installed and configured DataSync for ServiceNow, and have successfully set up a bulk share and understand how it works, there are various bulk share settings you can configure. These optional features can help you customize your DataSync experience and leverage DataSync's robust capabilities to make your organization's integration processes more powerful and streamlined.
Explore all of these available features below!
This feature allows you to share history set data for the table that you are bulk sharing. History sets in ServiceNow identify which particular records from an audited table contain historical information. History set records only contain a recent subset of historical information (within the past 30 days of inactivity). The following fields will be visible to you in list view for history sets:
Field | Description |
---|---|
ID | An identifier for the record whose history is being recorded |
Table | The audited table for the record whose history is being recorded |
Load Time | The amount of time it took to generate the history set |
For more information about ServiceNow history sets, see history sets and differences between audit and history sets.
Here's how to enable the sharing of history set data for a bulk share:
With DataSync Agent, this option will create a sys_history_xxx table on the target database. |
This feature allows you to simply clone an existing bulk share or clone and immediately run an existing bulk share.
This feature allows you to preview a bulk share before executing it. You can see how many records will be shared out and details about when the shared records were created.
|
Here's how to preview a bulk share:
This feature allows you to run a bulk share as another user, rather than the default system user that has access to all of the table's records. This feature can be useful when you want to limit the bulk share to a subset of data that certain users have access to, such as records in one domain in your domain separated instance.
Here's how to enable this feature:
This feature allows you to run an existing bulk share again, once it has a status of Completed. You may want to run the same bulk share multiple times if your intent is to migrate the same table data intermittently.
Here's how:
This feature allows you to bulk share out only the sys_ids for specific records. This is useful for when you don't want to create filters for a bulk share.
Here's how:
This feature allows you to properly share database views with a bulk share. By default, bulk shares run by querying for records from the selected table in 1,000 record increments, with records ordered by sys_id in ascending order. This is done to minimize performance impact on the ServiceNow instance. However, the records of database views do not have sys_ids and therefore cannot be shared in the same way. In order to properly share database views with a bulk share, you need to query for all records from the view at once.
To do this, you need to specify the glide.db.max_view_records integer system property to a number above the records in your view. If this property is not set, it defaults to 10,000. A good number to use is 999,999.
This feature allows you to limit the number of records shared once bulk share reaches a specific limit.
Here's how to turn it on:
This feature allows you to share only records that have been inserted or updated since the last time the bulk share ran. This is useful for Scheduled Bulk Shares when you want to ensure you capture all records changed since the last execution of the bulk share.
This feature does not work with database views which do not have a sys_updated_on or sys_created_on field for their records. |
Here's how to turn it on:
This feature will share archived records as regular table records as part of a bulk share, if and when there is an archived table for a specified table in the bulk share. If you choose to share child records, it will also share the records from the archived child tables.
This feature is to be used with the temporal feature in the DataSync Agent (see step 3).
Here's how to enable share as archive:
|
This feature allows you to distribute the work load of your bulk shares by allowing them to run on multiple nodes in your instance. This is useful if you're running multiple bulk shares or scheduled bulk shares, since they will run on the same node by default, and this could impact performance. If you use this feature, nodes will only be used if they are online ad set to accept "any" scheduler.
Here's how to turn it on:
This feature allows you to choose the number of jobs (1, 2, 4, 8, or 16) it will take to process the bulk shared records. Each job will process a subset of the total records, split by sys_id.
Here's how to customize the number of jobs:
This feature is not compatible with the following bulk share features:
|
This feature allows you to specify which node a bulk share will run on by referencing the sys_cluster_state table in ServiceNow. You need admin access (or a custom ACL rule) in order to see and set this value. You may want to run this feature in order to distribute the Perspectium processing across several nodes.
Here's how to specify nodes:
This feature allows you to customize the order in which your records in bulk share are sent out by choosing which field (such as Active, Updated, Created On etc.) the records will be ordered by when sending out the records. For example, if you choose the Created On field, the bulk share will send the records in ascending order based on the date and time at which they were created. By default, bulk shares will order records by the Sys ID (sys_id) field when querying for records to share out.
Here's how to send records in a specific order:
This feature allows you to share records from one or more related lists of the table being bulk shared. Here's how:
This feature allows you to create new tables in a subscribing ServiceNow instance by sharing a schema from one instance to another. This will create the table, field types, choices, labels, values, as well as the following:
Upon receiving the schema message, the table will be created in the instance and a generic subscribe record will be will be created to allow for replication of any records that come with the table schema.
Here's how to use share schema to replicate tables in a subscribing instance:
To optimize performance while avoid data loss, sharing is best done in parallel by using multiple bulk shares with multiple queues. So as to ease the configuration of having to configure multiple bulk shares and multiple queues, you can use a queue with the Number of Queues option specified to simply the setup.
This is an advanced feature for those who have heavy loads of data to share out. Use this feature with caution, as it may use more resources on your ServiceNow instance. |
When creating a bulk share and selecting a shared queue for the target queue, if the shared queue specified has the Number of Queues option with a value greater than 1 such that there are child queues created, the bulk share will follow the same approach.
That is, the bulk share you create will become the base bulk share and child bulk shares will be created for sharing data out. There will be an equal amount of child bulk shares created to match the child queues from the shared queue specified. For example, if the shared queue has four child queues, there will be four child bulk shares created.
These child bulk shares will be named in the format of the <Bulk Share Name> <##>_<Epoch Creation Time>
Where
<Bulk Share Name> is the name of the base bulk share as specified in its Name field
<##> is a number from 01-16 to correspond with the number of child bulk shares i.e. if you have four child bulk shares they would be numbered 01, 02, 03 and 04
<Epoch Creation Time> is the Epoch time when the child bulk share was created.
For example if you name your base bulk share Split Example and you choose a shared queue with four child queues for your target queue, it would create four child bulk shares as follows:
Child bulk share are automatically created to split based on the sys_id along with any filter conditions you've added to the bulk share in order to optimize performance while avoid data loss. This setup is based on the number of child bulk shares (which is based on the number of child queues) and child bulk shares are created as follows:
Number of child bulk shares | Filter conditions for each child bulk share |
---|---|
2 | Child bulk share 01: sys_id starts with 0, 1, 2, 3, 4, 5, 6, 7 Child bulk share 02: sys_id starts with 8, 9, A, B, C, D, E, F |
4 | Child bulk share 01: sys_id starts with 0, 1, 2, 3 Child bulk share 02: sys_id starts with 4, 5, 6, 7 Child bulk share 03: sys_id starts with 8, 9, A, B Child bulk share 04: sys_id starts with C, D, E, F |
8 | Child bulk share 01: sys_id starts with 0, 1 Child bulk share 02: sys_id starts with 2, 3 Child bulk share 03: sys_id starts with 4, 5 Child bulk share 04: sys_id starts with 6, 7 Child bulk share 05: sys_id starts with 8, 9 Child bulk share 06: sys_id starts with A, B Child bulk share 07: sys_id starts with C, D Child bulk share 08: sys_id starts with E, F |
16 | Child bulk share 01: sys_id starts with 0 Child bulk share 02: sys_id starts with 1 Child bulk share 03: sys_id starts with 2 Child bulk share 04: sys_id starts with 3 Child bulk share 05: sys_id starts with 4 Child bulk share 06: sys_id starts with 5 Child bulk share 07: sys_id starts with 6 Child bulk share 08: sys_id starts with 7 Child bulk share 09: sys_id starts with 8 Child bulk share 10: sys_id starts with 9 Child bulk share 11: sys_id starts with A Child bulk share 12: sys_id starts with B Child bulk share 13: sys_id starts with C Child bulk share 14: sys_id starts with D Child bulk share 15: sys_id starts with E Child bulk share 16: sys_id starts with F |
The child bulk shares will be the ones that are executed and they reflect which child queue they're sharing data to. When a base bulk share is executed by choosing the Execute Now or Execute Again option, it will execute the child bulk shares and show the status of Executed to indicate that it's been executed to run its child bulk shares as shown in the image above.
NOTE:
Once you've configured bulk shares and shared queues to optimize sharing in parallel, you can then configure multiple MultiOutput jobs to push this data to the Perspectium Mesh (MBS).
This feature uses the Table Compare feature to compare records shared out to a target database to confirm they have been successfully received. With this option selected, the bulk share will send out compare messages for the DataSync Agent or meshlet to validate the database has all the sys_ids that have been bulk shared.
To enable this feature:
With this feature enabled, a bulk share created with the name Reshare <table name> <UTC datetime> (for example Reshare ticket 2023-12-07 03:27:50) will be created to continually reshare any records that the DataSync Agent/meshlet say are not found in the database. Each time this reshare bulk share is run, a new compare message will be sent for the Agent/meshlet to process and validate the records are now in the database. If you don't want the Agent/meshlet to continually reshare records (because of an error in the database that needs to be fixed offline), you can uncheck the Enable confirmation box so the compare message is not sent the next time the reshare bulk share is run. |
This feature allows you to include only specific fields in records shared out. This can be useful when you don't want to share out every field of the table to the target system.
If you are using this feature when bulk sharing to a database with the Table Compare feature (Enable confirmation option on the bulk share), see below for how to properly configure the DataSync Agent to work with both features. |
Here's how to share only selected fields for a specific dynamic share:
When bulk sharing to a database with the Table Compare feature (Enable confirmation option on the bulk share), the DataSync Agent by default relies on comparing the sys_mod_count column in the table.
So if you are using the Share only selected fields option, to ensure the Agent is able to properly compare records, you can do one of the following two options:
Then, in databases.xml, you will need to change the following values in the <database_type></database_type> section that corresponds to your target database. For example, if you are sharing to a MySQL database, you would modify the values in the <database_type>mysql</database_type> section:
Old Value | New Value |
---|---|
<most_recent_record_query>SELECT sys_id FROM %s WHERE sys_id = "%s" AND %s = "%s"</most_recent_record_query> | <most_recent_record_query>SELECT sys_id FROM %s WHERE sys_id = "%s"</most_recent_record_query> |
This change will remove the second part of the query (AND %s = "%s") where it checks for sys_mod_count.
Snippet of databases.xml with this change:
<database> <database_type>mysql</database_type> . <most_recent_record_query>SELECT Id FROM %s WHERE Id = "%s"</most_recent_record_query> . </database> |