Once you have installed and configured DataSync for ServiceNow, and have successfully set up a bulk share and understand how it works, there are various bulk share settings you can configure. These optional features can help you customize your DataSync experience and leverage DataSync's robust capabilities to make your organization's integration processes more powerful and streamlined.

What's on this page?

Explore all of these available features below! 




Bulk share history set data

This feature allows you to share history set data for the table that you are bulk sharing. History sets in ServiceNow identify which particular records from an audited table contain historical information. History set records only contain a recent subset of historical information (within the past 30 days of inactivity). The following fields will be visible to you in list view for history sets:

Field

Description

IDAn identifier for the record whose history is being recorded
TableThe audited table for the record whose history is being recorded
Load TimeThe amount of time it took to generate the history set

For more information about ServiceNow history sets, see history sets and differences between audit and history sets

Here's how to enable the sharing of history set data for a bulk share:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to share the history set data for.

  3. In the resulting bulk share page, click the Miscellaneous tab, and check the Include history set box.

  4. Click Update

With DataSync Agent, this option will create a sys_history_xxx table on the target database. 

↑ Go to top of page




Clone a bulk share

This feature allows you to simply clone an existing bulk share or clone and immediately run an existing bulk share. 

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to clone.

  3. In the resulting bulk share page, scroll down to the Related Links section, and click Clone bulk configuration to create a duplicate of the current bulk share, or click Clone and run bulk share to create a duplicate of the current bulk share and then immediately execute the newly duplicated share. 


↑ Go to top of page




Preview a bulk share

This feature allows you to preview a bulk share before executing it. You can see how many records will be shared out and details about when the shared records were created.

What details will display on the preview page? (click to reveal)
  • The number of records that will be shared from the selected table.

  • If include attachments is selected, an estimate on the number of attachment records that will be shared. This will be the combined total of both sys_attachment and sys_attachment_doc records since both tables are required to re-construct attachments when replicating.

  • If include journal fields is selected, an estimate on the number of sys_journal_field records that will be shared.

  • If include audit log is selected, an estimate on the number of sys_audit records that will be shared.

  • A trend chart showing a monthly breakdown of when records were created in the selected table in the last 12 months.

  • If include all child tables is selected, the number of child records will be shared. 


Here's how to preview a bulk share:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to preview.

  3. In the resulting bulk share page, scroll down to the Related Links section, and click Preview.

  4. On the resulting preview page, you'll see a the number of records to be shared, and a graph showing when the records were created. Click the menu icon (≡) if you want to download the graph. Click Back to Bulk Share to return to the bulk share record. 


↑ Go to top of page




Run a bulk share as another user

This feature allows you to run a bulk share as another user, rather than the default system user that has access to all of the table's records. This feature can be useful when you want to limit the bulk share to a subset of data that certain users have access to, such as records in one domain in your domain separated instance.

Here's how to enable this feature:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to run as another user.

  3. In the resulting bulk share page, click the Miscellaneous tab, and click the search icon (magnifying glass) next to the Run as field.

  4. Choose the user to run as for your bulk share. 

  5. Click Update




Run a bulk share again

This feature allows you to run an existing bulk share again, once it has a status of Completed. You may want to run the same bulk share multiple times if your intent is to migrate the same table data intermittently. 

Here's how:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to run again. It must have a Completed status.

  3. In the resulting bulk share page, click Execute Again in the top right corner.


↑ Go to top of page




Bulk share only sys_id listed

This feature allows you to bulk share out only the sys_ids for specific records. This is useful for when you don't want to create filters for a bulk share. 

Here's how:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) for which you want to share only the sys_ids on specific records.

  3. In the resulting bulk share page, click the Data Enrichment tab, and check the Share only Sys IDs listed box.

  4. Then, scroll down and click the Sys IDs tab. This tab will only appear once you check the box from Step 3. 



  5. In the Sys IDs tab, click NewIf using Google Chrome, you may see a popup asking if you want to leave the site—click Leave.

  6. On the resulting page, type the sys_id for the first record you want to bulk share out in the Record Sys ID field.

  7. Click Submit.

  8. (Optional!) Repeat steps 5-7 for any other record sys_ids you want to bulk share out.

  9. Once you've added all sys_ids, click Update.


↑ Go to top of page




Share database views with bulk share

This feature allows you to properly share database views with a bulk share. By default, bulk shares run by querying for records from the selected table in 1,000 record increments, with records ordered by sys_id in ascending order. This is done to minimize performance impact on the ServiceNow instance. However, the records of database views do not have sys_ids and therefore cannot be shared in the same way. In order to properly share database views with a bulk share, you need to query for all records from the view at once.

To do this, you need to specify the glide.db.max_view_records integer system property to a number above the records in your view. If this property is not set, it defaults to 10,000. A good number to use is 999,999.


↑ Go to top of page




Limit the number of records shared

This feature allows you to limit the number of records shared once bulk share reaches a specific limit. 

Here's how to turn it on:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to limit the number of records shared on.

  3. In the resulting bulk share page, click the Runtime Settings tab, scroll to the Runtime Behavior section, and enter your desired record limit number in the Limit number of records shared field.

  4. Then, scroll down and click the Sys IDs tab. This tab will only appear once you check the box from Step 3. 

  5. Click Update.


↑ Go to top of page




Share updates since then

This feature allows you to share only records that have been inserted or updated since the last time the bulk share ran. This is useful for Scheduled Bulk Shares when you want to ensure you capture all records changed since the last execution of the bulk share.

This feature does not work with database views which do not have a sys_updated_on or sys_created_on field for their records.


Here's how to turn it on:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to configure.

  3. In the resulting bulk share page, click the Runtime Settings tab, scroll to the Triggers section, and check the Share updates since then box.

  4. Click Update.


↑ Go to top of page




Share as archive

This feature will share archived records as regular table records as part of a bulk share, if and when there is an archived table for a specified table in the bulk share. If you choose to share child records, it will also share the records from the archived child tables.

This feature is to be used with the temporal feature in the DataSync Agent (see step 3). 

Here's how to enable share as archive:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to share archives for.

  3. In the resulting bulk share page, use the Table name dropdown to select an archived table. 

  4. In the Runtime Settings tab, under the Runtime Behavior section, check the Share as archive box.

  5. Click Update.


  • Sharing additional records (i.e. attachments, journals, etc) are disabled
  • "Insert Only" will be disabled if "Share as archive" is enabled
  • Since archive tables do not retain table hierarchy (no parents or child tables), it stores one copy of the record to an archive leaf or child-most table. Therefore, “Include all child tables” and “Share base table” won’t behave the same as sharing a regular table. 

↑ Go to top of page




Distribute bulk share workload

This feature allows you to distribute the work load of your bulk shares by allowing them to run on multiple nodes in your instance. This is useful if you're running multiple bulk shares or scheduled bulk shares, since they will run on the same node by default, and this could impact performance. If you use this feature, nodes will only be used if they are online ad set to accept "any" scheduler.

Here's how to turn it on:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to configure.

  3. In the resulting bulk share page, click the Miscellaneous tab, and check the Advanced box. This will reveal an Advanced tab.



  4. Click the Advanced tab, and check the Distribute bulk share workload box.

  5. Click Update


↑ Go to top of page




Multiple bulk share jobs

This feature allows you to choose the number of jobs (1, 2, 4, 8, or 16) it will take to process the bulk shared records. Each job will process a subset of the total records, split by sys_id.

Here's how to customize the number of jobs:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to configure.

  3. In the resulting bulk share page, click the Miscellaneous tab, and check the Advanced box. This will reveal an Advanced tab (see Distribute bulk share workload for a screenshot).

  4. Click the Advanced tab, and under the Runtime Settings section, use the Number of Jobs dropdown to select the number of jobs.  

  5. Click Update.

This feature is not compatible with the following bulk share features:

  • Order by
  • Share only Sys IDs listed
  • Limit number of records shared
  • Enable confirmation features

↑ Go to top of page




Nodes to run on

This feature allows you to specify which node a bulk share will run on by referencing the sys_cluster_state table in ServiceNow. You need admin access (or a custom ACL rule) in order to see and set this value. You may want to run this feature in order to distribute the Perspectium processing across several nodes. 

Here's how to specify nodes:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to configure.

  3. In the resulting bulk share page, click the Miscellaneous tab, and check the Advanced box. This will reveal an Advanced tab (see Distribute bulk share workload for a screenshot).

  4. Click the Advanced tab, and under the Runtime Settings section, use the Node to run on field to search for and select a node.  

  5. Click Update.


↑ Go to top of page




Order by

This feature allows you to customize the order in which your records in bulk share are sent out by choosing which field (such as Active, Updated, Created On etc.) the records will be ordered by when sending out the records. For example, if you choose the Created On field, the bulk share will send the records in ascending order based on the date and time at which they were created. By default, bulk shares will order records by the Sys ID (sys_id) field when querying for records to share out. 

Here's how to send records in a specific order:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to configure.

  3. In the resulting bulk share page, click the Miscellaneous tab, and check the Advanced box. This will reveal an Advanced tab (see Distribute bulk share workload for a screenshot).

  4. Click the Advanced tab, and under the Runtime Settings section, use the Order by dropdown to select the field for the records to be ordered by when shared.  

  5. Click Update.


↑ Go to top of page




Share related lists

This feature allows you to share records from one or more related lists of the table being bulk shared. Here's how:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to configure.

  3. In the resulting bulk share page, click the Miscellaneous tab, and check the Advanced box. This will reveal an Advanced tab (see Distribute bulk share workload for a screenshot).

  4. Click the Advanced tab, and under the Runtime Settings section, check the Share related lists box. This will reveal a new Related lists field with a lock icon.



  5. Click the lock icon next to the Related lists field to unlock the field. This will reveal an empty field and a search bar.



  6. Use the search bar field to type your desired list, or click the search button to open all related lists of the table and select from there. Alternatively, you can click the Add/Remove multiple icon to add or remove multiple lists at once.

  7. Click Update.


↑ Go to top of page




Table replication in a subscribing instance

This feature allows you to create new tables in a subscribing ServiceNow instance by sharing a schema from one instance to another. This will create the table, field types, choices, labels, values, as well as the following:

  • Table sys IDs, columns, and the selection for choice fields
  • ACLs
  • Other table information, such as extends, auto-number, application process, etc.

Upon receiving the schema message, the table will be created in the instance and a generic subscribe record will be will be created to allow for replication of any records that come with the table schema.

Here's how to use share schema to replicate tables in a subscribing instance:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share > View Bulk Shares.

  2. Find and click into the bulk share (the clickable timestamp) that you want to share the schema for.

  3. In the resulting bulk share page, click the Miscellaneous tab, and check the Advanced box. This will reveal an Advanced tab (see Distribute bulk share workload for a screenshot).

  4. Click the Advanced tab, and under the Runtime Settings section, check the Share schema box. This will reveal a Target application dropdown.



  5. Select ServiceNow from the Target application dropdown.

  6. Click Update


↑ Go to top of page




Sharing to a queue with the Number of Queues option (Child Queues)

To optimize performance while avoid data loss, sharing is best done in parallel by using multiple bulk shares with multiple queues. So as to ease the configuration of having to configure multiple bulk shares and multiple queues, you can use a queue with the Number of Queues option specified to simply the setup.

This is an advanced feature for those who have heavy loads of data to share out. Use this feature with caution, as it may use more resources on your ServiceNow instance.

When creating a bulk share and selecting a shared queue for the target queue, if the shared queue specified has the Number of Queues option with a value greater than 1 such that there are child queues created, the bulk share will follow the same approach.

That is, the bulk share you create will become the base bulk share and child bulk shares will be created for sharing data out. There will be an equal amount of child bulk shares created to match the child queues from the shared queue specified. For example, if the shared queue has four child queues, there will be four child bulk shares created.

These child bulk shares will be named in the format of the <Bulk Share Name> <##>_<Epoch Creation Time>

Where

<Bulk Share Name> is the name of the base bulk share as specified in its Name field

<##> is a number from 01-16 to correspond with the number of child bulk shares i.e. if you have four child bulk shares they would be numbered 01, 02, 03 and 04

<Epoch Creation Time> is the Epoch time when the child bulk share was created.

For example if you name your base bulk share Split Example and you choose a shared queue with four child queues for your target queue, it would create four child bulk shares as follows:

Child bulk share are automatically created to split based on the sys_id along with any filter conditions you've added to the bulk share in order to optimize performance while avoid data loss. This setup is based on the number of child bulk shares (which is based on the number of child queues) and child bulk shares are created as follows:

Number of child bulk sharesFilter conditions for each child bulk share
2

Child bulk share 01: sys_id starts with 0, 1, 2, 3, 4, 5, 6, 7

Child bulk share 02: sys_id starts with 8, 9, A, B, C, D, E, F

4

Child bulk share 01: sys_id starts with 0, 1, 2, 3

Child bulk share 02: sys_id starts with 4, 5, 6, 7

Child bulk share 03: sys_id starts with 8, 9, A, B

Child bulk share 04: sys_id starts with C, D, E, F

8

Child bulk share 01: sys_id starts with 0, 1

Child bulk share 02: sys_id starts with 2, 3

Child bulk share 03: sys_id starts with 4, 5

Child bulk share 04: sys_id starts with 6, 7

Child bulk share 05: sys_id starts with 8, 9

Child bulk share 06: sys_id starts with A, B

Child bulk share 07: sys_id starts with C, D

Child bulk share 08: sys_id starts with E, F

16

Child bulk share 01: sys_id starts with 0

Child bulk share 02: sys_id starts with 1

Child bulk share 03: sys_id starts with 2

Child bulk share 04: sys_id starts with 3

Child bulk share 05: sys_id starts with 4

Child bulk share 06: sys_id starts with 5

Child bulk share 07: sys_id starts with 6

Child bulk share 08: sys_id starts with 7

Child bulk share 09: sys_id starts with 8

Child bulk share 10: sys_id starts with 9

Child bulk share 11: sys_id starts with A

Child bulk share 12: sys_id starts with B

Child bulk share 13: sys_id starts with C

Child bulk share 14: sys_id starts with D

Child bulk share 15: sys_id starts with E

Child bulk share 16: sys_id starts with F

The child bulk shares will be the ones that are executed and they reflect which child queue they're sharing data to. When a base bulk share is executed by choosing the Execute Now or Execute Again option, it will execute the child bulk shares and show the status of Executed to indicate that it's been executed to run its child bulk shares as shown in the image above.

(info) NOTE: 

  • Bulk shares using a shared queue with child queues are not compatible with the Order by, Conditional share, Share only Sys IDs listed, Truncate before share and the Limit number of records shared features.
  • When creating a new bulk share, the target queue field will only show base queues as selectable queue options and not child queues.
  • Base bulk shares will become read only after you click Execute Now to execute the child bulk shares. 
  • The child bulk shares are the ones actually running to share data and will contain its performance information. You can use the Execute Again option on the base bulk share to re-execute the child bulk shares.
  • The base bulk share will not contain any data on sharing since it isn't executing. It will contain the time when you clicked Execute Now/Execute Again to run the child bulk shares, storing this execution time in the Started field.
  • If the base bulk share and its child bulk shares have not yet been executed, you can edit the base bulk share's configuration so the changes are passed down to the child bulk shares. For example, if you update the base bulk share's encryption method, this encryption method will be reflected in the child bulk shares. Since the child bulk shares have not been executed yet, the child bulk shares will actually be deleted and then re-created with each update to the base bulk share to ensure configuration changes are captured properly.
  • Deleting a base bulk share will delete all child bulk shares.
  • If your bulk share is using a target queue with child queues and does not have child bulk shares, a message will display when the bulk share record is opened about using the Create child bulk shares for queues related list UI action.

Once you've configured bulk shares and shared queues to optimize sharing in parallel, you can then configure multiple MultiOutput jobs to push this data to the Perspectium Mesh (MBS).




Enable confirmation

This feature uses the Table Compare feature to compare records shared out to a target database to confirm they have been successfully received. With this option selected, the bulk share will send out compare messages for the DataSync Agent or meshlet to validate the database has all the sys_ids that have been bulk shared.

To enable this feature:

  1. Follow the steps to create a bulk share or go to a bulk share that you want to compare at Perspectium > Perspectium Core > Shares Bulk Share.

  2. On the bulk share page, switch to the Standard view.

  3. In the Advanced tab, in the Runtime Settings section, check the Enable confirmation box. 

  4. Execute the bulk share.

With this feature enabled, a bulk share created with the name Reshare <table name> <UTC datetime> (for example Reshare ticket 2023-12-07 03:27:50) will be created to continually reshare any records that the DataSync Agent/meshlet say are not found in the database. Each time this reshare bulk share is run, a new compare message will be sent for the Agent/meshlet to process and validate the records are now in the database.

If you don't want the Agent/meshlet to continually reshare records (because of an error in the database that needs to be fixed offline), you can uncheck the Enable confirmation box so the compare message is not sent the next time the reshare bulk share is run.




Share only selected fields

This feature allows you to include only specific fields in records shared out. This can be useful when you don't want to share out every field of the table to the target system.

If you are using this feature when bulk sharing to a database with the Table Compare feature (Enable confirmation option on the bulk share), see below for how to properly configure the DataSync Agent to work with both features.


Here's how to share only selected fields for a specific dynamic share:

  1. In your sharing ServiceNow instance, go to Perspectium > Perspectium Core Shares > Bulk Share View Bulk Shares.

  2. Find and click into the bulk share that you want to choose specific fields to share for.

  3. On the resulting page, in the Data Enrichment tab, check the Share only selected fields box. 

  4. At the bottom of the page, click the Share Selected Fields tab, and then click New. If using Google Chrome, you may see a popup asking if you want to leave the site—click Leave.

  5. On the resulting page, from the Field dropdown, choose the field that you want to share.

  6. Click Submit.

  7. (Optional!) Repeat steps 5 an 6 for additional fields as needed. 

  8. Once you've added all fields, click Update on the bulk share page to save the changes. 

Using Share only selected fields with Table Compare

When bulk sharing to a database with the Table Compare feature (Enable confirmation option on the bulk share), the DataSync Agent by default relies on comparing the sys_mod_count column in the table.

So if you are using the Share only selected fields option, to ensure the Agent is able to properly compare records, you can do one of the following two options:

  1. Include the sys_mod_count field as one of the selected fields you share or
  2. Modify the databases.xml to use a query without sys_mod_count. To set this up, do the following:
    1. Obtain version Krypton 8.0.7 and newer of the DataSync Agent that adds support to configure removing sys_mod_count comparisons
    2. Acquire the latest version of the databases.xml configuration file from Perspectium Support
    3. Move databases.xml into the Agent's conf folder i.e. <Perspectium_DataSync_Agent_Installed_Directory>/conf (same folder where you store agent.xml)
    4. Then, in databases.xml, you will need to change the following values in the <database_type></database_type> section that corresponds to your target database. For example, if you are sharing to a MySQL database, you would modify the values in the <database_type>mysql</database_typesection:

      Old ValueNew Value
      <most_recent_record_query>SELECT sys_id FROM %s WHERE sys_id = "%s" AND %s = "%s"</most_recent_record_query><most_recent_record_query>SELECT sys_id FROM %s WHERE sys_id = "%s"</most_recent_record_query>

      This change will remove the second part of the query (AND %s = "%s") where it checks for sys_mod_count.

      Snippet of databases.xml with this change:

      <database>
          <database_type>mysql</database_type>   
          .              
      	<most_recent_record_query>SELECT Id FROM %s WHERE Id = "%s"</most_recent_record_query>
          .           
       </database>	

       

    5. Save your changes and start the Agent




  • No labels