Attachments are stored in ServiceNow in two separate tables, sys_attachment and sys_attachment_doc. The sys_attachment record holds information about the attachment (such as the file name and file size), while the sys_attachment_doc table contains the actual byte data of the attachment separated into different chunks so as to not have one record take up too much space in a table row. Thus, sending attachment(s) to your database with the DataSync Agent will store the attachment(s) in the two tables just as they are stored in ServiceNow.
Merging Attachments in the Database
For edge encrypted attachments, ServiceNow will encrypt the attachment byte data when the attachment is in one piece and then split up the encrypted content into chunks for saving in the sys_attachment_doc table. As a result, this content is not usable since each sys_attachment_doc record is only one portion of the encrypted content.
By including a SysAttachmentHandler in the DataSync Agent configuration, the Agent will query the sys_attachment and sys_attachment_doc tables and merge the attachment records back together into one record. The records are removed from these two tables and the merged attachment records can then be stored in whole in the database (in a new table called attachments) or shared out as messages back to the Integration Mesh.
NOTE: Storing the attachments as complete records in the database will lead to large table rows. Ensure your database has the necessary storage space to save these large records.
When saving attachments into the database, an attachments table gets created to store the fully built attachments and has the following structure:
Column | SQL Data Type* | Description |
---|---|---|
sys_id | varchar(40) | sys_id of the sys_attachment record |
table_name | varchar(128) | Name of the table that the file is attached to |
file_name | varchar(128) | Name of the file |
content_type | varchar(128) | Type of file |
table_sys_id | varchar(32) | sys_id of the record the file is attached to |
position | bigint | Will have a value of 0 to represent a full attachment starting at the beginning (0) position |
attachment | varchar(max) | Attachment's complete byte data. This column's data can be used to get the whole attachment as one viewable file. |
*Data types shown are for MySQL/SQL Server. Other databases will create columns in their equivalent data types (for example, Oracle will create the position field as a number field data type, its equivalent of bigint).
Saving Locally or to a Cloud
By including a SysAttachmentHandler in the DataSync Agent configuration, the Agent will query the sys_attachment and sys_attachment_doc tables and construct the full attachment to be saved locally or to a cloud storage service such as Amazon S3, Azure Blob Storage, or Google Cloud Storage. The records are then removed from these two tables.
When adding a SysAttachmentHandler, all database configurations must match the database configurations from the subscribe task to ensure the correct sys_attachment and sys_attachment_doc tables are being queried and processed
Saving Locally
When saving locally, you will need the following directives under <share><task>:
Directive | Description | Required? |
---|---|---|
<destination> | Final location of the attachment (local, S3, GCS, ABS) | yes |
<directory> | Where the attachment will be saved locally | yes |
Saving to a Cloud
When saving to a cloud service, you will need the following directives under <share><task>:
Amazon S3
Directive | Description | Required? |
---|---|---|
<destination> | Final location of the attachment (local, S3, GCS, ABS) | Yes |
<directory> | Where the attachment will be saved locally | Yes |
<access_key> | Access Key associated with your AWS account | Yes, ONLY when the DataSync Agent is not installed on an EC2 instance. |
<region> | Region that your AWS S3 bucket resides in NOTE: If a region is specified for an Agent on an EC2 instance, then the region must match the region of the EC2 instance and the region of the S3 bucket | No |
<bucket_name> | Name of your AWS S3 bucket, including subdirectories if desired, to specify where the records will be uploaded e.g. bucketName/folder1/folder2. For example, with <bucket_name>psp-bucket</bucket_name> will save records into the psp-bucket S3 bucket. | Yes |
<secret_access_key> | Secret Access Key associated with your AWS account | Yes, ONLY when the DataSync Agent is not installed on an EC2 instance. |
Azure Blob Storage
Directive | Description | Required? |
---|---|---|
<destination> | Final location of the attachment (local, S3, GCS, ABS) | Yes |
<directory> | Where the attachment will be saved locally | Yes |
<connection_string> | String provided by Azure for connecting and uploading to the blob storage | Yes |
<bucket_name> | Name of your Azure Blob Storage container, including subdirectories if desired, to specify where the records will be uploaded e.g. bucketName/folder1/folder2. For example, with <bucket_name>psp-bucket</bucket_name> will save records into the psp-bucket Azure Blob Storage container. | Yes |
Google Cloud Storage
Directive | Description | Required? |
---|---|---|
<destination> | Final location of the attachment (local, S3, GCS, ABS) | Yes |
<directory> | Where the attachment will be saved locally | Yes |
<key_path> | JSON file provided by Googlefor connecting and uploading to the cloud | Yes |
<project_id> | Identifier of the Google project containing the bucket | Yes |
<bucket_name> | Name of your Google Cloud Storage bucket, including subdirectories if desired, to specify where the records will be uploaded e.g. bucketName/folder1/folder2. For example, with <bucket_name>psp-bucket</bucket_name> will save records into the psp-bucket Google Cloud Storage bucket | Yes |