Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

UI Button
colormagenta
tooltipInformation on this page pertains to the Perspectium Fluorine release
titleFluorine
urlhttps://docs.perspectium.com/display/fluorine


Perspectium DataSync Agents support the replication of data from your app to an Amazon Web Services (AWS) S3 bucket, or an AWS S3 Subscriber Agent. By configuring your AWS S3 Subscriber Agent, data from your app can be replicated and then saved as either .json or .xml file(s) in your AWS S3 bucket.


Prerequisites


(warning) First, you will need to set up one of the Perspectium DataSync Agents.

(warning) You should also stop running your DataSync Agent before making any Agent configuration changes.

Procedure

To configure your DataSync Agent to run as an AWS S3 Subscriber Agent, follow these steps:


UI Steps


UI Step

Add joda-time and aws-java-sdk libraries

Add the following .jar files to your DataSync Agent's extlib directory:

While newer versions may work, they have not been tested and it is suggested you use the versions listed above which have been confirmed to work with this release. 


UI Step

Access your agent.xml configuration file

Navigate to the directory where you saved your agent.xml file when installing your DataSync Agent.


UI Step

Delete database directives

Open your agent.xml file in a text editing application and delete the following directives nested within the <task> tag: 

  • <database_type>
  • <database_server>
  • <database_port>
  • <database_user>
  • <database_password>
  • <database_parms>
  • <database_column_max_size>
  • <database>


UI Step

Update the values for <task_name> and <handler>

Locate the <task_name> and <handler> directives nested within the <task> tag and update their values as follows:

DirectiveUpdate value to...
<task_name>s3_agent_subscribe
<handler>com.perspectium.replicator.sqlfile.S3Subscriber



UI Step

Add AWS directives

Within the <task> tag, nest the following directives:

DirectiveDescriptionRequired?
<access_key>Access Key associated with your AWS accountYes
<secret_access_key>Secret Access Key associated with your AWS accountYes
<region>Region that your AWS S3 bucket resides inYes
<s3_bucket>Name of your AWS S3 bucketYes
<file_format>

Format you want to save your data records in

e.g., json or xml

Yes



UI Step

Save your agent.xml

Save the changes you've made to your agent.xml and close the file. An example agent.xml configuration for an AWS S3 Subscriber Agent is shown below:

Code Block
languagexml
themeEclipse
<?xml version="1.0" encoding="UTF-8"?>
<config>
   <agent>
      <share />
      <subscribe>
         <task>
            <task_name>s3_agent_subscribe</task_name>
            <message_connection password="password" user="user">https://mesh.perspectium.net</message_connection>
    	    <instance_connection password="password" user="user">https://myinstance.service-now.com</instance_connection>   
            <handler>com.perspectium.replicator.file.S3Subscriber</handler>
            <decryption_key>The cow jumped over the moon</decryption_key>
            <access_key>AccessKey</access_key>
            <secret_access_key>SecretAccessKey</secret_access_key>
            <region>us-west-2</region>
            <s3_bucket>examples3bucket</s3_bucket>
            <file_format>json</file_format>
         </task>
      </subscribe>
      <polling_interval>40</polling_interval>
   </agent>
</config>

(info) NOTE: In this configuration example, your data records will be saved in your AWS S3 bucket as one file. To save each record from your app as an individual file in your AWS S3 bucket, use the following agent.xml configuration example as a guide:

Code Block
languagexml
themeEclipse
<?xml version="1.0" encoding="UTF-8"?>
<config>
   <agent>
      <share />
      <subscribe>
         <task>
            <task_name>s3_agent_subscribe</task_name>
            <message_connection password="password" user="user">https://<customer>.perspectium.net</message_connection>
    	    <instance_connection password="password" user="user">https://<instance>.service-now.com</instance_connection>   
            <handler>com.perspectium.replicator.file.S3Subscriber</handler>
            <decryption_key>The cow jumped over the moon</decryption_key>
            <access_key>AccessKey</access_key>
            <secret_access_key>SecretAccessKey</secret_access_key>
            <region>us-west-2</region>
            <s3_bucket>examples3bucket</s3_bucket>
            <file_format>json</file_format>
            <one_record_per_file/>
         </task>
      </subscribe>
      <polling_interval>40</polling_interval>
   </agent>
</config>

Files saved in the AWS S3 bucket will be named <task_name>.<randomized_unique_identifier>.<file_format>.  Using the above configuration example, a file would be named s3_agent_subscribe.00b470b7-901c-4447-9316-023a265d632f.json.


UI Step

Run your AWS S3 Subscriber DataSync Agent

After configuring your agent.xml file to support your AWS S3 Subscriber Agent, start running your DataSync Agent again.





Similar topics


Content by Label
showLabelsfalse
max5
showSpacefalse
sortmodified
cqllabel = "data-sync" and space = currentSpace()

Contact Perspectium Support


US: 1 888 620 8880

UK: 44 208 068 5953

support@perspectium.com