Versions Compared

Old Version 2

changes.mady.by.user user-1bc08

Saved on

compared with

New Version 3

changes.mady.by.user user-1bc08

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

You can configure your DataSync Agent to share data from ServiceNow or Salesforce to an Amazon Redshift database by changing some additional configurations in your agent.xml file. The Perspectium DataSync approach to sharing data from your app to your Amazon Redshift database uses an S3 bucket as a staging area where replicated data is stored before being populated in a Redshift table(s).


Prerequisites

(warning)(warning) First, you will need to Install the DataSync Agent

(warning)(warning) You will also need to create a ServiceNow dynamic share/bulk share or create a Salesforce dynamic share/bulk share.

(warning)(warning) Make sure to stop running your DataSync Agent before making any Agent configuration changes.

(warning)(warning) Finally, you will need to create an Amazon S3 bucket and launch an Amazon Redshift cluster.

Procedure

To set up your DataSync Agent to share application data to an Amazon Redshift database via an Amazon S3 bucket, follow these steps:

UI Steps


UI Step

Add joda-time and aws-java-sdk libraries

Add the following .jar files to your DataSync Agent's extlib directory:


UI Step

Install and configure your DataSync Agent

Follow the steps to Install the DataSync Agent or the DataSync for Salesforce Agent and make sure that your agent.xml file is configured correctly.


UI Step

Access your agent.xml configuration file

Navigate to the directory where you saved your agent.xml file when installing your DataSync Agent.


UI Step

Update database directives to point to Amazon Redshift

Open your agent.xml file in a text editing application. Then, locate the <task> directive(s) within your <subscribe> directive, and update the following database-related directives:

DirectiveValue to enter
<database_type>redshift
<database_server>

Endpoint URL for your Redshift database

<database_port>Port number for your Redshift database
<database_user>Username used to access your Redshift database
<database_password>Password used to access your Redshift database



UI Step

Update the <handler> directive

Within the <subscribe> → <task> directive(s), enter a new value for the <handler> directive:

DirectiveValue to enter
<handler>com.perspectium.replicator.sql.RedshiftSQLSubscriber



UI Step

Add directives for AWS

Still within the <subscribe> → <task> directive(s), nest the following directives:

DirectiveDescription
<access_key>

Access Key associated with your AWS account

<secret_access_key>Secret Access Key associated with your AWS account
<region>Region that your Redshift server resides in
<s3_bucket>Name of your AWS S3 bucket
(info) NOTE: Optionally, you can add the <batch_delete/> directive to enable batch processing of row deletes for enhanced data replication throughput. 


Your agent.xml file should look similar to the example shown below:

Code Block
languagexml
themeEclipse
<?xml version="1.0" encoding="ISO-8859-1" standalone="no"?>
<config>
   <agent>
      <share/>
      <subscribe>
	  <task>
        	<task_name>test_redshift1_subscribe</task_name>
            <message_connection password="encrypted:vlOtU71yu8N/EFIJH85SSBtaIt7qEEfvqiqft9VZyYE=" user="exampleuser">amqps://example.perspectium.net</message_connection>
            <instance_connection password="encrypted:vlOtU71yu8N/EFIJH85SSBtaIt7qEEfvqiqft9VZyYE=" user="example.user">https://myinstance.service-now.com</instance_connection>
            <handler>com.perspectium.replicator.sql.RedshiftSQLSubscriber</handler>
            <decryption_key>The cow jumped over the moon</decryption_key>
            <database_type>redshift</database_type>
            <database_server>pspexample.example.region.redshift.amazonaws.com</database_server>
            <database_port>5439</database_port>
            <database_user>exampleuser</database_user>
            <database_password>encrypted:vlOtU71yu8N/EFIJH85SSAZYM2li7vh9u6asPldRqe0=</database_password>
            <database>pspdev</database>
			<access_key>AccessKey</access_key>
            <secret_access_key>SecretAccessKey</secret_access_key>
            <region>us-west-2</region>
            <s3_bucket>examples3bucket</s3_bucket>
	  </task>
      </subscribe>
   <max_reads_per_connect>4000</max_reads_per_connect>
   <polling_interval>30</polling_interval>
   </agent>
</config>



UI Step

Save your agent.xml file

Save the changes you made to your agent.xml file and close the file.


UI Step

Run your DataSync Agent

After configuring your agent.xml file to support replication to your Amazon Redshift database, start running your DataSync Agent again.