Integrating AWS SQS with Orkes Conductor

This developer guide includes the steps to integrate AWS SQS with Orkes Conductor. This integration lets you connect your AWS SQS to Conductor to publish and receive messages from queues.

Get Configuration Credentials from AWS SQS

Before beginning the integration process in Orkes Conductor, you must get specific configuration credentials from your Amazon SQS account.

• AWS Account ID & region where the SQS is located.
• Amazon Resource Name (ARN) to identify & access the queue. ARN is generally of the format arn:aws:sqs:region:account-id:queue-name.
• External ID - When you assume a role belonging to another account in AWS, you need to provide the external ID, an ID that can be used in an IAM role trust policy to designate the person to assume the role. Learn more.
• Access Key & Secret from AWS SQS account.

Refer to the AWS SQS official documentation on how to get these credentials.

Integrating with AWS SQS as a Message Broker

Now, you have the required configuration credentials from AWS SQS. Let’s integrate with Orkes Conductor.

1. Navigate to Integrations from the left menu on the Conductor cluster.
2. Click + New integration from the top-right corner of your window.
3. Under the Message Broker section, choose AWS SQS. Click +Add and provide the following parameters:

Parameters	Description
Name	Provide a name to identify your event handler definition.
Connection Type	Choose the required connection type. Depending upon how the connection is to be established, it can take the following values: Current Conductor Role - Choose this if you are using the current Conductor role to establish the connection. Assume External Role - Choose this if you are assuming a role belonging to another AWS account. Learn more. Access Key/Secret - Choose this if you are establishing the connection using the access key and secret.
Region	Provide the valid AWS region where the SQS is located.
Account ID	Provide your AWS Account ID. This field is optional. Note: If ARN is not used for the “Sink” in the workflow definition, the account ID should be used.
Role ARN	Specify the Amazon Resource Name (ARN) required for setting up the connection. Note: This field is applicable only if the Connection Type is chosen as Assume External Role.
External ID	If applicable, provide the external ID for assuming the role. Note:This field is applicable only if the Connection Type is chosen as Assume External Role.
Access Key	Provide the AWS Access Key. Note:This field is applicable only if the Connection Type is chosen as Access Key/Secret.
Access Secret	Provide the AWS Access Secret. Note:This field is applicable only if the Connection Type is chosen as Access Key/Secret.
Description	Provide a description of the integration.

5. You can toggle on the Active button to activate the integration instantly.
6. Click Save.

Creating Event Handlers in Orkes Conductor

The integration is created successfully now. The next step is to create an event handler in Orkes Conductor.

To do this:

1. Navigate to Definitions > Event Handler from the left menu on your Conductor console.
2. Click the + Define event handler option from the top-right corner of the window.
3. Create an event handler with the following configurations:

   

Event Handler Parameters	Description
Name	Provide a name to identify your event handler definition.
Event	Provide the event integration you have created in the following format: Type : Config Name : Queue Name Example: sqs:john-test:arn:aws:sqs:us-east-1:XXXXXXXXX:eventstest-1 Notes: The drop-down automatically lists the integration you’ve added to the Conductor cluster. You can choose that and add the queue name you want to publish/receive messages.
Condition	Provide the ECMAScript to control the message processing if required. Check out the event handler documentation for more details.
Actions	Choose the required actions to be carried out on sending/receiving the events to/from AWS SQS. It can take the following values: Complete Task Terminate Workflow Update Variables Fail Task Start Workflow Each type of action requires and supports a certain set of input parameters. Check out the event handler documentation for more details.
Active	Set this to true or false. It determines if the event handler is running or not.

A sample JSON for the event handler is as follows:

{
 "name": "sqs-event-handler",
 "event": "sqs:sqstest:arn:aws:sqs:us-east-1:XXXXXXXXXXXXXX:eventstest-1",
 "condition": "true",
 "actions": [
   {
     "action": "start_workflow",
     "start_workflow": {
       "name": "event-handler-test",
       "version": 1,
       "correlationId": "",
       "input": {
         "payload": "${$}"
       }
     },
     "expandInlineJSON": false
   }
 ],
 "active": true,
 "evaluatorType": "javascript"
}

RBAC - Governance on who can use Integrations

Once the integration is added, the next step is to determine who can access these integrations.

The permissions can be granted to applications/groups within the Orkes Conductor cluster.

To provide explicit permission to Groups:

1. From the left menu on your Conductor cluster, navigate to Access Control > Groups.
2. Create a new group or choose an existing group.
3. Under the Permissions section, click +Add Permission.
4. From the Integrations sub-tab, choose the integration with required permissions.
5. Click Add Permissions. This ensures that all the group members can access these integration models in their workflows.

Similarly, you can also provide permissions to applications.

Creating Workflow in Orkes Conductor

This step involves creating a workflow with an event task in Orkes Conductor. Here, we are utilizing the SQS queue as a sink for the event.

You can quickly build a workflow from UI in Orkes Conductor.

For this,

1. Navigate to Definitions > Workflow, and click the Define Workflow button.
2. Create a workflow with an event task with the Sink in the format sqs:john-test:arn:aws:sqs:us-east-1:XXXXXXXXX:eventstest-1, where “john-test” is the integration name and “arn:aws:sqs:us-east-1:XXXXXXXXX:eventstest-1” is the ARN pointing to the SQS queue to which the Conductor should send/receive messages.

Executing Workflow in Orkes Conductor

The workflow can be run using different methods. You can use the Run Workflow button for quick testing, as shown in the image below:

Upon successful execution, go to AWS Web Console for that SQS queue, poll for messages, and inspect the payload to verify the message is consumed.

The action added in the event handler definition was to start the workflow “event-handler-test”. You can verify the same from the Executions > Workflow page.