Amazon MSK Integration with Orkes Conductor
The Amazon MSK configuration is deprecated. For new configurations, use Apache Kafka.
To use the Event task or enable Change Data Capture (CDC) in Orkes Conductor, you must integrate your Conductor cluster with the necessary message brokers. This guide explains how to integrate Amazon MSK with Orkes Conductor to publish and receive messages from topics. Here’s an overview:
- Get the required credentials from Amazon MSK.
- Configure a new Amazon MSK integration in Orkes Conductor.
- Set access limits to the message broker to govern which applications or groups can use them.
Step 1: Get the Amazon MSK credentials
To integrate Amazon MSK with Orkes Conductor, retrieve the following credentials from your Amazon MSK console:
- Bootstrap server
- Username and Password
- Consumer Group ID
Refer to the official Amazon MSK documentation for more details.
Step 2: Add an integration for Amazon MSK
After obtaining the credentials, add an Amazon MSK integration to your Conductor cluster.
To create an Amazon MSK integration:
- Go to Integrations from the left navigation menu on your Conductor cluster.
- Select + New integration.
- In the Message Broker section, choose Amazon MSK.
- Select + Add and enter the following parameters:
| Paremeters | Description |
|---|---|
| Integration name | A name for the integration. |
| Bootstrap Server | The bootstrap server of the Amazon MSK cluster. |
| Sending Protocol | The sending protocol for the integration. Supported values:
|
| Connection Security | The security mechanism for establishing connection. Supported values:
|
| Username | The username of the Amazon MSK account. |
| Password | The password associated with the username. |
| Consumer Group ID | The Consumer Group ID from Kafka. This unique identifier helps manage message processing, load balancing, and fault tolerance within consumer groups. |
| Description | A description of the integration. |
- (Optional) Toggle the Active button off if you don’t want to activate the integration instantly.
- Select Save.
Step 3: Set access limits to integration
Once the integration is configured, set access controls to manage which applications or groups can use the message broker.
To provide access to an application or group:
- Go to Access Control > Applications or Groups from the left navigation menu on your Conductor cluster.
- Create a new group/application or select an existing one.
- In the Permissions section, select + Add Permission.
- In the Integration tab, select the required message broker and toggle the necessary permissions.
The group or application can now access the message broker according to the configured permissions.
Next steps
With the integration in place, you can now:
- Create Event Handlers for use in Event tasks.
- Enable Change Data Capture (CDC) to send workflow state changes to message brokers.