Skip to main content

Integrating GCP Pub Sub with Orkes Conductor

This developer guide includes the steps to integrate Google Cloud Platform (GCP) Pub Sub with Orkes Conductor. This integration lets you connect your GCP Pub Sub to Conductor to publish and receive messages from topics.

Get Configuration Credentials from GCP Pub Sub

Before beginning the integration process in Orkes Conductor, you must get specific configuration credentials such as project ID, subscription ID, and Service Account JSON from the GCP console.

To get the project ID:

  1. Login to Google Cloud Console and create a project.
  2. Click the drop-down menu on the top left of the console to select your desired project.
  3. The Project ID will be displayed on the dashboard below the project name.

Get project ID from Google Cloud Console

Refer to the official documentation on creating and managing projects in GCP for more details.

To get the subscription ID:

  1. Go to the Pub/Sub section in the Cloud Console. (From the left menu navigation, go to Products & solutions > Categories - Analytics > Pub/Sub)
  2. From the left menu, click Subscriptions and choose the subscription you want to use, or create a new one.
  3. The Subscription ID will be displayed as shown below:

Get subscription ID from Google Cloud Console

To get the Service Account JSON:

  1. From the left menu, navigate to the IAM & Admin section.
  2. Select Service Accounts from the left menu.
  3. Click on an existing service account you want to use or create a new one.
  4. Under the Keys sub-tab, click Add Key.

Get Service Account JSON from Google Cloud Console

  1. Choose the option Create new key.
  2. Choose the key type as JSON and click Create to generate the JSON key.

Get Service Account JSON key from Google Cloud Console

Integrating with GCP Pub Sub as a Message Broker

Once you have the required configuration credentials from GCP Pub Sub, let’s integrate with Orkes Conductor.

  1. Navigate to Integrations from the left menu on the Conductor cluster.
  2. Click + New integration from the top-right corner of your window.
  3. Under the Message Broker section, choose GCP Pub Sub.
  4. Click +Add and provide the following parameters:

Integration configuration for GCP Pub Sub

Integration NameA name to identify your integration.
Project IDProvide the project ID containing the topic. Refer to the previous section on how to get the project ID.
Subscription IDProvide the subscription ID. Refer to the previous section on how to get the subscription ID.
Upload Service Account JSONUpload the Service Account JSON file, which is a key file containing the credentials for authenticating the Orkes Conductor cluster with the GCP Pub Sub services. Refer to the previous section on how to generate the service account JSON.
DescriptionProvide a description of the integration.
  1. You can toggle on the Active button to activate the integration instantly.
  2. Click Save.

Creating Event Handlers in Orkes Conductor

The integration is created successfully now. The next step is to create an event handler in Orkes Conductor.

To do this:

  1. Navigate to Definitions > Event Handler from the left menu on your Conductor console.
  2. Click the + Define event handler option from the top-right corner of the window.
  3. Create an event handler with the following configurations:

Configuring Event Handler for GCP Pub Sub Integration

Event Handler ParametersDescription
NameProvide a name to identify your event handler definition.
EventProvide the event integration you have created in the following format:

Type : Config Name : Topic Name

Example: gcppubsub:gcp-test:topic-name

Notes: The drop-down automatically lists the integration you’ve added to the Conductor cluster. You can choose that and add the topic name you want to publish/receive messages.
ConditionProvide the ECMAScript to control the message processing if required. Check out the event handler documentation for more details.
ActionsChoose the required actions to be carried out on sending/receiving the events from GCP Pub Sub. It can take the following values:
  • Complete Task
  • Terminate Workflow
  • Update Variables
  • Fail Task
  • Start Workflow
Each type of action requires and supports a certain set of input parameters. Check out the event handler documentation for more details.
ActiveSet this to true or false. It determines if the event handler is running or not.

A sample JSON for the event handler is as follows:

"name": "gcp-pub-sub-event-handler",
"event": "gcppubsub:gcp-pub-sub-test:gcp-topic-name",
"condition": "true",
"actions": [
"action": "start_workflow",
"start_workflow": {
"name": "event-handler-test",
"version": 1
"expandInlineJSON": false
"active": true,
"evaluatorType": "javascript",
"createdBy": ""

RBAC - Governance on who can use Integrations

Once the integration is added, the next step is determining who can access these integrations.

The permissions can be granted to applications/groups within the Orkes Conductor cluster.

To provide explicit permission to Groups:

  1. From the left menu on your Conductor cluster, navigate to Access Control > Groups.
  2. Create a new group or choose an existing group.
  3. Under the Permissions section, click +Add Permission.
  4. From the Integrations sub-tab, choose the integration with required permissions.
  5. Click Add Permissions. This ensures that all the group members can access these integration models in their workflows.

Configuring RBAC for GCP Pub Sub Integration

Similarly, you can also provide permissions to applications.

Creating Workflow in Orkes Conductor

This step involves creating a workflow with an event task in Orkes Conductor. Here, we are utilizing the GCP Pub Sub topic as a sink for the event.

You can quickly build a workflow from UI in Orkes Conductor.

For this,

  1. Navigate to Definitions > Workflow, and click the Define Workflow button.
  2. Create a workflow and add an event task at the required point with the Sink in the format gcppubsub:gcp-pub-sub-test:gcp-topic-name, where “gcp-pub-sub-test” is the integration name and “gcp-topic-name” is the topic to which Conductor should send/receive messages.

Event task for GCP Pub Sub Integration in Orkes Conductor

Executing Workflow in Orkes Conductor

The workflow can be run using different methods. You can use the Run Workflow button for quick testing, as shown in the image below:

Running workflow from Orkes Conductor UI

Upon successful execution, you can verify the message's delivery through the GCP Pub Sub console.

The action added in the event handler definition was to start the workflow event-handler-test. You can verify the same from the Executions > Workflow page.

Starting workflow on sending events