Using Variable Idempotency Keys in Webhook-Triggered Workflows

This tutorial demonstrates how to build an idempotent workflow using Orkes Conductor that creates a Jira ticket only once for each unique webhook event. By using a workflow input variable as the idempotency key in a webhook-triggered workflow, you can avoid creating duplicate tickets when the same webhook is triggered multiple times.

This approach is ideal for teams connecting alerting systems, bug reporters, or internal tools to Jira, where event retries or duplication are common.

In this tutorial, you will:

1. Generate an API token and configure project permissions in Jira.
2. Save the token as a secret in Orkes Conductor.
3. Create the ticket automation flow.
4. Create a webhook in Orkes Conductor.
5. Test the workflow using cURL

Prerequisites

To follow along, make sure you have access to the following:

• A free account in Orkes Developer Playground.
• An Atlassian account with access to Jira.

Ticket automation workflow

You’ll create an automation that:

• Accepts bug reports via a custom webhook.
• Triggers a Conductor workflow to create a Jira ticket.
• Prevents duplicate ticket creation for the same bug report using idempotency keys.

Here is the ticket automation workflow that you’ll build in this tutorial:

Step 1: Generate an API token and configure project permissions in Jira

This tutorial uses Jira to create tickets from event payloads received through a custom webhook. To enable this, you need to:

• Generate an API token.
• Update the Jira project permission.

Generate an API token

To authenticate the workflow against Jira, generate an API token from your Atlassian account. This token enables API access to Jira.

To generate an API token:

1. Log in to the Atlassian account.
2. Go to Security > API tokens.
3. Select Create API token.

4. Enter a name for the token and set an expiration.
5. Select Create.
6. Copy and securely store the token.

Update the Jira project permission

To allow ticket creation, the user must have the appropriate permissions in the Jira project.

To update the project permission in Jira:

1. In Jira, select Projects from the left navigation.
2. Select the Jira project where bug tickets will be created. For example, you can use the default project, Demo service project.
3. In your project, select Project settings.

4. In the left navigation menu, go to Access > People and access.
5. Assign the user with both Administrators and Service Desk Team roles.

Step 2: Store the token as a secret in Orkes Conductor

After generating the API token, encode it with Base64 and store it securely as a secret in Orkes Conductor so it can be used for authentication without being exposed during execution.

Jira uses Basic Authentication, which requires a Base64-encoded string containing your Atlassian email address and the API token.

Encode the token with Base64

To create the basic authentication string:

1. Open a Terminal.
2. Run the following command, replacing the placeholders with your credentials:

echo -n "your-email@example.com:<your_api_token>" | base64

3. Copy the output, which is your Base64-encoded authentication string.

Store as a secret in Orkes Conductor

To store the basic authentication token as a secret:

1. Go to Definitions > Secret from the left navigation menu on your Conductor cluster.
2. Select + Add secret.
3. Set the Secret name as atlassian-token.
4. Paste the Base64 encoded string as the Secret value.

5. Select Add to save the secret.

Step 3: Create the ticket automation workflow

Next, define a workflow in Orkes Conductor that creates a Jira ticket when triggered by a webhook. The workflow includes an HTTP task that sends an API request to Jira.

Create workflow

To create the ticket automation workflow:

1. Go to Definitions > Workflows and select + Define workflow.
2. In the Code tab, paste the following JSON:

{
 "name": "ticket-automation-workflow",
 "description": "Creates a Jira task for a new bug reported via webhook.",
 "version": 1,
 "tasks": [
   {
     "name": "create_jira_task",
     "taskReferenceName": "create_jira",
     "inputParameters": {
       "http_request": {
         "uri": "<YOUR-ATLASSIAN-PORTAL-URL>/rest/api/3/issue",
         "method": "POST",
         "headers": {
           "Authorization": "Basic ${workflow.secrets.atlassian-token}",
           "Content-Type": "application/json"
         },
         "body": {
           "fields": {
             "project": {
               "key": "<YOUR-JIRA-PROJECT-KEY>"
             },
             "summary": "${workflow.input.title}",
             "issuetype": {
               "name": "Task"
             },
             "description": {
               "type": "doc",
               "version": 1,
               "content": [
                 {
                   "type": "paragraph",
                   "content": [
                     {
                       "text": "Reported by: ${workflow.input.reported_by}\nBug ID: ${workflow.input.bug_id}",
                       "type": "text"
                     }
                   ]
                 }
               ]
             }
           }
         }
       }
     },
     "type": "HTTP"
   }
 ],
 "inputParameters": [
   "bug_id",
   "title",
   "reported_by"
 ],
 "schemaVersion": 2
}

3. Select Save > Confirm.

Edit workflow

Next, update the placeholders in your workflow definition:

• On line 13, update <YOUR-ATLASSIAN-PORTAL-URL> with your Jira portal URL. For example, if your Jira URL is https://your-atlassian.atlassian.net/, the endpoint URL will be https://your-atlassian.atlassian.net.rest/api/3/issue.
• On line 22, update <YOUR-JIRA-PROJECT-KEY> with your Jira project key. If you’re using the default demo project, the key is DEMO. You can find your project key on the Jira project settings page.

Select Save to update the workflow. The Conductor UI highlights the changes in real-time, making it easier to review and understand the edits before saving the workflow.

Step 4: Create a webhook in Orkes Conductor

In this example, we use a Custom webhook to simulate input from an internal bot. However, you can create a webhook based on your requirements using any supported platforms or a Custom webhook for integration with other systems.

Create a webhook that triggers the workflow you defined. Configure it to use the bug_id input variable as the idempotency key to prevent duplicate executions for the same event.

To create a webhook:

1. Go to Definitions > Webhook from the left navigation menu on your Conductor cluster.
2. Select + New webhook.
3. Configure the following parameters:

• In Webhook name, enter a webhook name such as CreateJiraTickets.
• Select Source platform as Custom.
• Set a request Header for the webhook.
• Enable Start workflow when webhook event comes.
• In the Workflow name, select ticket-automation-workflow.
• In the Idompotency key, enter ${workflow.input.bug_id}.
• Set the Idompotency strategy as Return Existing.

4. Select Save.

After saving, Conductor generates an unverified webhook URL.

URL verification depends on the type of webhook. For Custom webhooks, Conductor uses header-based verification. This means incoming requests must include headers that match the exact names and values specified during configuration. If the request doesn’t match all required headers, it will be ignored. The URL is marked as verified when the first valid request with matching headers is received.

By using ${workflow.input.bug_id} as the idempotency key, Conductor will prevent duplicate workflow executions for the same bug report, even if the webhook is triggered multiple times.

Step 5: Test the workflow using cURL

Simulate a sample input using a cURL command. You can adapt this to accept inputs from various internal sources, such as email systems, monitoring tools, or internal bots.

To send a cURL request:

1. Open Terminal.
2. Run the following command, replacing the placeholders with your actual values:

curl -X POST "<YOUR-CONDUCTOR-WEBHOOK-URL>" \
  -H "Content-Type: application/json" \
  -H "<YOUR-HEADER-NAME>: <YOUR-HEADER-VALUE>" \
  -d '{
    "bug_id": "BUG-2045",
    "title": "Java SDK workflow execution bug",
    "reported_by": "Internal Bot"
  }'

This verifies the Webhook URL in Conductor and triggers the ticket-automation-workflow.

To confirm the workflow execution, navigate to Executions > Workflow in Orkes Conductor and verify the execution.

In your Jira project, go to Queues > All Open to confirm that the ticket has been created. Then, open the ticket to verify that the description has been passed through the webhook correctly.

Test the idempotency behavior

Consider a scenario where the same bug report is submitted multiple times due to a system glitch or network retry. If not handled properly, this could lead to hundreds or thousands of duplicate tickets in large systems.

To simulate this behavior:

1. Re-run the same cURL request using the same bug_id value as before.

curl -X POST "<YOUR-CONDUCTOR-WEBHOOK-URL>" \
  -H "Content-Type: application/json" \
  -H "<YOUR-HEADER-NAME>: <YOUR-HEADER-VALUE>" \
  -d '{
    "bug_id": "BUG-2045",
    "title": "Java SDK workflow execution bug",
    "reported_by": "Internal Bot"
  }'

2. Check the webhook activity in Conductor. A new event is received, but no new workflow is triggered. You can verify this by comparing timestamps.

This confirms that Orkes Conductor correctly recognized the event as a duplicate. Since the webhook was configured with bug_id as the idempotency key and the strategy set to Return Existing, it avoids initiating a new workflow and instead references the original execution.

If you send a request with a different bug_id, Conductor treats it as a new event:

curl -X POST "<YOUR-CONDUCTOR-WEBHOOK-URL>" \
  -H "Content-Type: application/json" \
  -H "<YOUR-HEADER-NAME>: <YOUR-HEADER-VALUE>" \
  -d '{
    "bug_id": "BUG-3045",
    "title": "Task execution bug",
    "reported_by": "Internal Bot"
  }'

This triggers a new webhook event, starts a new workflow instance, and creates a new Jira ticket.

This approach keeps your system clean, reliable, and free from duplicate ticket noise, even under high-load or retry-heavy conditions.

Troubleshooting

Here are a few common issues you might encounter while setting up the Jira ticket automation workflow and how to resolve them:

Issue	Possible causes and fixes
Webhook doesn’t trigger the workflow	The webhook URL is incorrect or malformed. The webhook is not configured to Start workflow when webhook event comes. Required headers are missing or do not match the expected configuration. The workflow name in the webhook settings does not match the actual workflow definition.
Jira ticket isn’t created	Make sure your Jira portal and project key are correctly set in the workflow definition. Verify that the secret atlassian-token contains a valid Base64-encoded token. Confirm that your user has both Administrator and Service Desk Team roles in the Jira project.
Authentication fails with Jira API	Verify the format of the Base64-encoded authentication string email:api_token. Regenerate your API token from Atlassian if it has expired. Make sure the workflow uses the correct secret name (atlassian-token). Double-check that the email address used in the Base64 string matches the account tied to the API token.
Workflow execution fails	Review the workflow execution logs in Executions > Workflow. Make sure the webhook payload includes all required input parameters: bug_id, title, and reported_by.