Skip to main content

Overview: Workflows

Build workflows in Conductor for a range of uses: event-driven workflows, agentic workflows, process automation, human-in-the-loop processes, microservice orchestration, or real-time API orchestration.

Ways to create workflows

In Conductor, all workflow definitions are stored as JSON. There are four ways to create a Conductor workflow:

  1. Code—Using Conductor SDKs, define the workflow in any programming language (Python, Java, JavaScript, C#, Go, Clojure) and register it to the Conductor server. Once registered, the JSON workflow definition is generated and saved. Learn more in Write Workflows as Code.
  2. JSON—Write the workflow in JSON and register it to the Conductor server using the /api/metadata/workflow endpoint.
  3. Visual workflow editor—Using Conductor UI, define your workflow visually, which is formatted as JSON under the hood.
  4. BPMN importer—Convert existing BPMN files into Conductor workflows using the built-in importer. This provides an easy migration path from BPMN-based systems.

Static vs dynamic workflows

With Conductor, you can create workflows statically (ahead of time) or dynamically (at runtime).

Dynamic workflows are created using the workflow-as-code approach, allowing you to write your workflow and task workers in your preferred language and execute them immediately. This is useful in cases where workflows cannot be defined ahead of time, as the number of tasks and flow control depend on real-time factors. Learn more in Write Workflows as Code.

AI-powered and agentic workflows

An agentic workflow is an AI-driven process where the sequence of tasks are dynamically executed with minimal human interaction to achieve a particular goal. Using Conductor’s suite of AI Orchestration and human-in-the-loop features, you can rapidly build agentic or AI-powered workflows that are auditable, scalable, and safely embedded with human oversight. Learn more in Advanced Workflows.

Event-driven workflows

Create event-driven workflows and seamlessly integrate with event-driven pipelines with Conductor’s suite of features: webhooks, event handlers, message broker integrations, and more. Learn more in Advanced Workflows.

Workflow definition

A workflow definition contains configuration parameters that define the workflow behavior, including the flow of tasks, workflow inputs and outputs, schema, failure handling settings, and rate-limiting features. The most important parameter is the tasks array, which specifies the order and execution of the defined tasks and operators.

Here are the workflow configuration parameters:

ParameterDescriptionRequired/ Optional
nameWorkflow name.
  • For v4.0.1 and prior: Only letters, digits, hyphens (-), and underscores (_) are allowed.
  • For v4.0.2 and later: Only letters, digits, spaces, hyphens (-), underscores (_), and certain special characters (<, >, {, }, #) are allowed.
Note: It is recommended to use alphanumeric characters for workflow names. While special characters are allowed for backward compatibility, they are not fully supported and may cause unexpected behavior.
Required.
description​Workflow description.Required.
versionWorkflow version.Optional.
schemaVersionThe current Conductor Schema version. Must be 2.Required.
tasksArray of task configuration objects. This defines the flow of tasks in the workflow.Required.
inputParametersArray of workflow input keys.Optional.
outputParametersJSON template used to generate the workflow output.

If unspecified, the workflow output will be the output of the last executed task.
Optional.
enforceSchemaWhether to enforce input schema validation. Set to true to enable validation.

Learn more about using schemas in Schema Validation.
Optional.
inputSchemaName and type of the input schema to be associated with the workflow.Required if enforceSchema is set to true.
outputSchemaName and type of the output schema to be associated with the workflow.Required if enforceSchema is set to true.
restartableWhether the workflow can be restarted after completion. The default value is true.

Set to false if restarting a completed workflow causes side effects.
Optional.
workflowStatusListenerEnabledWhether to enable status callback for workflow state changes. The default value is false.

Learn more in Enabling CDC.
Optional.
workflowStatusListenerSinkThe message broker sink where workflow state changes are sent.Required if workflowStatusListener is set to true.
ownerEmailThe email ID of the workflow creator.Required.
timeoutPolicyThe workflow timeout policy.

Learn more about workflow timeouts in Handling Failures.
Required.
timeoutSecondsThe timeout duration in seconds.Required.
failureWorkflowThe compensation workflow to trigger upon failure of the current workflow execution. Useful for cleanup, notifications, or post actions.

Learn more about compensation workflows in Handling Failures.
Optional.
rateLimitConfigA map of the workflow rate limit configuration.

Learn more about workflow rate limits in Handling Failures.
Optional.
rateLimitConfig. rateLimitKeyA unique identifier to group workflow executions for rate limiting.Optional.
rateLimitConfig. concurrentExecLimitThe number of workflow executions that can run concurrently for a given key.Optional.
maskedFieldsAn array of fields to be masked in the workflow execution.
Learn more about masking fields in Masking Parameters.
Optional.

Example

Here is an example workflow definition JSON:

{
"name": "exampleWorkflow",
"description": "Example workflow definition.",
"version": 1,
"tasks": [],
"inputParameters": [
"someKey",
"anotherKey"
],
"outputParameters": {
"someOutput": "${task_reference_name.output.id}",
"anotherOutput": "${workflow.version}"
},
"schemaVersion": 2,
"restartable": true,
"workflowStatusListenerEnabled": false,
"ownerEmail": "user@example.com",
"timeoutPolicy": "ALERT_ONLY",
"timeoutSeconds": 0,
"failureWorkflow": "",
"maskedFields": []
}