Skip to main content

2 posts tagged with "workflow engine"

View All Tags

· 7 min read
Riza Farheen

A workflow engine is a tool to execute a sequence of activities to perform a specific business process. The entire process is broken down into a series of activities, each with its own rules and conditions. The individual processes are executed in the defined order and sequence and combined into a workflow representing the overall process.

What is a workflow engine

Conductor - An Overview

Orkes Conductor, built on the battle-tested Netflix Conductor, is a language and cloud-agnostic general-purpose workflow orchestration engine. Conductor as a workflow engine makes it easy to orchestrate services to build distributed applications by allowing developers to focus on implementing the business logic rather than mechanics of service communication, flow durability, and failure handling.

Your business logic to be implemented is handled as Workflows in Conductor. Similar to business logic being further simplified into various activities, the workflow divides the business flow into individual blocks, referred to as “tasks” in Conductor. Each of these tasks are individual microservices that are autonomous. This means they can be individually scaled without affecting the entire application. However, these microservices need to interact with each other to fulfill the business logic. But depending on the business requirements, the number of microservices can increase by a large number, which makes it much more challenging to maintain. That’s where orchestration comes into play.

Conductor as a Workflow Engine

Let’s go back to the Monolithic era when applications were built as a single unit. Since it’s a single unit, the application components are tightly coupled so that each component is so interdependent that even a minute change in a single component results in redeploying the entire application. Think of a situation where your application has scaled so much that a complete redeployment can cost a lot. In such situations, you can leverage a tool like Conductor.

The applications can be implemented as Conductor Workflows, which contain a series of tasks to be executed. Each task is an individual stand-alone service loosely coupled with other services and can be managed separately.

A Conductor Workflow can be defined as a combination of tasks and operators. The tasks can be System tasks executed by the Conductor server or Worker tasks that require an external worker outside the Conductor environment. The Operators are the natural programming language constructs such as loop, switch case, etc.

Core Features

Let’s look at Conductor's core features that help in streamlining your business operations.

Core Features in Conductor, the Workflow Engine

Language Agnostic

Conductor is language-agnostic, which means that you can code in the language of your choice. We also provide the flexibility to handle each task in different languages, thus enabling a diverse language environment.

Hybrid & Multi-Cloud Support

Conductor being a cloud-agnostic platform, you can run on any cloud infrastructure without being coupled to a specific cloud provider. This approach gives the organization greater flexibility as you can choose and even set up a multi-cloud infrastructure.

Workflow as Code

You can create intricate & flexible workflows as code using SDKs in popular programming languages like Java, Go, Python, C#, Typescript, and Clojure.

Low Code Configuration / Visibility

The approach of low-code configuration is the ability to incorporate building blocks into workflows/applications, eliminating the code. With Conductor’s UI approach, you can quickly build workflows using blocks. You can also create a JSON file, where the visual representation would be developed based on the JSON data.

The visual representation helps to quickly iterate workflows based on the updated business requirements. You can also easily debug issues while looking into the workflow.

Seamless Scaling

The applications built using Conductor are scalable, which means that you can make changes to the workflow without affecting the functionality of the entire application. You can add/remove tasks based on the changes in your business operations, and the workflow gets instantly updated without affecting other tasks within the workflow.

Error Handling

Yet another exciting feature of Conductor is its native ability to handle errors. With the in-built support for handling retries, your application becomes more resilient. The tasks/workflows can be configured to handle timeouts, rate limits, or failures, thus helping to maintain your business operations smoothly.

Long Running Workflows

Specific workflows should run for a prolonged duration based on your requirements. The best way to handle this is to automate workflows that run at regular cadences. With Conductor, you can automate the workflows in different ways. One way is to allow the same workflow to run repeatedly by calling them using the start workflow concept. Another way is to Schedule the workflows to run at regular intervals using Workflow Scheduler.

Role Based Access Control (RBAC)

RBAC is a mechanism that allows you to provide access control to individuals based on their role in the organization. It helps limit individual access to required applications, workflows, secrets, etc.

Workflow Versioning

Version control for Conductor workflow allows you to run different versions of the same workflow depending on your organization’s requirements. Think of a situation like an automatic renewal of subscriptions, where you need to rerun the same workflow for every renewal cycle but a different version of it. In such cases, you can call the second version of the same workflow using the start workflow concept.

Stateful Serverless

One of the critical challenges in building powerful workflows across microservices is maintaining the state of each service, especially when the services need to interact with each other. This can be tackled using a workflow engine like Conductor, which provides a way to execute workflows by defining the sequence of microservices.

Benefits of Workflow Engines

You are now familiar with the core features of Conductor; let’s delve into the benefits of implementing a workflow engine in an organization.

  • Automate Routine Tasks

Every organization has repetitive, routine tasks that can consume hours of human labor. Automating these processes using a workflow engine aids in reducing possible human errors, which ultimately helps in increasing customer retention.

  • Continuous Improvement

The analytics and insights provided by the workflow engine help in analyzing the data and identifying the areas of improvement, thus reducing the redundancy in the organization.

  • Analytical Queries

The data captured by the orchestration framework can help in queries related to specific instances.

  • Better Scalability

Components can be added/removed and deployed independently based on the changing business requirements without affecting the entire workflow.

  • Faster Testing & Deployment

Compared to monolithic architecture, the application built using microservices can be tested and deployed at a faster rate.

  • Increased Productivity with Easier Troubleshooting

Most of the developer's time is spent on fixing the bugs rather than developing the code. With the workflow engine’s visual representation feature, a developer can easily identify the task with issues which helps them quickly fix the bugs rather than going through the entire code. The productivity is thus increased as the testing and scaling of the application is to be done on the particular task alone.

In a nutshell, a workflow engine helps streamline your business operations, reducing manual errors and delays, thus improving the overall productivity of your organization. A workflow engine like Conductor is now widely implemented in industries such as fintech, healthcare, banking, real estate, e-commerce, insurance, retail, logistics, media & entertainment, and much more.

If you’re looking for a powerful workflow engine for your enterprise, it’s high time to check out Orkes Cloud, the cloud-hosted, fully managed version of Netflix Conductor.

Have queries on setting up your Cloud instance? Our Conductor Experts are here to help!

Get Started with Orkes Cloud now!

Meanwhile, try Conductor through Playground, a free-to-use cloud-hosted version from Orkes.

· 5 min read

Netflix Conductor is a popular platform for building resilient stateful applications by creating workflows that span across services. You can try out the workflows from this article at Playground, a free hosted version of Conductor.

Dynamic workflows as Code in Conductor

What is a Workflow?

Before diving into how to create workflows using Conductor, let’s first define what a workflow is. In simple terms, a workflow is a series of tasks or steps executed in a specific order to accomplish a goal. Workflows are used to automate complex processes and ensure that all the necessary steps are completed in a logical sequence. Conductor workflows are composed of Tasks and Operators.

Workflow = {Tasks + Operators}

Tasks are services encapsulating the business logic that runs outside the Conductor server and are implemented as a Microservice, Lambda, or Worker. Workers run outside the Conductor server and can be implemented in any supported language. A workflow can contain multiple workers written in different languages.

Operators are primitives from programming languages that are used to control the flow of execution inside a workflow. Conductor supports operators such as switch, loop, fork/join, and sub-workflows, allowing you to define complex workflows.

Separation of Workflows from Re-usable Services

Conductor promotes clear separation between application workflow and services that are used as building blocks of the workflow as ‘Tasks’. This separation ensures that the tasks follow the single responsibility principle and are generally stateless in nature.

This model ensures two things:

  1. Re-usability of services across many workflows promoting greater collaboration across teams.
  2. Stateless Task Workers can quickly scale up/down based on the volume while maintaining the state only at the Conductor server.

Workflow definition – JSON or Code?

Why not both?
Conductor server stores the workflow definitions as JSON on the server side. However, this does not restrict users from expressing their workflows as JSON alone. Conductor supports creating workflows using Code and executing both pre-registered as well as dynamic workflows expressed using code.

Simple Example

Let’s take an example of a simple two-task workflow:

Two-Task workflow in Conductor


"name": "simple_two_task_workflow",
"version": 1,
"schemaVersion": 2,
"tasks": [
"name": "task1",
"taskReferenceName": "task1",
"inputParameters": {},
"type": "SIMPLE"
"name": "task2",
"taskReferenceName": "task2",
"inputParameters": {},
"type": "SIMPLE"

The same workflow in different languages:


ConductorWorkflow workflow = new ConductorWorkflow(workflowExecutor);

//Add tasks
workflow.add(new SimpleTask("task1", "task1"));
workflow.add(new SimpleTask("task2", "task2"));



conductorWorkflow := workflow.NewConductorWorkflow(executor.NewWorkflowExecutor(client.NewAPIClient(nil, nil)))

//Add Tasks
Add(workflow.NewSimpleTask("task1", "task1")).
Add(workflow.NewSimpleTask("task2", "task2"))



workflow = ConductorWorkflow(

#Two Tasks
task1 = SimpleTask('task1', 'task1')
task2 = SimpleTask('task2', 'task2')

#Add tasks to the workflow using >> operator
workflow = workflow >> task1 >> task2

# Execute the workflow

Creating workflows using code opens up use cases where it might be impossible to define workflows using static definitions — this could be when the number of tasks and their flow depends on the data that is dependent on the other factors.

Complex Example

Let’s take an example of a hypothetical workflow that is created dynamically based on the user data:

ConductorWorkflow workflow = new ConductorWorkflow(workflowExecutor);

//Get the list of users to send notification to
List<UserInfo> users = getUsers();
Task<?>[] tasks = new Task[users.size()];

int counter = 0;
for (UserInfo user : users) {
if(user.sendEmail) {
SimpleTask task = new SimpleTask("send_email", "send_email_" + counter);
tasks[counter++] = task;
} else {
SimpleTask task = new SimpleTask("send_sms", "send_sms_" + counter);
tasks[counter++] = task;

//Run all the tasks in parallel
workflow.add(new ForkJoin("run_in_parallel", tasks));

//Execute workflow and get the future to wait for completion.
CompletableFuture executionFuture = workflow.execute(new HashMap<>());

//Alternatively, kick off the workflow if it's going to be a long-running workflow
String workflowId = workflowClient.startWorkflow(new StartWorkflowRequest().withWorkflowDef(workflow.toWorkflowDef()));

In the above example, we load up the user list from a backend store, and for each user, create a task to send sms or email. A workflow is created by adding appropriate notification tasks for each user here and is then executed.

Depending on how long the workflow takes to complete, Conductor provides a way to wait for the workflow completion using ‘Futures’ or kick off a workflow that returns the workflow execution id that can be used by workflowClient to monitor the execution. (or can be searched and viewed in the UI)

You can write workflows using code in Java, Golang, Python, CSharp, Javascript, and even Clojure with Conductor.

Visit Conductor SDK on GitHub for the latest SDKs with fully working example apps.

Dynamic Workflows and Conductor

The ability to dynamically create workflows using code allows developers to address very complex use cases where it is impossible to pre-define workflows. With Conductor, you can still do this, with the full power of Conductor visualization that allows you to visualize the entire execution in the UI. It’s like having your cake and eating it too!


Netflix Conductor is a powerful platform that lets you create the most complex workflows while making it very easy to handle runtime scenarios with powerful debugging and visualization tools, reducing the mean time to detect and resolve issues in the production environment.

Be sure to check out Netflix Conductor on GitHub. Our Orkes team do provide Conductor Playground —  a free version of Conductor.

Join our Slack community