Skip to main content

14 posts tagged with "microservices"

View All Tags

· 8 min read

Editor’s Note: This post was originally published in Sep 2022 in Medium.

Workflow Start Request

Netflix Conductor is a well known platform for service orchestration allowing developers to build stateful applications in the cloud without having to worry about resiliency, fault tolerance and scale. Conductor allows building stateful applications — automation as well as human actor driven flows by composing services that are mostly stateless.

At Orkes, we regularly scale Conductor deployments to handle millions of workflows per day and we often get questions from the community on how to scale the Conductor and if the system can be scaled to handle billions of workflows.


We ran a benchmark to test the limits of Conductor’s scalability using a simple single node setup on a laptop. The results were not so surprising, given our experience with Conductor, — with the right configuration, Conductor can be scaled to handle the demands of really large workloads.







We measured the peak stable throughput Conductor can achieve on the following parameters:

  • Number of workflow start request per second
  • Number of task updates per second — translates to no. of state transitions happening per second
  • Number of workflows completing per second with moderate complexity
  • Amount of data processed per second


We measured p50, p95 and p99 latencies for the critical APIs in Conductor:

  • Workflow Start
  • Task Update

Benchmarking Tools

We used wrk2, a fantastic tool to generate stable load on the server. Wrk2 improves on wrk and adds the ability to generate sustained load at a specific rate (-R parameter).

We created a load testing workflow which is complex enough with a total of 13 steps.

For the experiment a set of load testing workers were created that were used to poll for the tasks in the workflow, and produce a dummy output.


Conductor has a pluggable metrics system and we used Prometheus to capture and Grafana to visualize various metrics.

The Setup

The experiment was designed to send a sustained workload of 200+ workflow executions / sec. The test workflow also embedded a sub-workflow with a single step inside, so during the experiment, we were starting approximately 400+ workflow/sec.

For the Conductor version, we used Orkes’ build of Conductor that is tuned and customized to handle large workflows with minimal latency possible and optimize the operational costs.

We have a version of this available under open source at

Conductor servers are a horizontally scalable system, for this test we ran a single node server — of course your production setup should have at-least 3 nodes in multiple availability zones / racks for higher availability.


The Conductor server was running on a Macbook Pro with M1 Max CPU and 64GB of RAM. The same machine also ran a single node redis server. Postgres database was running on another Macbook Pro with M1 Max CPU and 32GB of RAM. The same machine also ran task workers. A Core i9 Macbook pro with 32 GB RAM ran Prometheus, Grafana, a load generator docker container and task workers.

The systems communicated over WiFi 6 network — no wired connectivity, which potentially could have improved latencies a bit. The machines were roughly equivalent to a c7g.2xlarge instance type of AWS (memory consumption in the tests was below 16GB offered by this instance type).

Peak Workflow Start Requests

Testing the limits of how many workflows can be started in a burst. For this experiment we wanted to remove any network latencies so we ran the wrk on the same host as the Conductor server running thereby removing any network latencies out of equation. This gives us a theoretical max request/sec assuming the network is limitless (which it isn’t’).

Workflow Start Requests / sec. Under the normal load the no. of start requests averages at about 1.8K/sec

Latencies under high load with ~2K request/sec

Workflow Completion Metrics

For this experiment, we asked a question:

If all the tasks in a workflow were instantaneous, how many workflows can we start and complete in a second with a sustained load such that there is minimal backlog of the tasks.

To test this, we used wrk2 to send a sustained load of 210 workflow execution requests, where each workflow contains a sub workflow and worker tasks.

Workflow Execution Graph

Workflows getting completed per second

Task Level Metrics

Conductor publishes the depth of the pending task queue, this is useful when deciding when to scale up/down your worker clusters. Here is the snapshot of the worker queue depth. High numbers for a given task indicates the worker resource starvation and need to scale the worker cluster to fulfill the demand.

Pending queue size of tasks at a given point in time. Sustained high numbers indicate worker starvation and a need to scale out workers.

We achieved a consistent throughput of 1450+ task executions / sec. This included polling for the task, executing the business logic (in our case producing randomized test output) and updating the task status on the server by workers. Each successful task completion initiates the state transition of the workflow.

Number of worker tasks getting updated per second.

Critical to the throughput is the update of the task execution back to the Conductor server. This is the most critical API operation we found that is directly responsible for the throughput of the server. We optimized the server to limit the tail latencies ensuring p99 stays well under the check.

Task update from worker latencies in milliseconds.

Task Poll

Conductor uses a long-poll for the task polling, and the request waits until there is a task available for the worker or the timeout, with the timeout set to 100 ms in this experiment. Polling implements batching for more efficient use of the network and connection. In the test, the batch size was set to 10 for the tasks by workers.

Latencies for task poll

Data Processed by Workflow

We generated fake data for the test to simulate the real world scenarios in terms of data transfer. The amount of data being processed by workflows impacts the requirements for provisioned IOPS (on cloud environments) and Network throughput requirements.

The experiment averaged a sustained rate of ~80 MB/sec data processed.

Amount of data (task inputs and outputs) being processed at a given point in time.

Scaling to handle billions of workflows / month

The experiment used a total of 3 commodity hardware (roughly equivalent to c7g.2xlarge instance types on AWS). Throughout the experiment JVM heap size consistently remained below the 2GB mark.

The experiment created a workload of 210 moderately complex workflows per second, which if run constantly for a month will generate about 540M workflows.

Conductor Servers

Conductor servers themselves are stateless and can be scaled out to handle larger workload demands. Each of the server nodes can handle the workload based on the available CPU and Network and can be scaled out to handle larger workloads.


Redis serves as the backbone for state management and queues used by servers to communicate. Scaling higher workload requires either scaling up redis or using redis cluster to better distribute workload.


Postgres is used for indexing of workflow data. Beyond the disk storage requirements, scaling postgres requires two factors 1) adequate CPU and 2) IOPS required (especially on cloud environment) to handle the writes under heavy workloads. Writes to postgres are asynchronous done using durable queues (check out orkes-queues) but longer delays means completed workflows remain in Redis for longer periods of time, requiring larger Redis memory.


We often get asked, can Conductor be scaled to handle billions of workflows per month? The answer is a resounding YES it can. If you would like to give it a try, check out our developer playground at

Orkes, founded by the founding engineers of Netflix Conductor, is a fully managed service offering Conductor as a hosted service in the cloud and on-prem. Checkout our community edition for a fully open source version of Orkes stack.

If you are an enterprise and have a use case that you would like to run a PoC, please reach out to us.

Don’t forget to give us a ⭐️

· 8 min read
Mohammed Osman

Businesses must be able to provide high-quality, innovative services to clients quickly in order to meet market demand. That can be difficult if an organization’s internal architecture doesn’t offer the needed agility and speed. The tightly coupled nature of monolithic architecture can block an IT team’s ability to make changes, separate team responsibilities, and perform frequent deployments. Microservices can provide a better alternative.

In microservices architecture, an application is built as a collection of separate, independently deployable services that are loosely coupled and more easily maintained.

In this article, you’ll learn about the benefits of switching to microservices and what factors to consider as you migrate your monolithic application toward microservices architecture.

Why Use Microservices Architecture?

Structuring your application as microservices offers you a range of benefits. AWS cites several of them, below.

· 10 min read
James Walker

Architecture diagram comparing monoliths and microservices

Monoliths are systems developed as one homogeneous unit. The architecture revolves around a single focal point that contains the system’s entire functionality. Distinct logical areas such as the client-side UI and backend APIs are all developed within one unit.

Breaking up monoliths is one of the most common objectives of modern software refactoring. Untamed monoliths can become bloated beasts full of interlinked functionality that’s difficult to reason about and maintain. Breaking the architecture up to reflect the system's logical areas makes for a more manageable codebase and can accelerate the implementation of new features.

This article looks at what monoliths are, how they differ from modern microservice-based approaches, and how you can start to break up a monolithic system. As we'd be remiss to claim microservices are a perfect solution, we'll also assess the situations where this migration might not make sense.

What's a Monolith?

"Monolith" has become a widely used term in the software industry, but it can mean slightly different things depending on who you ask. You’re probably dealing with a monolith if the system has multiple distinct units of functionality, but the project's codebase structure doesn’t mirror these. This results in little or no modularity within the system.

Monolith-based development strategies involve everyone working in the same repository irrespective of the type of feature they're building. User interface components sit side-by-side with business logic, API gateway integrations, and database routines. There’s little separation of concerns; components may directly interface with each other, resulting in fragility that makes it hard to make safe changes.

Here are some more common problems associated with monoliths:

  • Tight coupling: When all your components sit alongside each other, it can be difficult to enforce rigid separation of concerns. Over time, pieces become tightly coupled to each other, preventing you from replacing components and making the accurate mapping of control flows more difficult.

  • Fragility: The tight coupling observed in monolith systems leads to innate fragility. Making a change could have unforeseen consequences across the application, creating a risk of new problems each time you deploy a feature.

  • Cognitive burden: Putting all your components into one functional unit makes it harder for developers to find the pieces they need and understand how these relate to each other. You need to keep the entire system’s operation in your mind, creating a cognitive burden that only grows over time. Eventually, the monolith becomes too complex to understand; at this point, more errors can start creeping in.

  • Longer build and deployment times: Having everything in one codebase often leads to longer CI pipeline durations. Tools such as source vulnerability scanners, linters, and stylizers will take much longer to run when they have to look at all the code in your system each time they're used. Longer builds mean reduced throughput, limiting the amount of code you can ship each day. Developers can end up sitting idly while the automation runs to completion.

If you're experiencing any of the above, it might be time to start breaking up your monolith.

How Did We Get Here? Or, Why Monoliths Prevail

Monoliths aren't without their benefits. Here are a few good reasons to use a monolith that help to explain why the strategy remains so pervasive:

  • Reduced overhead: Not having to juggle multiple projects and manage the lifecycles of individual components does have advantages. You can focus more on functionality and get to work on each new feature straight away, without needing to set up a new service component. Please note that the simplicity of the monolith strategy is being considered here, not its impact on understanding the system you're encapsulating. As we've already seen, monoliths can make it harder to reason about characteristics of your system because everything is coupled together.

  • Easier to debug: When problems occur in a monolith, you know they can only derive from one source. As your whole system is a single unit, log aggregation is simple, and you can quickly jump between different areas to inspect complex problem chains. Determining the root cause of issues can be trickier when using microservices because faults may ultimately lie outside the service that sent the error report.

  • Straightforward deployment: Monoliths are easy to deploy because everything you need exists within one codebase. In many cases, web-based applications can be uploaded straight to a web server or packaged into an all-in-one container for the cloud. This is a double-edged sword: as shown above, your deployments will be rigid units with no opportunity for granular per-component scaling.

Though monoliths aren't all bad, it's important to recognize where things fall apart. Trouble often stems from teams not realizing they’re dealing with a monolith. This speaks to a disorganized development approach fixated on code, new features, and forward motion at the expense of longevity and developer experience.

Despite these pitfalls, monoliths can still be effectively used by intentionally adopting a similar structure. The Monorepo approach, for example, uses one source control repository to encapsulate multiple logical components. You still break your system into loosely coupled units, but they can sit alongside each other in a single overarching project. This approach forces you to be deliberate in your design while offering some of the benefits of both monoliths and microservices. Many large organizations opt for a monolith-like approach, including Google and Microsoft.

Why Should You Break up a Monolith?

Monoliths often develop organically over many years. Your codebase's silently growing scale may go unnoticed or be disregarded as a necessary by-product of the system's growth. The challenges associated with monoliths tend to become apparent when you need to scale your application or integrate a new technology.

A system treated as one all-encompassing unit makes it difficult to scale individual parts to meet fluctuations in demand. If your database layer starts to perform poorly, you'll need to "scale" by starting new instances of the entire application. Replacing specific components is similarly complex; they may be referenced in hundreds of places throughout the codebase, with no defined public interface.

Separating the pieces allows you to develop each one independently. This shields individual components from bugs in the broader system, helps developers focus on their specific areas, and unlocks the ability to scale your deployments flexibly. Now you can run three instances of your login gateway, two instances of your web UI, and a single replica of your little-used social media synchronization tool. This makes your system more efficient, lowering infrastructure costs.

Breaking up a monolith also gives you greater opportunities to integrate additional technologies into your stack. New integrations can be developed as standalone modules plugged in to your system. Other components can access the modules by making network calls over well-defined APIs, specifying what functionality is needed and how it will be used.

Monolith destruction often enhances the developer experience too, particularly in the case of new hires getting to grips with your codebase for the first time. Interacting with an unfamiliar monolith is usually a daunting experience that requires bridging different disciplines. Seemingly straightforward day-one tasks like adding a new UI component might need knowledge of your backend framework and system architecture, just to be able to pull data out of the tightly coupled persistence layer.

An Alternative Approach: Microservice Architectures

Microservice architectures are the effective antithesis to the monolithic view of a system as a single unit. The microservice strategy describes an approach to software development where your distinct functional units are spun out to become their own self-contained services. The capabilities of individual services are kept as small as possible, adhering to the single-responsibility principle and creating the "micro" effect.

Services communicate with each other through clearly defined interfaces. These usually take the form of HTTP APIs; services will make network calls to each other when they need to exchange data or trigger an external action. This decoupled approach is straightforward to extend, replace, and maintain. New implementations of a service have no requirements imposed on them other than the need to offer the same API surface. The replacement service can be integrated into your system by reconfiguring the application to call it instead of the deprecated version.

Microservices let you reason about logical parts of your stack in isolation. If you're working on a backend login system, you can concentrate on the parts that belong to it, without the distractions of your UI code. Changes are much less likely to break disparate parts of the system as each component can only be accessed by the API it provides. As long as that API remains consistent, you can be reasonably confident the broader application will stay compatible.

This architecture also solves the scalability challenges of monoliths. Splitting your application into self-contained pieces lets you treat each one as its own distinct deployment. You can allocate free resources to the parts of the system that most need them, reducing waste and enhancing overall performance.

Microservices do have some drawbacks, especially for people accustomed to a monolith approach. The initial setup of a distributed system tends to be more complex: you need to start each individual component, then configure the inter-component connections so services can reach each other. These steps require an understanding of your deployment platform's networking and service discovery capabilities.

Microservices can also be hard to reason about at the whole-system level. Fault sources are not always immediately clear. New classes of error emerge when the links between services are broken by flaky networking or misconfiguration. Setting up resilient monitoring and logging for each of your services is vital for tracing issues through the layers of your application. Microservice monitoring and log aggregation are distinct skills which have helped shape the modern operations engineer role, which is focused on the day-to-day deployment and maintenance of complex distributed systems.

Using an orchestration tool like Netflix Conductor - and using Orkes as a cloud based version of Conductor simplify many of these issues.


Monolithic systems contain all their functionality within a single unit, which initially seems like an approachable and efficient way to add functionality and evolve a system over time.

In practice, monoliths are often unsuitable for today’s applications. Breaking up monoliths is an important task for software teams to guarantee stable, ongoing development at a steady pace. Separating a system into its logical constituent parts forces you to acknowledge, understand, and document the connections and dependencies within its architecture.

This article explored the problems with monoliths and looked at how microservice approaches help to address them. You also learned how monolith-like systems can still be effective when microservices aren't suitable. Deciding whether you should break up a monolith comes down to one key question: Is your architecture holding you back, making it harder to implement the changes you need? If the answer is yes, your system's probably outgrown its foundations and would benefit from being split into self-contained functional units.

When you are looking at breaking up your monolith into microservices, look at Conductor as a tool to orchestrate your microservices. Try it for free in the Orkes Playground!

· 9 min read

In large applications consisting of loosely coupled microservices, it makes sense to design the internal architecture of each microservice to suit its function rather than adhere to a single top-down architectural approach.

By design, each microservice is an independent entity that has its own data as well as business logic. So it’s intuitive to use a design approach and architecture that’s best suited to its requirements, irrespective of high-level microservices architecture. However, detractors would like you to believe that using multiple languages should be avoided as it adds unnecessary complexity and overheads to microservices operations.

But there are multiple use cases where multilanguage architecture makes sense, and technology can be used to efficiently manage the overheads introduced. In this article we will unpack:

  • When to build multilanguage microservices.
  • The challenges introduced in microservices communication due to the use of multiple languages.
  • Some tools and techniques to make multilanguage microservices implementation easier.

· 9 min read
Yulia Gavrilova

Microservice architecture is becoming more and more common in the realization of business ideas. Developers can easily add or remove features; update specific parts of applications without interfering with general workflow; and concurrently use diverse technologies and programming languages based on their business needs.

The microservice orchestrator controls the execution of processes in this distributed architecture and its role is increasingly important, as it makes it easier to uncover and fix problems, as well as manage the development of the entire system, even when we’re talking about systems that have more than one programming language in their tech stack.

In this article, you’ll learn about workflow orchestration and its benefits and limitations when building a microservice platform that features several programming languages.

· 5 min read
Doug Sillars

What is a workflow? Wikipedia says "A workflow consists of an orchestrated and repeatable pattern of activity, enabled by the systematic organization of resources into processes that transform materials, provide services, or process information."

None of that is incorrect. But it is certainly a mouthful. If I am asked in an elevator what a workflow orchestration is, I like to use analogies to make the point as approachable as possible:

"It's like a recipe for code. A recipe has a series of steps, that must be run in a specific order. Iyt can also have different options for food allergies, or different ingredients you might have on hand? A workflow is a recipe for your code, and can be built to handle many of the changes that might reasonably occur when the code runs."

Workflows as recipes

You may have seen one of the name videos out there of parents teaching kids code by writing out the steps to create a Peanut butter and Jelly sandwich.

We're not going to be quite as silly as that Dad, but we'll run through some instructions on how to create a PB&J from Instructables..

If you look at the URL of that recipe - it appears it took 4 tries to get it right :D

The steps are (skipping some substeps for clarity):

  1. Gather Your Ingredients for the Sandwich
  2. Pull Out Two Slices of Bread
  3. Open Peanut Butter and Jelly
  4. Spread the Peanut Butter Onto One Slice of Bread
  5. Spread the Jelly Onto the Other Slice of Bread
  6. Combine the Two Slices
  7. Clean Up Your Workspace
  8. Enjoy Your Sandwich

(if you read closely, I skipped 3 steps - wearing gloves, removing the crust and cutting the sandwich in half... because, well - that's all just ridiculous)

The eight steps above are a workflow. They must be performed in that order to create a sandwich - you cannot spread the PB before you lay out the bread.

Building a Conductor Workflow

We can turn these 8 steps into a Conductor workflow.

PB&J example workflow

If you'd like to see the definition of this workflow, you can check out the code on the Orkes Playground. It's free to sign up.

This workflow shows clear steps with arrows pointing to the next step - so it is visually possible to see how the workflow will progress.

Improving the workflow

Each task is run by a microservice, so making changes and adding tasks is easy to do. In this example - you see that the recipe calls for PB to be spread first, and then the jelly. This works for a single human - these steps each take 2 hands. But there is no reason that the jelly cannot go first, and THEN the PB. Or - if you had more hands - the PB and the Jelly could be spread simultaneously.

Independent tasks

Let's remove the PB dependency from the jelly spreading - as the order of PB vs. jelly does not matter.

We can do this in Conductor with a FORK. A fork splits your workflow into 2 asynchronous tasks, and then a JOIN reconnects the the workflow into a single path.

We can now apply a fork to apply PB, and a second fork to apply the jelly:

PB&J example workflow

This is version 2 of the workflow, and you can see it in the playground

Now, these operations are independent, and if there is space in the jelly task queue - that can be completed ahead of the PB.

Recipe variations

Often, recipes have variations to preparation, and they can be read like an IF statement in programming (If (fresh tomatoes) {do x}, else if (tinned tomatoes) {do y}.

Cooking a burrito

Let's look at a common example - from a burrito in my freezer. The preparation directions vary depending on the cooking method.

burrito instructions

We can emulate this in a workflow using a switch task. The Switch task takes in the workflow ovenType input ${workflow.input.ovenType} and based on this value will make a decision. The default case is for the microwave, and then second decisionCase is set to "oven". From that input, the different tasks can be run.

burrito workflow


Workflows are a series of tasks that must be followed in a certain order. In this post, we used cooking recipes as an analogy to a workflow - they too are a series of tasks that must be followed in a certain order.

We created sample workflows for making a peanut and jelly sandwich (version 1 and version2) and another workflow to cook a frozen burrito with microwave or oven instructions.

<<<<<<< Local Changes We are able to reuse a number of tasks:

  • The zap task is used twice in the microwave branch.
  • The bake task in the oven branch.
  • The flip task is used in both branches.

When reusing tasks, the taskReferenceName (shown at the top of the box) must be unique.======= If you're curious about how to build a workflow orchestration - it might be a fun exercise to try your favorite recipe as a workflow. You can build on the workflows from this post in our free playground. Feel free to share what you came up with in our Discord. We love seeing creative uses of workflows!>>>>>>> External Changes

· 9 min read
Paul Ibeabuchi

Microservice architecture is an architecture where an application is split into separate services, and each service is run and managed independently. In a microservice architecture, every service is focused on handling one major function and is solely responsible for its own data management.

Microservice architecture is often recommended for larger applications because it allows services to be managed by dedicated teams. Testing and deployment also become easier, as they can be carried out independently for each service without affecting the overall application.

In this article, you'll learn what a microservice architecture is and when to use it, and about workflow orchestration, its benefits, and how it can be utilized in a microservice architecture. You'll also look at an example use case of microservice architecture so you can better understand the benefits, strengths, and weaknesses of this style of architecture.

· 14 min read
Nikhila Jain

A 2020 survey by the research and advisory firm Gartner has highlighted the rapid pace of innovation in cloud computing. According to the research, forty percent of enterprise solutions will host their applications on cloud infrastructure by 2023. This shifting trend will cause an increased demand for cloud services, as well as for hybrid cloud architecture.

The hybrid cloud is gaining popularity as enterprise IT leaders seek flexible, scalable options that increase cost efficiency while maintaining control over enterprise data and information. Many organizations combine on-premise infrastructure with private/public cloud resources to meet these needs.

But without the right strategy, hybrid clouds can pose a number of challenges. Through a hypothetical case study, this article will help you learn about the strengths and limitations of hybrid cloud architecture.

· 3 min read
Doug Sillars

Conductor is a workflow orchestration engine that connects all of your microservices together to create fully functional workflows that can run at scale. Each workflow is comprised of tasks - and many of these tasks are powered by external workers - or microservices. These workers can be written in any language - from Conductor's point of view - data goes in, and results come out - the language that processes the data is irrelevant.

Each worker has to connect to your Conductor instance, and regularly poll for work in the queue. There has long been Java, Go and Python SDKs to easily connect your apps to Conductor, but for building in other languages, this code had to be created by each development team.

Today, we announce that major improvements to the Golang and Python SDKs, and announce C# and Clojure SDKs.

Further, all of the (non-Java) SDKs have a new GitHub home: the Conductor SDK repository is your new source for Conductor SDKs:

Coming soon:

· 9 min read
Azeez Lukman

Microservices are a common and popular approach to building modular, scalable software with autonomous services. Large complex products are broken down into individual services responsible for a specific business function, such as user authentication or store checkout.

A microservice-based application might require several services to interact with each other to complete a business scope. The coordination of these interactions is known as a workflow or a saga. There are two models for implementing a workflow: choreography and orchestration. With choreography, you let each part of the system inform the other of its job and let it work out the details, while with orchestration, you rely on a central brain to guide and drive the execution processes.

As orchestrated systems have grown more expansive, the problem of efficiently orchestrating related business logics has become more pronounced. In this article, you will learn about the microservice orchestration workflow and its importance in relation to modern software architecture practices.

What is Microservice Orchestration?

A microservice orchestration pattern involves a central orchestration service (the orchestrator) that typically contains the entire business workflow logic and issues commands to and awaits responses from worker microservices. Think of this as an orchestra where a central conductor is responsible for keeping the orchestra in sync and coordinating the members to produce a cohesive musical piece. Using orchestrators for your application is essential for efficiently managing applications based on microservices.

Before going into the specifics of microservice orchestration, it is helpful to familiarize yourself with the components of microservice-based architecture. For example, in a microservice-based e-commerce application, the following could come into play during the process of purchasing a product:

  • a service for listing all products;

  • a service for adding products to the cart and reserving that product from the inventory;

  • a service for handling the payment; and

  • a service that manages the shipment of the item.

Each of these microservices is autonomous. In other words, microservices can be individually scaled up or down without having to worry about the entire application. However, they are all required to interact with each other to fulfill the purchase. It might be tempting to have the services talk to each other directly as needed. However, as your architecture and the number of services grow, this can quickly get messy and difficult to maintain. This is where orchestration comes into play.

A microservice orchestration workflow is an architectural method of coordinating microservices for software systems and applications, in which loosely coupled services receive commands from a central controller, referred to as the orchestrator. The orchestrator acts as a brain, driving the execution processes; it sends a call to each service and awaits a reply before proceeding. The concept of a microservice orchestration workflow can be best described through a hypothetical use case.

microservice orchestration workflow architecture diagram

The architectural diagram of this hypothetical use case shows the interactions between the various services involved in the process when following an orchestration workflow. Looking at the diagram above:

  1. The orchestrator receives a trigger that initializes the workflow, starting with “Products Service.”

  2. When this service has created an order with the products in the customer's cart, it returns some response to the orchestrator.

  3. The orchestrator then calls the “Inventory Service” to reserve the products in the cart.

  4. Next, the orchestrator calls the “Payment Service” to handle the payment.

  5. After successful payment, the orchestrator moves on to the “Shipping Service,” which clears the products for shipment.

In a choreography workflow, on the other hand, the microservices are not managed by a central service; however, they are all aware of the business goals and rely on certain events from other services that determine how they function. Each service publishes the actions it has taken to a message stream such as SQS or Kafka. Other services subscribe and listen for events they are interested in from these streams and take the appropriate actions.

choreography orchestration workflow architecture diagram

In the choreography architecture above:

  1. The “Products Service” creates an order with the items in the customer's cart and publishes an "Order Created" event to a stream on the messaging platform.

  2. The “Inventory Service” and “Payment Service” consume from this message stream. The “Inventory Service” handles reserving the products in the cart, and the “Payment Service” handles the payment and publishes the “Payment Success” event.

  3. On receipt of an inventory “Payment Success” event, the “Shipping Service” goes ahead and clears the products that were reserved for shipment to the customer.

Why is Microservice Orchestration Important?

Microservice architecture involves decomposing your application into a set of services to improve agility and allow teams to scale. One of the main purposes of this architectural pattern is to have each service as an independently deployable component with well-defined interfaces; in this way, the scope of implemented changes can be limited to a single service.

However, you must coordinate the execution of multiple microservices to deliver the outcomes that users want, and this is why microservice orchestration is important. Orchestration allows you to put a service in charge of the other services. The service in charge is aware of the entire flow that is required and is responsible for putting the other services to work to achieve those aims.

Microservice orchestration enables you to process flows ranging from simple linear workflows to very complex dynamic workflows that run for multiple days with minimal effort and high visibility into the processes. To properly illustrate the benefits you can obtain with an orchestration workflow when managing your microservices, let’s take a look at a case study from Netflix.

Netflix is an enterprise company that has shifted toward orchestration workflows. The streaming service had traditionally used the choreography method, which involves peer-to-peer tasks that are tightly coupled; this became harder to scale with growing business needs and associated increasing complexities like determining what remains for a movie setup to be complete and updating their SLAs.

Later, Netflix switched to an orchestration workflow and eventually built their own container orchestration engine—Conductor—which has helped orchestrate over 2.6 million process flows, from simple linear workflows to complex dynamic workflows over multiple days.

Why Are So Many Developers Adopting This Architectural Paradigm?

As mentioned, there are two major techniques you can use if you need to execute many services to get your desired result. Orchestration, in which a central orchestrator component serves as the coordinator and is in charge of activating each service, and choreography, in which the services perform independently and are only loosely connected.

Developers are increasingly adopting orchestration because it has significant benefits that can make development and management easier for individual microservices without compromising the big picture. However, it should be noted that microservice orchestration is not without its limitations.

Benefits and Limitations of Microservice Orchestration

There are several benefits and challenges associated with the implementation of an orchestration workflow, many of which are related to how microservices interact with one another to achieve a business outcome.

Benefits of Microservice Orchestration

Central observability of process definition, status, and metrics: The orchestration framework can capture detailed information about each executed process instance, which it can make available for analytics. This allows you to answer questions about specific instances (such as, “Where is my order?”), as well as analytical queries (such as, how many products were ordered).

Synchronous processes: These provide a good way to control the process flow. For example, when a product’s service needs to successfully complete before the inventory service is processed.

Scalable orchestration on cloud-native platforms: When you scale up these services, you scale with errors in mind. Microservice orchestration provides you with insights into your processes, helping you coordinate various transactions that involve a large number of independent services.

Single fail point: Orchestration workflow allows you to easily trace out any error that occurs during the process flow, figure out why it failed, and debug. Writing tests for your microservices is important to help prevent errors from making it to the live service.

Limitations of Microservice Orchestration

When orchestrating microservices in an enterprise environment, you’ll find that some business functions can require hundreds or even thousands of microservices. Since the orchestration workflow is synchronous, it's possible that such processes will take a long time to finish.

Furthermore, as the orchestrator needs to communicate with each service and get a response before moving to the next, this makes services highly dependent upon each other. Failure at any point could cause the entire process to fail. While for some business processes this is required behavior, others might require the process to complete regardless; for instance, running analytics on an order that’s being processed shouldn’t prevent the checkout flow from being completed.

Who Can Benefit from Microservice Orchestration?

With microservices gradually becoming the default pattern for managing business logics, a strong architecture is needed for their coordination. Adopting an orchestration workflow could improve the seamless interaction between these services.

Many businesses still implement service-oriented architectures (SOAs) orchestrated by an enterprise service bus (ESB). However, as business needs grow, adding more business logics and microservices to the system can be challenging; the entire flow is not immediately visible, making it harder to alter a service without the risk of disrupting another.

Microservice orchestration offers a solution here, as it helps you visualize the end-to-end processes across your microservices, so you know what services would be affected by your updates, allowing you to easily address your increasing business needs.

More concretely, an orchestration workflow might be ideal for you if one or more of the following are critical for your business:

  • The ability to track and manage workflows from a single point.

  • A user interface to visualize process flows.

  • The ability to synchronously process all tasks.

  • The ability to efficiently scale to a high number of concurrently running process flows.

  • A queuing service abstracted from clients.

  • The requirement to operate services over HTTP or other transport layers such as gRPC.


In this article, you learned about the importance of keeping your microservices autonomous and flexible. You also learned about the use of microservice orchestration to effectively communicate, visualize, identify, and resolve the challenges of managing microservices.

The downside of the process of building an orchestration system that implements all of the features your business requires is that it’s rather complex and time consuming. A purpose-built framework offering scalable and low-overhead orchestration— like Netflix’s Conductor, is an open source tool that fits this purpose.

Orkes is a platform that offers a fully managed, cloud-hosted version of Conductor with tiered support. Orkes builds on top of Netflix Conductor to abstract out installation, tuning, patching, and managing high-performing Conductor clusters. Learn more about Orkes and get started for free within minutes here.