Skip to main content

Rate Limits

Rate limits are a strategy for limiting request traffic, so that system resources will not be overloaded. Orkes Conductor automatically handles workflow and task rate limits without the need to write custom code. You can set and adjust them in the workflow or task definition ahead of time, which will take effect during execution.

Task rate limits​

A task’s rate limit controls the number of task executions in a given period and serves as a critical strategy for managing task load and worker capacity.

When the number of tasks scheduled within a given duration exceeds the defined rate limit, the Conductor server will place the additional tasks in a PENDING status. Once an IN_PROGRESS task is completed, the rate limit is freed up and the server will make the next PENDING task available for polling.

Rate limit configuration​

You can configure rate limit behavior for tasks in its task definition. The parameters for defining a task’s rate limit behavior are:

  • Rate limit
  • Rate limit frequency
  • Concurrent executions
ParameterDescriptionRequired/ Optional
rateLimitPerFrequencyThe maximum number of task executions that can be scheduled in a given duration. Default value is 0.Optional.
rateLimitFrequencyInSecondsThe frequency window (in seconds) for the rate limit.Optional.
concurrentExecLimitThe number of task executions that can be executed concurrently. Default value is 0.

For example, if you have 1000 task executions waiting in the queue and 1000 workers polling this queue for tasks, but if you have set concurrentExecLimit to 10, only 10 tasks would be given to workers (which would lead to starvation). If any of the workers finish execution, a new task(s) will be removed from the queue while still keeping the current execution count to 10.		Optional.
note

To configure tasks with no rate limits, set rateLimitPerFrequency and concurrentExecLimit to 0.

Example

// task definition
{
 "name": "someTaskDefName",
 "rateLimitPerFrequency": 12,
 "rateLimitFrequencyInSeconds": 5,
 "concurrentExecLimit": 0
}

For example, let’s set rateLimitFrequencyInSeconds=5, and rateLimitPerFrequency=12. This means our frequency window is 5 seconds in duration, and for each frequency window, the Conductor would only give 12 tasks to workers. So, in a given minute, the Conductor would only give 12*(60/5) = 144 tasks to workers, irrespective of the number of workers polling for the task.

Unlike concurrentExecLimit, rate limiting doesn't consider the tasks already in progress or a terminal state. Even if all the previous tasks are executed within 1 sec or would take a few days, the new tasks are still given to workers at configured frequency, 144 tasks per minute in the above example.

Workflow rate limits​

A workflow’s rate limit controls the number of concurrent executions that can be active. Beyond this limit, workflows will be queued for execution based on their start time.

When the number of scheduled workflows exceeds the defined rate limit, the Conductor server will place these workflows in a RUNNING state with the first task set to a PENDING status. Once a workflow completes, the rate limit is freed up and the server will schedule the next PENDING task for polling.

Rate limit configuration​

You can configure the limit on concurrent workflow executions in its workflow definition.

ParameterDescriptionRequired/ Optional
rateLimitConfigA map of the workflow rate limit configuration.Optional.
rateLimitConfig. rateLimitKeyA unique identifier to group workflow executions for rate limits.

Can be a fixed value (for example, “max”) or a dynamic variable passed only from the workflow input (for example, ${workflow.input.correlationId}).		Optional.
rateLimitConfig. concurrentExecLimitThe number of workflow executions that can run concurrently for each rate limit key. Cannot be passed as a dynamic variable.Optional.

Routing rate limits with dynamic key​

Using a dynamic rateLimitKey, you can apply separate rate limit queues based on workflow inputs like correlationId or version. The rate limit for each group of workflows will be the same, based on the concurrentExecLimit.

For example, if workflow executions are grouped according to their correlation ID with concurrentExecLimit set to 100, workflows triggered with correlation IDs 1 and 2 will have their rate limit queues capped at 100 each.

Example

// Workflow definition
{
  ...
  "rateLimitConfig": {
    "rateLimitKey": "${workflow.input.correlationId}",
    "concurrentExecLimit": 100
  }
}

Client SDK methods​

Using a fixed rateLimitKey

        RateLimitConfig rateLimitConfig = new RateLimitConfig();
        rateLimitConfig.setRateLimitKey("http");
        rateLimitConfig.setConcurrentExecLimit(3);
        workflowDef.setRateLimitConfig(rateLimitConfig);

Using a dynamic rateLimitKey

In this case, a separate rate limit is applied for each correlation ID.

        RateLimitConfig rateLimitConfig = new RateLimitConfig();
        rateLimitConfig.setRateLimitKey("${workflow.correlationId}");
        rateLimitConfig.setConcurrentExecLimit(3);
        workflowDef.setRateLimitConfig(rateLimitConfig);