Rate Limits

Rate limits are a strategy for limiting request traffic, so that system resources will not be overloaded. Orkes Conductor automatically handles workflow and task rate limits without the need to write custom code. You can set and adjust them in the workflow or task definition ahead of time, which will take effect during execution.

Task rate limits

A task’s rate limit controls the number of task executions in a given period and serves as a critical strategy for managing task load and worker capacity.

When the number of tasks scheduled within a given duration exceeds the defined rate limit, the Conductor server will place the additional tasks in a PENDING status. Once an IN_PROGRESS task is completed, the rate limit is freed up and the server will make the next PENDING task available for polling.

Rate limit configuration

You can configure rate limit behavior for tasks in its task definition. The parameters for defining a task’s rate limit behavior are:

• Rate limit
• Rate limit frequency
• Concurrent executions

Parameter	Description	Required/ Optional
rateLimitPerFrequency	The maximum number of task executions that can be scheduled in a given duration. Default value is 0.	Optional.
rateLimitFrequencyInSeconds	The frequency window (in seconds) for the rate limit.	Optional.
concurrentExecLimit	The number of task executions that can be executed concurrently. Default value is 0.	Optional.

note

To configure tasks with no rate limits, set rateLimitPerFrequency and concurrentExecLimit to 0.

Example

// task definition
{
  "name": "someTaskDefName",
  "rateLimitPerFrequency": 0,
  "rateLimitFrequencyInSeconds": 1,
  "concurrentExecLimit": 0
}

Workflow rate limits

A workflow’s rate limit controls the number of concurrent executions that can be active. Beyond this limit, workflows will be queued for execution based on their start time.

When the number of scheduled workflows exceeds the defined rate limit, the Conductor server will place these workflows in a RUNNING state with the first task set to a PENDING status. Once a workflow completes, the rate limit is freed up and the server will schedule the next PENDING task for polling.

Rate limit configuration

You can configure the limit on concurrent workflow executions in its workflow definition.

Parameter	Description	Required/ Optional
rateLimitConfig	A map of the workflow rate limit configuration.	Optional.
rateLimitConfig. rateLimitKey	A unique identifier to group workflow executions for rate limits. Can be a fixed value (for example, “max”) or a dynamic variable from the workflow input (for example, ${workflow.input.correlationId}).	Optional.
rateLimitConfig. concurrentExecLimit	The number of workflow executions that can run concurrently for each rate limit key. Cannot be passed as a dynamic variable.	Optional.

Routing rate limits with dynamic key

Using a dynamic rateLimitKey, you can apply separate rate limit queues based on workflow inputs like correlationId or version. The rate limit for each group of workflows will be the same, based on the concurrentExecLimit.

For example, if workflow executions are grouped according to their correlation ID with concurrentExecLimit set to 100, workflows triggered with correlation IDs 1 and 2 will have their rate limit queues capped at 100 each.

Example

// Workflow definition
{
  ...
  "rateLimitConfig": {
    "rateLimitKey": "${workflow.input.correlationId}",
    "concurrentExecLimit": 100
  }
}

Client SDK methods

Java

Using a fixed rateLimitKey

        RateLimitConfig rateLimitConfig = new RateLimitConfig();
        rateLimitConfig.setRateLimitKey("http");
        rateLimitConfig.setConcurrentExecLimit(3);
        workflowDef.setRateLimitConfig(rateLimitConfig);

Using a dynamic rateLimitKey

In this case, a separate rate limit is applied for each correlation ID.

        RateLimitConfig rateLimitConfig = new RateLimitConfig();
        rateLimitConfig.setRateLimitKey("${workflow.correlationId}");
        rateLimitConfig.setConcurrentExecLimit(3);
        workflowDef.setRateLimitConfig(rateLimitConfig);