Skip to main content

Creating Task Definitions

Tasks are the building blocks of workflow in Conductor. A task can be an operator, system task, or custom code written in any programming language. This API is used to create task definitions in Orkes Conductor.

Input Payload​

You can configure task definitions directly via UI and using API. The task definitions includes the following parameters

AttributeDescription
nameProvide a unique identifier for the task. This field is mandatory.
descriptionA brief description of the task’s purpose. This field is optional.
retryCountThe number of retries to attempt if the task fails. The default value is 3.
retryDelaySecondsIndicates the time (in seconds) to wait before each retry occurs. The default value is 60.
retryLogicIndicates the mechanism for the retries. It can take any of the following values:
  • FIXED - Retries occur based on the retryDelaySeconds interval.
  • LINEAR_BACKOFF - Retries occur with a delay that increases linearly based on the backoff rate and attempt number.
  • EXPONENTIAL_BACKOFF - Retries occur with a delay that doubles each time, calculated as retryDelaySeconds*2^(retryCount)
timeOutSecondsTime (in seconds), after which the task is marked as TIMED_OUT if not completed after transitioning to IN_PROGRESS status for the first time. No timeout occurs if the value is set to 0.
pollTimeoutSecondsTime (in seconds), after which the task is marked as TIMED_OUT if not polled by a worker. No timeout occurs if the value is set to 0.
timeoutPolicyIndicates the condition at which the task should time out. It can take any of the following values:
  • RETRY - Retries the task again.
  • TIME_OUT_WF - The status of the task is marked as TIMEOUT and the task is terminated.
  • ALERT_ONLY - Registers a counter.
Note: To create a task that never times out, set timeoutSeconds and pollTimeoutSeconds to 0.
responseTimeoutSecondsTime (in seconds) to wait before rescheduling the task if its status is not updated. The default value is 600.
inputKeysAn array of keys representing the expected input for the task.
outputKeysAn array of keys representing the expected output from the task.
inputTemplateInput templates are defined as part of task definitions. It acts as default input to the task while adding to the workflow. It can be overridden by values provided in workflow definitions.
Note: You cannot view the input templates in the workflow definition JSON code as they are part of only task definitions. However, when clicking the task, you can view the input templates supplied from the UI.
concurrentExecLimitThe maximum number of tasks that can be executed simultaneously.

For example, if you have 1000 task executions waiting in the queue and 1000 workers polling this queue for tasks, but if you have set concurrentExecLimit to 10, only ten tasks would be given to workers (which would lead to starvation). If any of the workers finish execution, a new task(s) will be removed from the queue while still keeping the current execution count to 10.
backOffScaleFactorThe value to be multiplied with retryDelaySeconds in order to determine the interval for retry.
rateLimitFrequencyInSeconds, rateLimitPerFrequency
  • rateLimitFrequencyInSeconds sets the frequency window, which is the duration to be used in events per duration.
  • rateLimitPerFrequency defines the number of tasks that can be given to workers per given frequency window.
Note: Rate limiting is only supported for the Redis persistence module and is unavailable with other persistence layers.

For example, let’s set rateLimitFrequencyInSeconds=5, and rateLimitPerFrequency=12. This means our frequency window is 5 seconds in duration, and for each frequency window, the Conductor would only give 12 tasks to workers. So, in a given minute, the Conductor would only give 12*(60/5) = 144 tasks to workers, irrespective of the number of workers polling for the task.
Unlike concurrentExecLimit, rate limiting doesn't consider the tasks already in progress/completed. Even if all the previous tasks are executed within 1 sec or would take a few days, the new tasks are still given to workers at configured frequency, 144 tasks per minute in the above example.
ownerEmailThis field will be auto-populated with the user's email address.

API Endpoint​

POST /api/metadata/taskdefs

Client SDK Methods​

void OrkesMetadataClient.registerTaskDefs(List<TaskDef> taskDefs) throws ApiException