Skip to main content

Creating Task Definitions

Tasks are the building blocks of workflow in Conductor. A task can be an operator, system task or custom code written in any programming language.

Input Payload​

You can configure task definitions directly via UI and using API. The task definitions include the following parameters:

AttributeDescription
nameProvide a unique name to identify the task. This field is mandatory.
descriptionInclude a description that indicates the purpose of the task. This field is optional.
retryCountThe number of retries to attempt when a task fails. The default value is 3.
retryDelaySecondsIndicates the time (in seconds) to wait before each retry occurs. The default value is 60.
retryLogicIndicates the mechanism for the retries. It can take any of the following values:
  • FIXED - Every retry occurs as per the time set under retryDelaySeconds.
  • EXPONENTIAL_BACKOFF - Every retry occurs as per retryDelaySeconds*2^(retryCount)
timeOutSecondsTime (in seconds), after which the task is marked as TIMED_OUT if not completed after transitioning to IN_PROGRESS status for the first time. No timeout occurs if the value is set to 0.
pollTimeoutSecondsTime (in seconds), after which the task is marked as TIMED_OUT if not polled by a worker. No timeout occurs if the value is set to 0.
timeoutPolicyIndicates the condition at which the task should time out. It can take any of the following values:
  • RETRY - Retries the task again.
  • TIME_OUT_WF - The status of the task is marked as TIMEOUT and the task is terminated.
  • ALERT_ONLY - Registers a counter.
Note: To create a task that never times out, set timeoutSeconds and pollTimeoutSeconds to 0.
responseTimeoutSecondsSet a time (in seconds) to reschedule the task if the task status is not updated. Default value is 3600.
inputKeysAn array of keys for the task’s expected input.
outputKeysAn array of keys for the task’s expected output.
inputTemplateInput templates are defined as part of task definitions. It acts as default input to the task while adding to the workflow. It can be overridden by values provided in Workflow definitions.
Note: You cannot view the input templates in the workflow definition JSON code as they are part of only task definitions. But, on clicking the task, you can view the input templates supplied from the UI.
concurrentExecLimitIndicates the number of tasks that can be executed at any given time.

For example, if you have 1000 task executions waiting in the queue and 1000 workers polling this queue for tasks, but if you have set concurrentExecLimit to 10, only ten tasks would be given to workers (which would lead to starvation). If any of the workers finish execution, a new task(s) will be removed from the queue while still keeping the current execution count to 10.
backOffScaleFactorThe value to be multiplied with retryDelaySeconds in order to determine the interval for retry.
rateLimitFrequencyInSeconds, rateLimitPerFrequency
  • rateLimitFrequencyInSeconds sets the frequency window, which is the duration to be used in events per duration.
  • rateLimitPerFrequency defines the number of tasks that can be given to workers per given frequency window.
Note: Rate limiting is only supported for the Redis persistence module and is unavailable with other persistence layers.

For example, let’s set rateLimitFrequencyInSeconds=5, and rateLimitPerFrequency=12. This means our frequency window is 5 seconds in duration, and for each frequency window, the Conductor would only give 12 tasks to workers. So, in a given minute, the Conductor would only give 12*(60/5) = 144 tasks to workers, irrespective of the number of workers polling for the task.
Unlike concurrentExecLimit, rate limiting doesn't consider the tasks already in progress/completed. Even if all the previous tasks are executed within 1 sec or would take a few days, the new tasks are still given to workers at configured frequency, 144 tasks per minute in the above example.
ownerEmailThis field will be auto-populated with the user's email address.

API Endpoint​

POST /api/metadata/taskdefs

Client SDK Methods​

void OrkesMetadataClient.registerTaskDefs(List<TaskDef> taskDefs) throws ApiException