Dynamic Task Inputs
When you start a Conductor workflow, you can pass inputs to the workflow - which can be used as parameters for various steps in the workflow. As the tasks progress in a workflow - each task will create its own outputs. So how can this data from previous tasks be used in the steps succeeding it? In Conductor, data can be passed to a task in different ways, such as from workflow inputs/outputs, prior task inputs/outputs, variables, and secrets.
Let’s take a look at how to define this.
Expression
The task inputs are evaluated using the following expression:
Let’s break down this expression:
| Attribute | Description | ||||
|---|---|---|---|---|---|
| variableName | A string that represents the variable name you want to assign this expression output. | ||||
| ${...} | A notation that indicates the variable that would be dynamically replaced at run time. | ||||
| type | One of the following options:
| ||||
| subtype | One of the following options:
| ||||
| jsonpath | JSONPath expression (look for examples below). If we are referring to variables/secrets, this will be the variable/secret name. |
Sample Expressions
- Workflow Inputs
- Task Outputs
- Secrets
- Workflow Variables
If the task inputs are taken from workflow inputs,
"key": "${workflow.input.somekey}"
This expression defines a property called key, with a value derived from the input parameter somekey provided to the current workflow.
"key": "${task_reference_name.output.somekey}"
In summary, this expression defines a property called key, with a value derived from the output parameter somekey from a task with reference task_reference_name. This value cannot be run under the following situations:
- Task with the reference specified doesn't exist
- Task has not executed when this reference was looked up
- Task didn't produce this output value
"key": "${workflow.secrets.your_secret_name_stored_in_conductor}"
In summary, this expression defines a property called key, with a value that is replaced dynamically with the secret value for name your_secret_name_stored_in_conductor. This value cannot be run under the following situations
- Secret does not exist
During execution, if the workflow doesn't have permission to read the secret, it will fail with an error saying so.
"key": "${workflow.variables.variable_name}"
In summary, this expression defines a property called key, with a value that is replaced dynamically with the variable name variable_name. This value cannot be run under the following situations:
- Variable does not exist
Depending on the context from where the data is being passed, the expression can take the following variants:
| Workflow Attribute | Description |
|---|---|
| input | ${workflow.input.somevalue} If the workflow has an input parameter “somevalue”, then the same parameter can be referred to using this expression. |
| output | ${workflow.output.somevalue} If the workflow has an output parameter “somevalue”, then the same parameter can be referred to using this expression. |
| status | ${workflow.status} This expression can be used when you want to check the workflow status and proceed. The possible values the expression can return are RUNNING, PAUSED, TIMED_OUT, TERMINATED, FAILED, or COMPLETED. |
| workflowId | ${workflow.workflowId} This expression can be used when you need to retrieve the workflowId of the current workflow. |
| parentWorkflowId | ${workflow.parentWorkflowId} This expression can be used in the sub-workflows to retrieve the workflowId of the parent workflow. |
| parentWorkflowTaskId | ${workflow.parentWorkflowTaskId} This expression can be used in the sub-workflows to retrieve the subworkflow’s task execution Id in the parent workflow. |
| workflowType | ${workflow.workflowType} This expression can be used to retrieve the workflow name. |
| version | ${workflow.version} This expression can be used when you need to retrieve the version of the current workflow. |
| correlationId | ${workflow.correlationId} This expression can be used when you need to retrieve the correlationId provided to the current workflow instance. |
| schemaVersion | ${workflow.schemaVersion} This expression can be used to retrieve the schema version of the current workflow. |
| variables | ${workflow.variables.variable_name} This expression is used when a variable name is already stored in the workflow definition, and you need to retrieve the variable name in the preceding tasks. |
| createTime | ${workflow.createTime} This expression can be used to retrieve the workflow execution time. |
| secrets | ${workflow.secrets.secret_name} This expression can be used to retrieve the secret values without exposing them directly in the workflow definitions. Referring using this expression ensures that it takes the value dynamically while executing the workflow. |
| taskToDomain | ${workflow.taskToDomain.domain_name} This expression can be used to retrieve the domain name used while invoking the workflow. |
Detailed Examples
Task Inputs referred from Workflow Inputs
{
"workflowInputNumberExample": 1,
"workflowInputTextExample": "SAMPLE",
"workflowInputJsonExample": {
"nestedKey": "nestedValue"
}
}
We can refer to these values as inputs to the task using the following expression:
{
"taskInput1Key": "${workflow.input.worfklowInputNumberExample}",
"taskInput2Key": "${workflow.input.workflowInputJsonExample.nestedKey}"
}
On evaluating the first expression,
"taskInput1Key": "${workflow.input.workflowInputNumberExample}"
"${workflow.input.workflowInputNumberExample}"- Refers to the workflow input parameter workflowInputNumberExample.
In the workflow input, the value of workflowInputNumberExample is 1, so the value of taskInput1Key in this example is also 1.
Similarly, evaluating this expression "taskInput2Key": "${workflow.input.workflowInputJsonExample.nestedKey}" would result in "taskInput2Key": "nestedValue". So, the input to the task referred from the workflow input looks like this:
{
"taskInput1Key": 1,
"taskInput2Key": "nestedValue"
}
Task Inputs referred from other Task Outputs
Let’s assume that a task with the task reference name previousTaskReference produced the following output:
{
"taskOutputKey1": "outputValue",
"taskOutputKey2": {
"nestedKey1": "outputValue-1"
}
}
We can refer to these values as inputs to our new task using the following expression:
{
"taskInput1Key": "${previousTaskReference.output.taskOutputKey1}",
"taskInput2Key": "${previousTaskReference.output.taskOutputKey2.nestedKey1}"
}
The above expression can be evaluated using the same mechanism explained above, and finally, the task will receive the following inputs from the previous task output:
{
"taskInput1Key": "outputValue",
"taskInput2Key": "outputValue-1"
}
The expression format is based on JSON Path and you can construct complex input params based on the syntax.
Task Inputs referred from Secrets
Let’s assume that a secret named “api_key” is stored on your Conductor console, and you need to refer to this secret. The sample expression can look like this:
"taskInputKey": "${workflow.secrets.api_key}"
If the api_key has value Xxhhjiu0nbfdinvdHyj. Then the task input becomes:
"taskInputKey": "Xxhhjiu0nbfdinvdHyj"
Referring to task inputs using the secret functionality ensures that your secrets are not exposed in the workflow definitions; instead, it takes the value dynamically while executing the workflow.
Task Inputs referred from Variables
If the variable name is stored via the Set Variable task, the JSON looks like this:
"name": "set_variable_task_anmz4_ref",
"taskReferenceName": "set_variable_task_anmz4_ref",
"inputParameters": {
"name": "Orkes"
},
"type": "SET_VARIABLE",
So, here the variable name is set to Orkes. We can refer to this variable in the same workflow as follows:
"variable_name": "${workflow.variables.name}"
This results in "variable_name": "Orkes".
Task Input Templates
While creating task definitions, you can define the task input templates. These values will be supplied to each execution of the task and can be overridden within the workflow. It means that the parameters are included in the task definition, and whenever the task is used in a workflow, these parameters would be included by default in the workflow.
While creating a task definition, you can define the task input template as follows:
Using UI forms:
Using JSON code:
{
"name": "test-task",
"description": "Edit or extend this sample task. Set the task name to get started",
"retryCount": 3,
"timeoutSeconds": 3600,
"inputKeys": [],
"outputKeys": [],
"inputTemplate": {
"key": "value"
},
"ownerEmail": "name@example.com"
}
Adding this task to the workflow definition would already include the parameters supplied via the input task template.
These values can also be overridden by supplying input parameters to the task in the workflow definition.
Here, initially, some “key” were supplied through input templates. Now, when the same “key” was provided as input parameters in the workflow definition, the values got overridden.
You cannot view the input templates in the workflow definition JSON code as they are part of only task definitions. But, on clicking the task, you can view the input templates supplied from the UI.