Scheduling Workflows
The Workflow Scheduler allows workflows to be triggered on a specific schedule. This enables workflows to be configured to run at any desired frequency using crontab expressions. The Scheduler can be used for various use cases, such as:
- Running a workflow in a fixed cadence, such as every hour
- Running the same workflow at different cadences using different fixed inputs
- Running a workflow at a cadence between a set start date and an end date
- Running the workflow once at a future date
Examples include:
- Renewing a TLS certificate for an infrastructure component
- Sending notifications ahead of account renewal times
- Daily scanning of the S3 or Blob storage to verify policies are not breached
- Check the health of a server URL every 30 seconds
Creating Scheduler
For creating a scheduler,
- From your Conductor server, navigate to Definitions > Scheduler.
- Click the + Define Schedule button at the top right corner.
- Configure the scheduler by providing the following parameters:
Parameter | Description |
---|---|
Name | A unique name for the scheduler. Ensure the scheduler name contains no spaces or special characters. Note: Renaming a scheduler creates a new scheduler. |
Description | A description for the scheduler. |
Cron Expression | The schedule on which the workflow will run. You can either input the cron expression or choose from the available templates. See the section Cron Expression for more details. |
Select Timezone | Select the required time zone for the scheduler to run. If you create the scheduler via UI, it displays the next five scheduled runs. |
Workflow name | Name of the workflow to be run. The author of the scheduler should have execution permission for the workflow. |
Workflow version | The selected workflow may have different versions. Choose the version to be used in this scheduler. If left blank, the latest version will be run. |
Input params | Provide fixed input values for the workflow as a JSON object. This field is optional. |
Correlation id | User-supplied ID to correlate or search the scheduled workflows. |
Idempotency key | A user-generated key to avoid conflicts with other workflows. Idempotency data is retained throughout the life of the workflow executions. |
Schedule start | Choose the date and local time for the scheduler to start running. It should be in the format yyyy-mm-dd hh:mm (a/p)m. This field is optional. |
Schedule end | Choose the date and local time for the scheduler to stop running. It should be in the format yyyy-mm-dd hh:mm (a/p)m. This field is optional. |
Task-to-domain mapping | Use this to limit the workflow execution to specific domains. This field is optional. |
Start schedule paused? | If selected, the schedule will be paused (so it will not run) upon creation. It is helpful when you don’t want to run the workflow scheduler, such as during maintenance. To resume the scheduled executions, this needs to be unpaused. |
- Click Save.
For non-admin users to work with schedulers created by an admin, they need to be provided access to the required schedulers and workflows. This can be granted through a user group. Here’s how you can achieve this:
- Add the required tag to the scheduler and workflow to which non-admins may need access. Let’s add the tag “team:execute”.
- Create a user group with required members and provide execute and read access to the tag “team:execute”.
Once the schedule is created, you can carry out the following actions:
- Pause: Use this option to pause your scheduler during situations such as maintenance. You can resume the scheduler once you're done.
- Clone: Use this option to duplicate your scheduler, making it easier to reuse your existing scheduler settings.
- Add/Edit Tags: Use this option to add a new tag to the scheduler or edit an existing tag in the scheduler.
- Delete: Use this option to permanently remove the scheduler from the Conductor console. This action cannot be undone.
Cron Expression
The standard cron expression can be used to schedule workflows.
Cron Expression
The UI has a Cron Expression Helper to describe the options of the cron expression. You can also test the expression after entering it, and the UI will automatically calculate the next few runs of the scheduler.
The CRON expression has six terms; you can denote a blank entry with an asterisk.
* * * * * *
The terms, in the order from left to right, define the timings:
- Second: Allowed values: 0-59
- Minute: Allowed values: 0-59
- Hour: Allowed values: 0-23
- Day of Month: Allowed values: 1-31
- Month: Allowed values are 1-12 or JAN-DEC
- Day of Week: Allowed values are 0-7 or MON-SUN, where 0 & 7 is Sunday
Macros are also supported when setting a schedule.
Macro | Meaning |
---|---|
@yearly (Or @annually) | Once a year (0 0 0 1 1 *) |
@monthly | Once a month (0 0 0 1 * *) |
@weekly | Once a week (0 0 0 * * 0) |
@daily (Or @midnight) | Once a day (0 0 0 * * *) |
@hourly | Once an hour, (0 0 * * * *) |
- A workflow can be invoked by any number of schedules.
- Execution history is maintained up to a configured maximum limit (Default of 1000 executions per scheduler).
- When there are a large number of schedulers, you must adjust the server capacity to manage the load.
- When the workflow is invoked, the following field will be injected automatically as the input:
{
"_executedTime": <EPOCH EXECUTED TIME>,
"_startedByScheduler": "<SCHEDULER NAME>",
"_executionId": "<EXECUTION ID>",
"_scheduledTime": <EPOCH SCHEDULED TIME>
}
Scheduler Executions
Once the schedulers are defined, you can view the executions from the Executions > Scheduler page.
The execution page lists the details of all the scheduler executions. It includes details such as scheduled time, execution time, execution ID, schedule name, workflow name, workflow ID, status, reason for failure, and error details. You can view the workflow execution by directly clicking on the workflow ID. You can also filter further information using the available options.
In addition, you can view the scheduler execution as code directly from the UI. To do this, click the arrow in the Search button and choose “Show as code”.
This will display the execution search data in various code formats. Currently, you can retrieve them in cURL and JavaScript.
Scheduling Workflows from Completed Executions
To schedule a workflow from a completed execution page,
- Navigate to the workflow execution page.
- From Actions, choose Create Schedule.
- This pre-fills a Scheduler Definition with the workflow name, version, input parameters, and task-to-domain mapping. You only need to provide a name for the scheduler and choose the schedule time.