Scheduling Workflows

The Scheduler allows you to trigger a workflow at a specific schedule. You can configure workflows to run at any desired frequency using cron expressions. A single workflow can be associated with multiple schedules. If there are many schedules, ensure the server capacity is sufficient to manage the load.

A schedule can be used for various cases, such as:

• Running a workflow at a fixed cadence, such as every hour.
• Running the same workflow at various cadences using different fixed inputs.
• Scheduling a workflow to run between specific dates.
• Running the workflow once at a future date.

Examples include:

• Renewing a TLS certificate for infrastructure components.
• Sending notifications ahead of account renewals.
• Scanning of S3 or Blob storage daily to verify compliance with policies
• Monitoring the health of a server, such as every 30 seconds.

Schedule parameters

Configure these parameters when creating a schedule.

Parameter	Description	Required/ Optional
name	The name of the schedule. Must be alphanumeric and can include underscores. Renaming an existing schedule creates a new one.	Required.
description	A description of the schedule.	Optional.
cronExpression	A 6-field string defining the cadence using cron syntax. For configuration details, refer Using cron expression.	Required.
zoneId	The timezone in which the schedule runs. Default is UTC.	Required.
startworkflowRequest	A JSON object containing the details of the workflow to be scheduled.	Required.
startworkflowRequest. name	The name of the workflow to run. The creator of the schedule must have execute permission for this workflow.	Required.
startworkflowRequest. version	The version of the workflow to run. If not specified, the latest version will be used.	Optional.
startworkflowRequest. input	The input parameters for the workflow, provided as a JSON object.	Optional.
startworkflowRequest. correlationId	A unique identifier for the workflow execution, used to correlate the current workflow instance with other workflows.	Optional.
startworkflowRequest. idempotencyKey	A unique, user-generated key to prevent duplicate workflow executions. Idempotency data is retained for the life of the workflow execution. Available since v3.3.52 and later	Optional.
startworkflowRequest. idempotencyStrategy	The idempotency strategy for handling duplicate requests. Supported values: RETURN_EXISTING—Return the workflowId of the workflow instance with the same idempotency key. FAIL—Start a new workflow instance only if there are no workflow executions with the same idempotency key. FAIL_ON_RUNNING—Start a new workflow instance only if there are no RUNNING or PAUSED workflows with the same idempotency key. Completed workflows can run again.	Required if idempotencyKey is used.
startworkflowRequest. priority	The priority of the workflow. Supports values from 0-99 and can be passed as a variable.	Optional.
scheduleStartTime	The start date and time for the schedule in Unix timestamp format (in milliseconds).	Optional.
scheduleEndTime	The end date and time for the schedule in Unix timestamp format (in milliseconds).	Optional.
taskToDomain	A mapping of task reference names to domain-specific values to route the task to defined workers.	Optional.
paused	Whether the schedule is paused upon saving. If set to true, the schedule will not run. Set to false to start the schedule immediately upon saving. Default is false.	Optional.
runCatchupScheduleInstances	If true, executes any pending schedules, such as when the server starts after downtime.	Optional

Using cron expression

The cronExpression defines the cadence of the scheduler using a 6-field string. An asterisk (*) denotes a blank entry.

* * * * * *

The terms, listed from left to right, are as follows:

• Second–Allowed values are 0-59.
• Minute–Allowed values are 0-59.
• Hour–Allowed values are 0-23.
• Day of Month–Allowed values are 1-31.
• Month–Allowed values are 1-12 or JAN-DEC.
• Day of Week–Allowed values are 0-7 or MON-SUN (where both 0 and 7 represent Sunday).

You can also use macros for common schedules, such as:

• @yearly or @annually–Executes once a year at Midnight on January 1st (0 0 0 1 1 *).
• @monthly–Executes once a month at midnight on the first day of the month (0 0 0 1 * *).
• @weekly–Executes once a week at midnight on Sunday (0 0 0 * * 0).
• @daily or @midnight–Executes once a day at midnight (0 0 0 * * *).
• @hourly–Executes once every hour (0 0 * * * *).

When the schedule triggers the workflow, the following fields are automatically injected as input to the workflow:

{
   "_executedTime": <EPOCH EXECUTED TIME>,
   "_startedByScheduler": "<SCHEDULE NAME>",
   "_executionId": "<SCHEDULE EXECUTION ID>",
   "_scheduledTime": <EPOCH SCHEDULED TIME>
}

Schedule configuration

This is the JSON schema for a workflow schedule.

{
 "paused": false,
 "runCatchupScheduleInstances": false,
 "name": "schedule_a",
 "description": "Sample schedule",
 "cronExpression": "0 0 * ? * *",
 "scheduleStartTime": 1735606800000,
 "scheduleEndTime": 1738285200000,
 "startWorkflowRequest": {
   "name": "demo-feb",
   "version": "",
   "input": {
     "id": "xyz"
   },
   "correlationId": "45",
   "idempotencyKey": "8",
   "idempotencyStrategy": "RETURN_EXISTING",
   "taskToDomain": {
     "task_x": "test"
   },
   "priority": 0
 },
 "zoneId": "UTC"
}

Configuring a schedule

Using Conductor UI (Scheduler)

To configure a schedule:

1. Go to Definitions > Scheduler from the left menu on your Conductor cluster.
2. Select + Define schedule.
3. Enter a Name and Description.
4. In Choose a template to get started, select a cron expression template or manually enter it. Use the Cron Expression Help to verify your expression.
5. Select your preferred timezone for the schedule. The UI will display the next five scheduled runs in both the selected timezone and the browser’s local time.
6. Select the Workflow name and Workflow version to be scheduled.
7. Enter any additional parameters such as Input params, Correlation Id, Idompotency key, and Idempotency strategy.
8. In Schedule start/end, enter the date and local time for the schedule to start and stop running. Supports the format YYYY-MM-DD hh:mm am/pm.
9. (Optional) Enable Pause schedule to prevent it from running upon creation. You can unpause the schedule later to resume.
10. Select Save > Confirm.

Upon schedule creation, you can perform the following actions:

• Pause–Pause the schedule, useful for situations like maintenance. Resume it once you're ready.
• Clone–Duplicate the schedule, allowing you to reuse existing schedule settings easily.
• Add/Edit Tags–Add new tags or modify existing ones for the schedule.
• Delete–Permanently remove the schedule from the Conductor cluster.

Conductor UI (Workflow Execution)

You can also easily create a schedule from a completed workflow execution.

To schedule a workflow from a workflow execution page:

1. Go to Definitions > Workflow from the left navigation menu.
2. Select the required workflow execution.
3. From Actions, choose Create Schedule.

This automatically fills a schedule definition with the workflow details, such as name, version, input parameters, correlation ID, idempotency key, and task-to-domain mapping. You only need to provide a name for the schedule and set the schedule time.

Using API

Use the following API to create a new schedule:

POST /api/scheduler/schedules

tip

For non-admin users to access schedules created by an admin, they must be granted permissions to the required schedules and workflows. This can be done using user groups and tags. Here’s how:

1. Add the necessary tag to the schedule and workflow to which non-admins need access.
2. Create a user group and grant execute and read access to the tag.

This ensures the group members have the necessary permissions to interact with the tagged schedule and workflows.

Monitoring scheduled executions​

To monitor the scheduled executions:

Go to Executions > Scheduler from the left navigation menu.

• The execution page displays the details for all scheduled executions, including scheduled time, execution time, execution ID, schedule name, workflow name, workflow ID, status, failure reason, and error details.
• You can view the workflow execution by directly clicking the workflow ID.
• Additional filtering options are available to refine your search.

To get the scheduled workflow execution details as code in the UI, click the arrow next to the Search button and select Show as Code.

This will display the execution search data in cURL and JavaScript.

FAQs

How do I set up a crontab to schedule a workflow every 15 minutes?

Using the crontab expression (0 */15 * ? * *), you can schedule a workflow to run every 15 minutes.

How do I set up a crontab to schedule a workflow every 90 minutes?

Crontab does not directly support scheduling workflows every 90 minutes as the minute field only supports values between 0 and 59. However, you can achieve this using two separate schedules with the crontab expressions (0 0-21/3 * * ? *) and (30 1-22/3 * * ? *).

Is there a limit to how many schedules you can set up with the same schedule?

No, there is no limit to the number of schedules you can assign to the same schedule.

Is there a limit to how many schedules a user can create?

No, there is no restriction on the number of schedules a single user can create.

Is there a limit to how many scheduled executions can run on the same cluster?

No, there is no limit to the number of scheduled executions that can run on the same cluster.

What is the smallest granularity to set up a schedule?

The scheduler supports up to a second granularity; however, this is done on a best-effort basis. A minimum of a 30-second gap is recommended as the cadence for workflow scheduling.

Can I get a summary view of all the schedules that ran in a given period, like last month, last week, etc.?

Yes. From the Scheduler executions page, you can leverage the time filtering options. You can provide the start and end times to view the schedules that ran within the stipulated period. In addition, the Lookback (days) field can give the number of schedules that ran on the specified days.

A schedule is created for a workflow for version 1. The workflow definition for that version changed later and now has new or updated input parameters. The schedule should also pass in the updated input parameter. Is there a way to find this?

When creating a schedule, you can choose a specific workflow version. If no version is selected, the latest version runs by default. If a version is chosen, you must manually update the schedule to use the latest version and the updated input parameters.