Query Processor

Conductor supports querying from two primary sources:

Conductor Search API: This query type retrieves workflow execution data from the Conductor Search API using various parameters.
Conductor Metrics (Prometheus): This query type retrieves workflow and task performance metrics as well as system statistics through Prometheus. This query type requires your Conductor cluster to be connected to a Prometheus server.

Note

The Query Processor task is not enabled by default. Contact Orkes support to enable it for your cluster.

Task parameters

Configure these parameters for the Query Processor task.

Parameter	Description	Required/ Optional
inputParameters. queryType	The type of query. Supported types: `CONDUCTOR_API`: For querying using Conductor Search API. `METRICS`: For querying using Prometheus metrics. Values are case-sensitive. If `queryType` is missing or has an unsupported value, the workflow fails.	Required

CONDUCTOR_APIMETRICS

Parameter	Description	Required/ Optional
inputParameters. workflowNames	The names of the workflows to query. Can be a string or an array.	Optional.
inputParameters. correlationIds	The correlation IDs of the workflows to query. Can be a string or an array.	Optional.
inputParameters. statuses	The statuses of the workflows to query. Can be a string or an array. Supported values: RUNNING COMPLETED FAILED TIMED_OUT TERMINATED PAUSED	Optional.
inputParameters. startTimeFrom	The beginning of the start-time range, in minutes before the current time. Use with `startTimeTo` to query workflows that started between the two times. For example, `startTimeFrom` 15 and `startTimeTo` 0 query workflows that started in the last 15 minutes.	Optional.
inputParameters. startTimeTo	The end of the start-time range, in minutes before the current time. Use with `startTimeFrom`. Setting this to 0 includes workflows up to the current time. The value must be lower than `startTimeFrom`, or the task fails.	Optional.
inputParameters. endTimeFrom	The beginning of the end-time range for the query, measured in minutes from the current time.	Optional.
inputParameters. endTimeTo	The end of the end time range for the query, measured in minutes from the current time.	Optional.
inputParameters. freeText	Free text search parameter.	Optional.

Parameter	Description	Required/ Optional
inputParameters. metricsQuery	The Prometheus query to execute. Refer to the list of metrics published to the Conductor server to formulate your query.	Required.
inputParameters. metricsStart	The beginning of the time range for the query, in minutes from the current time. For example, setting this to 15 means the query will include data starting from 15 minutes ago. Must be a string, for example `"15"`.	Required.
inputParameters. metricsEnd	The end of the time range for the query, in minutes from the current time. Setting this to 0 means the query will include data up to the current time. Must be a string, for example `"0"`.	Required.
inputParameters. metricsStep	The time duration, in seconds, between data points in the query result (also known as the step or interval of the metrics query). For example, setting this to 1 means the query will include a data point for every 1 second. Must be a string, for example `"60"`.	Required.

Task configuration

This is the task configuration for a Query Processor task.

CONDUCTOR_APIMETRICS

{
     "name": "query_processor",
     "taskReferenceName": "query_processor_ref",
     "inputParameters": {
       "workflowNames": [
         "<WORKFLOW-NAME>"
       ],
       "statuses": [
         "<STATUS>"
       ],
       "correlationIds": [],
       "queryType": "CONDUCTOR_API",
       "startTimeFrom": 15,
       "endTimeFrom": 15,
       "startTimeTo": 0,
       "endTimeTo": 0,
       "freeText": "your_input_here"
     },
     "type": "QUERY_PROCESSOR"
}

{
  "name": "query_processor",
  "taskReferenceName": "query_processor_ref",
  "inputParameters": {
    "metricsQuery": "avg_over_time(cpu_usage{instance=\"your_instance\"}[1h])",
    "metricsStart": "60",
    "metricsEnd": "0",
    "metricsStep": "60",
    "queryType": "METRICS"
  },
  "type": "QUERY_PROCESSOR"
}

Task output

The Query Processor task will return the following parameters.

CONDUCTOR_APIMETRICS

Parameter	Description
result	An object that contains the query results.
result.count	The number of workflows returned in the current response.
result.totalHits	Total number of workflows matching the query criteria.
result.workflows	An array containing detailed information about each workflow returned by the query.
result.workflowsUrl	URL linking to the queried workflow executions in the Conductor UI, with specific query parameters.

Parameter	Description
result	An object that contains the Prometheus query response.
result.data.result[].metric	A key-value map of the details of the Conductor cluster that was queried, such as the cluster name, pod, and workflow name.
result.data.result[].values	An array containing the queried metric at each timestamp interval.

Examples

Here are some examples for using the Query Processor task.

Using CONDUCTOR_API

This example shows how to use the Query Processor task to retrieve completed executions of a workflow by querying the Conductor Search API.

You will build and run a workflow that queries completed executions of a particular workflow from the last 15 minutes.

Prerequisites

An existing workflow that needs to be queried.
At least one completed execution of the workflow within the last 15 minutes

Step 1 : Create a workflow in Orkes Conductor

Go to Definitions > Workflow, from the left navigation menu on your Conductor cluster.
Select + Define workflow.
In the Code tab, paste the following code.

{
 "name": "WorkflowTest",
 "description": "Sample workflow",
 "version": 1,
 "tasks": [
   {
     "name": "query_processor",
     "taskReferenceName": "query_processor_ref",
     "inputParameters": {
       "workflowNames": [
         "<YOUR-WORKFLOW-NAME-TO-BE-MONITORED>"
       ],
       "statuses": [
         "COMPLETED"
       ],
       "correlationIds": [],
       "queryType": "CONDUCTOR_API",
       "startTimeFrom": 15,
       "startTimeTo": 0
     },
     "type": "QUERY_PROCESSOR"
   }
 ],
 "schemaVersion": 2
}

Save the workflow.

Replace <YOUR-WORKFLOW-NAME-TO-BE-MONITORED> with the name of the workflow you want to query. In this example, the Query Processor task retrieves completed executions of that workflow from the last 15 minutes.

Select Execute to run the workflow. When the workflow runs, the Query Processor task queries the Conductor Search API and returns execution details for matching workflows.

The output includes information such as:

Workflow ID
Execution status
Start and end times
Execution duration
Workflow inputs and outputs

Using METRICS

In this example, the Query Processor task searches the Prometheus for the number of workflows started within the past minute. The query resolution is set at an interval of 10 seconds.

Task Configuration

{
  "name": "query_processor",
  "taskReferenceName": "query_processor_ref",
  "inputParameters": {
    "metricsQuery": "workflow_start_request_seconds_count{workflowName=\"indexed_qna_slack\"}",
    "metricsStart": "1",
    "metricsEnd": "0",
    "metricsStep": "10",
    "queryType": "METRICS"
    },
  "type": "QUERY_PROCESSOR"
}

Example Output

The query returns a Prometheus time series response. The values array contains the queried metrics at each given timestamp.

{
  "result": {
    "data": {
      "result": [
        {
          "metric": {
            "container": "conductor",
            "cluster_name": "someCluster",
            "endpoint": "default-app-port",
            "instance": "00.00.0.000:0000",
            "pod": "somePod",
            "__name__": "workflow_start_request_seconds_count",
            "service": "conductor-app",
            "namespace": "someNameSpace",
            "workflowName": "indexed_qna_slack",
            "job": "conductor-app"
          },
          "values": [
            [1723110950, "1390"],
            [1723110960, "1390"],
            [1723110970, "1390"],
            [1723110980, "1390"],
            [1723110990, "1390"],
            [1723111000, "1390"],
            [1723111010, "1390"]
          ]
        }
      ],
      "resultType": "matrix"
    },
    "status": "success"
  }
}

Query Processor

Task parameters

Task configuration

Task output

Examples

Related pages