Unit and Regression Testing Workflows

Unit Tests

Conductor workflows can be unit-tested using POST /workflow/test endpoint. The approach is similar to how you unit test using mock objects in Java or similar languages.

Why Unit Test Workflows?

Unit tests allow you to test for the correctness of the workflow definition, ensuring:

1. Given a specific input, workflow reaches the terminal state in a COMPLETED or FAILED state.
2. Given a specific input, the workflow executes a specific set of tasks. This is useful for testing branching and dynamic forks.
3. Task inputs are wired correctly - e.g., if a task receives its input from the output of another task, this can be verified using the unit test.

Unit Testing Workflows

Java SDK provides the following method that allows testing a workflow definition against mock inputs:

public abstract Workflow testWorkflow(WorkflowTestRequest testRequest);

The actual workflow is executed on a real Conductor server, ensuring you are testing the behavior that will match the ACTUAL execution of the server.

Setting up Conductor server for testing

Tests can be run against a remote server (useful for integration tests) or a local containerized instance. A recommended approach is to use testcontainers.

Examples

Unit Test

• LoanWorkflowTest.java

/**
 * Unit test a workflow with inputs read from a file.
 */
public class LoanWorkflowTest extends AbstractWorkflowTests {

    /**
     * Uses mock inputs to verify the workflow execution and input/outputs of the tasks
     */
    @Test
    public void verifyWorkflowExecutionWithMockInputs() throws IOException {
        WorkflowDef def = getWorkflowDef("/workflows/calculate_loan_workflow.json");
        assertNotNull(def);
        Map<String, List<WorkflowTestRequest.TaskMock>> testInputs = getTestInputs("/test_data/loan_workflow_input.json");
        assertNotNull(testInputs);

        WorkflowTestRequest testRequest = new WorkflowTestRequest();
        testRequest.setWorkflowDef(def);


        LoanWorkflowInput workflowInput = new LoanWorkflowInput();
        workflowInput.setUserEmail("user@example.com");
        workflowInput.setLoanAmount(new BigDecimal(11_000));
        testRequest.setInput(objectMapper.convertValue(workflowInput, Map.class));

        testRequest.setTaskRefToMockOutput(testInputs);
        testRequest.setName(def.getName());
        testRequest.setVersion(def.getVersion());

        Workflow execution = workflowClient.testWorkflow(testRequest);
        assertNotNull(execution);

View the complete code here.

• Testing workflows that contain sub-workflows: SubWorkflowTest.java

/**
 * Demonstrates how to test workflows that contain sub-workflows
 */
public class SubWorkflowTest extends AbstractWorkflowTests {


    @Test
    public void verifySubWorkflowExecutions() throws IOException {
        WorkflowDef def = getWorkflowDef("/workflows/kitchensink.json");
        assertNotNull(def);

        WorkflowDef subWorkflowDef = getWorkflowDef("/workflows/PopulationMinMax.json");
        metadataClient.registerWorkflowDef(subWorkflowDef);


        WorkflowTestRequest testRequest = getWorkflowTestRequest(def);

        //The following are the dynamic tasks which are not present in the workflow definition but are created by dynamic fork
        testRequest.getTaskRefToMockOutput().put("_x_test_worker_0_0", List.of(new WorkflowTestRequest.TaskMock()));
        testRequest.getTaskRefToMockOutput().put("_x_test_worker_0_1", List.of(new WorkflowTestRequest.TaskMock()));
        testRequest.getTaskRefToMockOutput().put("_x_test_worker_0_2", List.of(new WorkflowTestRequest.TaskMock()));
        testRequest.getTaskRefToMockOutput().put("simple_task_1__1", List.of(new WorkflowTestRequest.TaskMock()));
        testRequest.getTaskRefToMockOutput().put("simple_task_5", List.of(new WorkflowTestRequest.TaskMock()));


        Workflow execution = workflowClient.testWorkflow(testRequest);
        assertNotNull(execution);

View the complete code here.

Regression Test

Workflows can be regression tested with golden inputs and outputs. This approach is useful when modifying workflows that are running in production to ensure the behavior remains correct.

See the below RegressionTest.java for an example, which uses previously captured workflow execution as golden input/output to verify the workflow execution.

/**
 * This test demonstrates how to use execution data from the previous executed workflows as golden input and output
 * and use them to regression test the workflow definition.
 * <p>
 * Regression tests are useful ensuring any changes to the workflow definition does not change the behavior.
 */
public class RegressionTest extends AbstractWorkflowTests {

    @Test
    //Uses a previously executed successful run to verify the workflow execution, and it's output.
    public void verifyWorkflowOutput() throws IOException, ExecutionException, InterruptedException, TimeoutException {
        //Workflow Definition
        WorkflowDef def = getWorkflowDef("/workflows/workflow1.json");

        //Golden output to verify against
        Workflow workflow = getWorkflow("/test_data/workflow1_run.json");

        WorkflowTestRequest testRequest = new WorkflowTestRequest();
        testRequest.setInput(new HashMap<>());
        testRequest.setName(def.getName());
        testRequest.setVersion(def.getVersion());
        testRequest.setWorkflowDef(def);

        Map<String, List<WorkflowTestRequest.TaskMock>> taskRefToMockOutput = new HashMap<>();
        for (Task task : workflow.getTasks()) {
            List<WorkflowTestRequest.TaskMock> taskRuns = new ArrayList<>();
            WorkflowTestRequest.TaskMock mock = new WorkflowTestRequest.TaskMock();
            mock.setStatus(TaskResult.Status.valueOf(task.getStatus().name()));
            mock.setOutput(task.getOutputData());
            taskRuns.add(mock);
            taskRefToMockOutput.put(def.getTasks().get(0).getTaskReferenceName(), taskRuns);
        }

        testRequest.setTaskRefToMockOutput(taskRefToMockOutput);
        Workflow execution = workflowClient.testWorkflow(testRequest);
        assertNotNull(execution);
        assertEquals(workflow.getTasks().size(), execution.getTasks().size());
    }


}

View the complete code here.