Agents on Conductor — Architecture for Production AI

Here is a question for every team building AI agents: when your model emits a tool call, what actually stands between that call and execution?

In most agent stacks, the honest answer is: the model’s good behavior. The tool call goes straight to a dispatcher whose job is “do what the model said.” Whether a guardrail runs, whether an approval is required, whether the tool is even in scope — all of it depends on the model having behaved.

A guardrail that runs because the model chose to invoke it is not a guardrail — It is a suggestion.

Distributed systems solved this shape of problem a long time ago, and the pattern is one every infrastructure engineer already lives with: separate the control plane from the data plane. Agents need the same split. The model belongs in a reasoning plane — it looks at state, weighs the goal, and decides what should happen. The execution plane — deterministic, policy-enforcing, durable — decides what actually runs. Nothing crosses from one plane to the other without being validated.

We think that missing split is the single biggest thing keeping agents out of production. So we built it in, and today we are shipping it.

Conductor has been orchestrating production workflows since its Netflix days, and for years the community’s favorite question was how far it scales (a billion workflows a month, it turns out). The question has changed. Today it is: how do we put agents in production?

This is our answer.

Conductor (https://github.com/conductor-oss/conductor) now runs AI agents as first class citizens. You can write agents using any of the supported SDKs in Conductor OR bring the agent with the framework you already use — LangChain, LangGraph, OpenAI, Vercel, Google ADK — Conductor compiles it into a durable workflow and executes it under one rule that never breaks: the LLM reasons; it does not execute.

The LLM reasons, Conductor executes

The LLM is used purely as a reasoning engine: it looks at the current state, the goal, and the tools it has, and proposes what should happen next. That proposal is data. The engine decides whether it becomes an action.

Concretely: if an agent declares tools A, B and C, the LLM cannot decide to execute tool X — and when it emits a call to X anyway, the engine rejects it before anything is scheduled and returns a recorded error to the conversation:

Unknown tool 'str_replace'.
Available tools: check_inventory, create_ticket, notify_oncall

That example comes straight from a comment in our compiler — models really do emit tool names nobody ever offered them. Because the check is compiled into the workflow graph rather than written into the prompt, it holds no matter what the model outputs.

Remember, none of this makes the LLM deterministic — it won’t be, and it doesn’t need to be. Determinism is the execution plane’s job, and every rule of engagement is a property of the compiled graph, enforced no matter what the model returns:

Table of rule of engagement and what it's enforced by

How does an agent become a workflow?

The agent definition compiles into a workflow definition with a ReAct loop. Session state is maintained using workflow variables and reasoning is done by an LLM using LLM_CHAT_COMPLETE task. Every task in Conductor has built-in retry policies and durability built-in — they come for free. Tool execution happens via FORK_JOIN_DYNAMIC fan-out or SUB_WORKFLOW task that can run entire graphs.

Guardrails are compiled so if the tool has a guardrail it gets scheduled and executed before the actual tool is called.

A tool is just a task that the platform already has: your worker function, an HTTP endpoint, an MCP tool, a person, another agent, or an entire workflow.

Every execution also carries a principal, and credentials resolve against it at poll time — your agent code never embeds a key.

LLM-written plans compile or they don’t run. When the agent produces a multi-step plan, the plan JSON is compiled into a validated workflow definition before any of it runs: unknown tools, cycles, dangling references and out-of-grammar conditions all fail compilation — success conditions go through a real AST whitelist parser, because a regex denylist in front of a JS engine is not defense in depth — and each step’s arguments are validated against the tool’s input schema. An invalid plan fails as a unit and routes to a fallback path; it never executes halfway.

Bring the agent you already wrote

Here is the full lifecycle with LangChain:

from conductor.ai.agents import AgentRuntime
from langchain.agents import create_agent
from langchain_core.tools import tool

@tool
def check_token() -> str:
    """Check a token."""
    return "available"

agent = create_agent("openai:gpt-4o-mini", tools=[check_token],
                     system_prompt="You are a helpful assistant.")

with AgentRuntime() as runtime:
    runtime.run(agent, "Is the token set?")  # develop: compile and execute once
    runtime.deploy(agent)                    # release: register without executing
    runtime.serve(agent)                     # operate: run tool workers and block

run() executes the agent durably — the whole execution is visible in the UI from the first try.

deploy() registers the agent on the server under a name and a version, after which anyone can invoke it without importing your framework or its dependencies.

serve() runs the worker process for your tool functions.

Long Horizon Agents

Real agent work is long-horizon: it waits for approvals, rate limits, people, arrival of end of day closing data etc. Most runtimes hold the loop in memory, so the agent lives exactly as long as its process. On Conductor the loop is a workflow — every transition is persisted before the next task is scheduled, so a restart, a deploy or a crash resumes the run from where the work actually was.

That durability changes the horizon:

Waiting is free. A HUMAN approval or a message wait is durable — no thread parked, no worker polling, nothing burned while nothing happens.
You can talk to a running agent. A signal lands in the next model turn as context, so new facts reach a mid-flight agent without a restart.
You can stop one gracefully. The agent finishes its current turn and completes with its last output instead of dying mid-tool-call.
Agents run on events and cadences. A deployed agent is a workflow, so Conductor’s event-driven recipes and scheduler apply unchanged: a Kafka, SQS, NATS or AMQP event can start one, a signal can resume one, and a nightly agent is one cron expression.
Retries and replay are built-in. These are not specialized scripts or manual checkpoints you have to write.

The single-responsibility agent

The teams that succeed in production stop building an agent and start building many small ones: one agent, single responsibility, a well defined set of tools and guardrails, a clear contract at the boundary — the same discipline that took us from monoliths to microservices, for the same reason.

A small agent’s failure modes are enumerable: you can list everything it touches, write the eval set, hand it the narrowest credentials. An agent with forty tools and a system prompt describing your entire business is not a powerful agent. It is a distributed system with no interfaces.

How do you build something big out of small agents? The answer: composition.

Agents and workflows, living together in harmony

Complex systems — agent harnesses — are essentially workflows: deterministic tools and processes interwoven with agent execution. The wiring goes both directions: a workflow calls an agent as an ordinary step, and an agent calls an entire workflow as a tool.

An agent monitors your Kubernetes cluster; an alert event starts it — nothing sat polling. The agent reads the logs, applies your policies, and decides the cluster needs a restart. That is reasoning, exactly what you want an LLM for. But restarting a cluster is not reasoning: check the cluster type, notify the owners, get an approval if it is production, drain, restart, verify health, roll back on failure. Those are deterministic paths that must not deviate from run to run — a workflow.

So the agent decides that the cluster restarts; the workflow determines how, the same way every time. Reasoning where judgment is needed, deterministic execution everywhere else — that is the shape we believe production agentic systems converge on.

Durability, explainability, determinism

Strip away the mechanics and this release stands on three pillars — the properties the execution plane exists to provide, each answering a different question in an agent’s life.

Determinism answers the question before the run: what can this agent do? Tools, guardrails and bounds are compiled and enforced, so the answer is a finite list your security team can read, not a prediction about model behavior.

Durability answers the question during the run: will the work survive? The loop is a workflow, not a process — restarts, deploys and three-day approvals do not lose it.

Explainability answers the question after the run: what did it actually do? Because the engine can only act by recording, the execution history is not telemetry about the run — it is the run, stored for months and walkable in the UI, turn by turn.

Miss any one of the three and you have a demo. Deliver all three and you have an agent you can put in production.

Try it

Your first agent — a working agent on a local server in minutes

Agent guardrails — enforcement points and failure outcomes

Multi-agent architecture — supervisor, swarm and pipeline shapes

Can you run agents in production the way you run services in production — bounded, governed, observable, durable? With this release, the answer is a resounding yes.

Build the brain with whatever framework you like. Let Conductor be the hands.

Don’t forget to give us a ⭐️ https://github.com/conductor-oss/conductor