Skip to content

Massively Parallel Agents

%%{init: {'look': 'handDrawn', 'theme': 'base', 'themeVariables': {'primaryColor': '#eef2ff', 'primaryBorderColor': '#1e40af', 'primaryTextColor': '#1e293b', 'lineColor': '#1e3a8a', 'edgeLabelBackground': '#ffffff', 'clusterBkg': '#fbfcff', 'clusterBorder': '#2563eb', 'fontFamily': '-apple-system, system-ui, Segoe UI, Roboto, Helvetica, Arial, sans-serif', 'fontSize': '15px'}, 'flowchart': {'nodeSpacing': 50, 'rankSpacing': 58, 'padding': 14, 'htmlLabels': true, 'curve': 'basis'}}}%%
flowchart LR
  R(["Request"]) --> C("Coordinator<br/>splits the work")

  subgraph fan["100 sub-agents · all at once"]
    direction TB
    W1("worker 1")
    W2("worker 2")
    WN("worker 100")
  end

  C ==> W1
  C ==> W2
  C ==> WN
  W1 --> S("Coordinator<br/>synthesizes")
  W2 --> S
  WN --> S
  S --> O(["Report"])
  style fan stroke-dasharray: 6 5

Outcome: a coordinator decomposes one request into a hundred independent sub-tasks, runs them all in parallel as durable sub-workflows, and writes up the combined result.

How it works

  • scatter_gather() builds the coordinator for you — decompose, fan out, synthesize.
  • The fan-out width is decided at runtime by the model, not hardcoded in the graph.
  • Each sub-task is its own sub-workflow with its own retries.
  • Partial results are the default. fail_fast=False means one dead worker doesn't sink the batch.
  • Use a bigger model to synthesize. It has to read all hundred results at once.

Prerequisites

A Conductor server with an LLM provider, and CONDUCTOR_SERVER_URL set. This run makes roughly 100 worker calls plus one large synthesis call — check your provider's rate limits first.

The agents

Save this as agent_scatter_gather.py:

"""Scatter-gather — one coordinator fans out to 100 parallel sub-agents.

Derived from sdk/python-sdk/examples/agents/58_scatter_gather.py.

scatter_gather() builds a coordinator that decomposes the request, dispatches
the worker agent N times through FORK_JOIN_DYNAMIC, and synthesizes the results.
N is decided by the model at runtime. Every sub-task is its own durable
sub-workflow, so one flaky worker retries on its own and the coordinator still
synthesizes partial results.
"""

from conductor.ai.agents import Agent, AgentRuntime, scatter_gather, tool

MODEL = "openai/gpt-4o-mini"
SYNTHESIS_MODEL = "openai/gpt-4o"  # larger context, it sees all 100 results


@tool
def search_knowledge_base(query: str) -> dict:
    """Look up a topic. Replace with a real search or vector-DB call."""
    return {
        "query": query,
        "results": [
            f"{query}: mid-sized economy with a services-led profile",
            f"{query}: population growth close to the regional average",
        ],
    }


researcher = Agent(
    name="country_researcher",
    model=MODEL,
    instructions=(
        "You profile one country. Call search_knowledge_base exactly once, then "
        "write 2-3 sentences covering economy, population and one distinctive fact. "
        "Do not call the tool more than once."
    ),
    tools=[search_knowledge_base],
    max_turns=5,
)

COUNTRIES = ['Afghanistan', 'Albania', 'Algeria', 'Andorra', 'Angola', 'Argentina', 'Armenia', 'Australia', 'Austria', 'Azerbaijan', 'Bahamas', 'Bahrain', 'Bangladesh', 'Barbados', 'Belarus', 'Belgium', 'Belize', 'Benin', 'Bhutan', 'Bolivia', 'Bosnia and Herzegovina', 'Botswana', 'Brazil', 'Brunei', 'Bulgaria', 'Burkina Faso', 'Burundi', 'Cambodia', 'Cameroon', 'Canada', 'Chad', 'Chile', 'China', 'Colombia', 'Congo', 'Costa Rica', 'Croatia', 'Cuba', 'Cyprus', 'Czech Republic', 'Denmark', 'Djibouti', 'Dominican Republic', 'Ecuador', 'Egypt', 'El Salvador', 'Estonia', 'Ethiopia', 'Fiji', 'Finland', 'France', 'Gabon', 'Georgia', 'Germany', 'Ghana', 'Greece', 'Guatemala', 'Guinea', 'Haiti', 'Honduras', 'Hungary', 'Iceland', 'India', 'Indonesia', 'Iran', 'Iraq', 'Ireland', 'Israel', 'Italy', 'Jamaica', 'Japan', 'Jordan', 'Kazakhstan', 'Kenya', 'Kuwait', 'Laos', 'Latvia', 'Lebanon', 'Libya', 'Lithuania', 'Luxembourg', 'Madagascar', 'Malaysia', 'Mali', 'Malta', 'Mexico', 'Mongolia', 'Morocco', 'Mozambique', 'Myanmar', 'Nepal', 'Netherlands', 'New Zealand', 'Nigeria', 'North Korea', 'Norway', 'Oman', 'Pakistan', 'Panama', 'Paraguay']

country_list = "\n".join(f"{i + 1}. {c}" for i, c in enumerate(COUNTRIES))

coordinator = scatter_gather(
    name="country_coordinator",
    worker=researcher,
    model=SYNTHESIS_MODEL,
    instructions=(
        f"Create EXACTLY {len(COUNTRIES)} country_researcher calls, one per country "
        f"below, passing just the country name. Issue ALL calls in a SINGLE response.\n\n"
        f"Countries:\n{country_list}\n\n"
        f"When all {len(COUNTRIES)} results are back, compile a short report grouped "
        f"by region."
    ),
    retry_count=3,
    retry_delay_seconds=5,
    timeout_seconds=900,
)


if __name__ == "__main__":
    with AgentRuntime() as runtime:
        result = runtime.run(
            coordinator,
            f"Profile all {len(COUNTRIES)} countries in the list.",
        )
        result.print_result()
        print("execution id:", result.execution_id)

Run it

python agent_scatter_gather.py

A verified run finished in 41 seconds using 56,371 tokens. Inspecting the execution shows what actually happened: 100 SUB_WORKFLOW tasks under a single FORK/JOIN, all dispatched together.

Open Executions and open the coordinator — the parallel branches are laid out side by side, and you can drill into any one of the hundred.

The same example in other SDKs

The agent API is the same shape in every SDK. These are the upstream sources this recipe was derived from:

SDK Example
Python 58_scatter_gather.py
Java Example58ScatterGather.java
TypeScript 58-scatter-gather.ts
C# Program.cs

Production notes

  • Rate limits bite before Conductor does. A hundred simultaneous calls will hit a provider quota long before the engine struggles.
  • Cap the worker's turns. max_turns stops one worker looping and holding the join open.
  • Watch the synthesis context. A hundred verbose workers can exceed the coordinator's window; keep worker output short.
  • Partial success needs a decision. Decide what an 97-of-100 result means for your caller before you ship it.
  • Cost scales linearly. Test the shape with five workers before running a hundred.