How to Use Parallel Tool Calls

When you give a model a set of tools (functions it can call), it normally responds either with a final answer or with a request to call one tool. **Parallel tool calling** is when the model returns multiple tool-call requests in the same assistant turn — for example, asking to call `get_weather(city)` and `get_traffic(route)` and `get_calendar(date)` simultaneously because none of them depends on the others.

Your application receives all three call requests, executes them concurrently, and then sends all three results back to the model in one message — each result tagged with the tool-call ID it answers. The model then reasons over the complete result set and produces its response or its next batch of calls.

The key insight is independence: parallel calls only make sense when the calls don't depend on each other's output. If call B needs the result of call A, those must run sequentially. The model is generally good at recognizing independence, but your prompt and tool descriptions strongly influence whether it batches correctly.

Most major providers support this. OpenAI's function calling and Anthropic's tool use both allow multiple tool calls per turn — see the OpenAI models docs and the Anthropic models overview for current support. The MCP ecosystem works the same way at the protocol level.

**Use them for independent reads.** Fetching several unrelated pieces of data — three API lookups, multiple file reads, querying different services — is the canonical case. They have no dependency on each other, so running them concurrently is pure latency savings.

**Use them for fan-out research.** An agent gathering context from several sources (search + database + a docs lookup) can request all of them at once instead of trickling through them one per turn.

**Avoid them when calls depend on each other.** If you need an order ID before you can fetch the order's line items, those are sequential by nature. Forcing parallelism here just produces a call with a missing argument.

**Be careful with writes and side effects.** Two parallel calls that both modify the same resource can race. Reads parallelize cleanly; writes need the same concurrency discipline you'd apply to any concurrent code — locking, idempotency keys, or simply keeping mutations sequential.

**Watch rate limits and cost.** Firing ten tool calls at once can hit downstream API rate limits or blow a budget faster than serial calls. Add concurrency caps (e.g., run at most N at a time) for high-fan-out cases.

Consider an assistant that answers "Should I bike to my 9am meeting downtown?" It needs the weather, the traffic, and the meeting location. **Before — serial**, the conversation takes three full round-trips:

``` Turn 1: model -> call get_weather("downtown") app -> result: "rain, 12°C" Turn 2: model -> call get_traffic("home->downtown") app -> result: "heavy, 25 min" Turn 3: model -> call get_calendar("today 9am") app -> result: "Acme review, 3rd Ave office" Turn 4: model -> final answer ```

Three serial waits, each paying full model + network latency. **After — parallel**, the model batches the independent calls into one turn:

``` Turn 1: model -> [ call get_weather("downtown"), call get_traffic("home->downtown"), call get_calendar("today 9am") ] app -> [ {id: call_1, result: "rain, 12°C"}, {id: call_2, result: "heavy, 25 min"}, {id: call_3, result: "Acme review, 3rd Ave office"} ] Turn 2: model -> final answer ```

The three tools run concurrently in your code; the model waits once instead of three times. In pseudo-code, your execution layer looks like:

``` const calls = assistantTurn.toolCalls; const results = await Promise.all( calls.map(c => runTool(c.name, c.arguments)) ); // return results back, each tagged with its call ID sendToolResults(calls.map((c, i) => ({ tool_call_id: c.id, content: results[i], }))); ```

The critical detail: **match every result to its call ID** and return them all in one follow-up message. Mismatched or dropped IDs are the number-one parallel-tool bug.

**Mismatched call IDs.** Each tool result must reference the ID of the call it answers. If you reorder results or drop one, the model gets confused about which answer goes with which question. Always map results back by ID, never by position assumption.

**Returning partial results too early.** Wait for all parallel calls to settle before sending the batch back. If one tool is slow, decide deliberately: wait, or time it out and return an explicit error result for that call — don't silently omit it.

**Letting the model parallelize dependent calls.** If your tool descriptions don't make dependencies clear, the model may try to call a tool with an argument it doesn't have yet. Describe in each tool's spec what inputs it needs and where they come from.

**Unbounded fan-out.** A model that can call a tool in a loop can request many at once. Cap concurrency and total calls per turn to protect downstream rate limits and your budget. See tool use and MCP in production for production guardrails.

**Errors in one call.** A failed call shouldn't crash the whole batch. Catch per-call errors and return them as error results so the model can decide to retry, route around, or report the failure — this mirrors how robust agent design patterns handle tool failures.

Parallel tool calling is an optimization layer on top of the standard tool-use / ReAct loop, where a model alternates between reasoning and acting. Within a single "act" step, instead of one action, the model can emit several independent actions to run together. This is fully compatible with structured output and MCP — the transport differs, but the batch-and-return-by-ID pattern is the same.

If you're choosing between returning structured data and calling tools, our function calling vs structured output guide covers the distinction; parallelism applies specifically to the tool-calling path. For the schema design behind clean tool arguments, see structured output schema design patterns.