Function Calling vs. Structured Output: Which One for Production LLM Apps?

**What it does:** You define one or more 'tools' (functions) the model can invoke. Each tool has a name, description, and parameter schema. The model decides whether to call a tool, which tool to call, and what parameters to pass. The actual function execution happens in your code; the model gets the result back in the next turn.

**Mechanism:** Model receives the tool definitions in the system prompt context. When the model wants to invoke a tool, it returns a structured tool-call object instead of plain text. Your code parses the tool call, executes the function, and feeds the result back to the model as a tool-result message.

**Best for:** Tasks where the model needs to interact with external systems (API calls, database queries, calculations) AND the model should decide whether/when to interact. The decision-making is part of the value.

**Documentation:** Anthropic tool use, OpenAI function calling.

**What it does:** You define a JSON schema (or equivalent structured spec). The model is forced to produce output that conforms to the schema. No decision-making about whether to produce output — the output IS the response.

**Mechanism:** The model's token sampling is constrained to only produce tokens that maintain schema-compliance. The output is guaranteed valid against the schema at the API level.

**Best for:** Tasks where you need reliable extraction or transformation INTO a specific shape — pulling fields from a document, converting natural language to a structured command, generating data records. The model isn't deciding whether to extract; you've already decided.

**Documentation:** OpenAI structured outputs, Google Gemini structured output. Anthropic supports structured output via tool use with a 'forced tool choice' parameter — slightly different mechanic, same outcome.

**Scenario 1 — Customer-support chatbot that sometimes needs to look up account info.** Model decides whether to call the lookup. **Function calling.** Forcing structured output here would require always returning the lookup, even when not needed.

**Scenario 2 — Extracting 12 fields from invoice PDFs.** Output shape is fixed (the 12 fields). Model isn't deciding; you're requiring extraction. **Structured output.** Forcing function calling here adds the unnecessary 'should I call this' overhead.

**Scenario 3 — Agent that does research across multiple tools.** Model decides which tools to call, in what order. **Function calling.** Multiple iterations possible.

**Scenario 4 — Classifying support tickets into 12 categories with confidence scores.** Output shape is fixed (category + confidence). Model isn't deciding whether to classify. **Structured output.**

**Scenario 5 — Natural-language-to-SQL converter.** Output shape is the SQL string (or structured query AST). **Structured output** — JSON schema for the query shape works well.

**Scenario 6 — Chat interface that should answer most questions conversationally but sometimes invoke a calculator or web search.** Mixed mode: model needs to decide. **Function calling.** Conversational answers come back as plain text; tool invocations come back as structured tool calls.

**Function calling per-query cost:** 1-3 model calls typically (initial + tool result + final response). Tool definitions add ~500-2000 tokens to system prompt context per call. Latency: 2-8 seconds depending on tool invocations.

**Structured output per-query cost:** 1 model call. No additional system prompt overhead. Latency: similar to plain-text generation (1-5 seconds).

**Reliability:** Both are highly reliable when used in their right context. Structured output guarantees schema compliance at the API level (the model literally cannot return invalid JSON). Function calling produces tool calls reliably but you still validate the tool arguments yourself.

**The cost difference:** Structured output is 2-3× cheaper per task than function calling for tasks where both could work. Function calling pays for the additional intelligence (the model's decision about whether to invoke); structured output doesn't have that intelligence but also doesn't pay for it.

Many production workflows benefit from combining. Common patterns:

**Function calling for tool selection + structured output for tool argument generation.** Model decides which tool to call (function calling), then a sub-call with structured output generates the exact arguments. Reduces tool-argument errors at modest cost premium.

**Routing via structured output + function calling in handlers.** Initial router call uses structured output to classify the request into a handler category; handler call uses function calling for the multi-tool interactions within the category.

**Final-answer structured output after a function-calling research phase.** ReAct-style research using function calling, then a structured-output final-answer pass that formats the result into a guaranteed-schema response. Common in agentic systems that need to return parseable results.

These combinations cost more than either pattern alone but produce more reliable end-to-end results for complex workflows. Per Anthropic's agent design guide, the highest-quality production agents typically combine multiple mechanisms rather than using one purely.

Picking either pattern by default: uses function calling for everything (paying for unneeded decision-making) or structured output for everything (forcing model to always return data when it should sometimes refuse).
Picking by workload signature: structured output when output shape is fixed and required; function calling when model decides whether/how to invoke; both combined for complex workflows. 2-3× cost difference per query on the simple cases; meaningful reliability difference on the complex ones.