Function Calling vs Structured Output APIs: Which to Use and When

You want the model to return data your application can consume programmatically. You have two main paths: function calling (also called tool use) and structured output mode. They look similar on the surface — both produce machine-readable output — but they model different assumptions about how the LLM will interact with your system. Choosing the wrong one for your use case adds unnecessary complexity and introduces subtle failure modes.

What Function Calling Actually Is

Function calling gives the model the ability to request that your application execute a function. You define functions with names, descriptions, and parameter schemas. When the model decides a function should be called, it outputs a structured call request — function name plus arguments — instead of a text response. Your application executes the function, returns the result, and the model continues.

The key concept: the model is deciding when and whether to call a function. It is an agentic mechanism. The model reads the function descriptions, reasons about whether any of them apply to the current situation, and chooses to invoke one. This makes function calling powerful for tool-augmented agents and multi-step workflows. It also makes it less appropriate when you simply want structured output — you are adding an agent-style decision layer to what is essentially a formatting requirement.

What Structured Output Mode Is

Structured output mode (available as response_format: json_schema in OpenAI's API, or via tool use with a single forced tool in Anthropic's API) constrains the model's generation so that every token it produces conforms to your schema. The model does not decide whether to produce structured output — it always does. You define the schema, and the output is guaranteed to validate against it.

This is the right mechanism when you want to extract or generate data in a defined shape. The model is not making a routing decision. It is answering your prompt in a constrained format.

Comparing the Two Approaches

Reliability of structure

Structured output mode guarantees schema-valid output through constrained decoding — invalid tokens are masked at generation time. Function calling produces structured output by asking the model to format its response correctly, and the schema enforcement depends on the provider's implementation. For most major providers, function call arguments are also schema-constrained, but the mechanism is less universal and the guarantees vary.

For extraction tasks where structural reliability is the primary goal, structured output mode is the more direct tool.

Model agency

Function calling requires the model to decide whether to call a function and which one. This agency is useful when the model should make routing decisions — a support agent that can either answer directly or escalate to a ticketing tool, for example. It is unnecessary overhead when you always want structured output. Forcing a decision the model should not need to make adds tokens and creates a subtle failure mode: the model may decide not to call the function when you expected it to.

If you always want structured output from a given prompt, configure the model to always produce it. In OpenAI's API, set response_format. In Anthropic's API, force a tool call. Do not leave the decision to the model.

Multi-step and multi-tool scenarios

Function calling is the right choice for agentic workflows where the model may need to call multiple tools in sequence, combine results, and produce a final response. Structured output mode produces a single response in a defined format — it is not designed for multi-step tool orchestration.

If your use case requires the model to search a database, then look up a customer record, then format a summary — that is an agentic workflow and function calling is appropriate. If your use case requires the model to read a document and output a JSON object with extracted fields, that is a formatting task and structured output mode is appropriate.

Debugging and observability

Function calling introduces a conversation turn boundary — the model outputs a function call, your application executes it, and the result comes back. This creates a natural logging point for every tool invocation. Debugging an agentic workflow benefits from this structure.

Structured output mode is a single-turn interaction. Simpler to log and debug for non-agentic tasks. Harder to instrument for multi-step workflows because the "steps" all happen inside the model's context, not as discrete API calls.

Practical Decision Guide

Use structured output mode when:

• You always want the model's response in a specific schema, with no routing decision required
• You are extracting data from documents, classifying inputs, or generating structured records
• Structural reliability is a primary requirement
• You want the simplest possible implementation for a data-in, data-out task

Use function calling when:

• The model should decide whether and which tool to invoke based on the input
• Your system has multiple tools the model can choose between
• You are building an agent that performs multi-step operations using external systems
• You want the model to use external data (APIs, databases) before formulating a response

Use both together when:

• You have an agentic workflow where tool calls should return structured data
• Each tool in your function set returns data that must conform to a schema

A Note on Forced Tool Calls

Some use cases call for function-calling mechanics (because the model output goes through the tool use API pathway for compatibility reasons) but without model agency over whether to call. Most providers support forcing a specific tool call — the model generates arguments for the specified function, but the decision to call it is removed. This pattern gives you schema-constrained output through the tool use pathway without ceding the routing decision to the model.

This is a useful bridge when your infrastructure expects function call format but your task is fundamentally a structured extraction.

Building LLM systems that correctly separate agentic decisions from formatting requirements is a design judgment that affects reliability and maintainability at scale. Clixo designs and ships these systems — reach out to discuss your architecture.