You ask an LLM to return JSON. Most calls come back clean. Then one call wraps the answer in markdown fences, another renames a field, a third sends "total" as a string instead of a number, and your pipeline fails at two in the morning. Prompting is not a contract.
Providers now solve the "valid JSON" half of this with structured outputs: grammar-constrained decoding that forces the response to match a schema you provide. You can stop parsing with regex and hoping. This guide covers how to use structured outputs on OpenAI and Anthropic, where the providers differ, and why you still need a validation layer on top.
Prerequisites
- Python 3.10+ and pip
- An OpenAI and/or Anthropic account with an API key
pip install openai anthropic pydantic
The failure prompt engineering cannot fix
The classic approach is putting "Return JSON" in the system prompt. It works most of the time, and "most" is the problem. Models occasionally return valid JSON with wrong field names, wrap the response in code fences, or emit an extra key the schema never listed. No amount of wording in the prompt guarantees structure. Structured outputs turn that guarantee into a platform feature enforced at sampling time.
Step 1: OpenAI structured outputs with Pydantic
Define the shape once in code, pass it to the API, and get a typed object back.
from openai import OpenAI
from pydantic import BaseModel, Field
client = OpenAI()
class Invoice(BaseModel):
invoice_number: str
date: str
currency: str = Field(description="ISO 4217 code, e.g. USD")
total_amount: float
def extract_invoice(text: str) -> Invoice:
response = client.responses.parse(
model="gpt-5.6",
input=[
{"role": "system", "content": "Extract the invoice fields from the text."},
{"role": "user", "content": text},
],
text_format=Invoice,
)
return response.output_parsed
output_parsed is already an Invoice object, so no json.loads or manual field checks, and safety refusals come back flagged instead of as silent garbage.
Two rules shape every strict schema on OpenAI. Every object must set additionalProperties: false, and every field must appear in required. That means no truly optional fields. If a value can be absent, type it as nullable instead, so str | None rather than an empty string.
Step 2: Anthropic, two ways
Anthropic's Messages API has a native JSON output mode on newer models: add output_config.format with a raw JSON Schema, and the response is guaranteed to match it. For function calling, which every supported model exposes, the same guarantee works through strict tool use: declare a tool whose input_schema is your JSON shape and pin tool_choice to force that tool.
from anthropic import Anthropic
client = Anthropic()
schema = {
"type": "object",
"properties": {
"invoice_number": {"type": "string"},
"date": {"type": "string"},
"currency": {"type": "string"},
"total_amount": {"type": "number"},
},
"required": ["invoice_number", "date", "currency", "total_amount"],
}
response = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=1024,
tools=[
{
"name": "record_invoice",
"description": "Extract structured invoice fields",
"input_schema": schema,
"strict": True,
}
],
tool_choice={"type": "tool", "name": "record_invoice"},
messages=[{"role": "user", "content": text}],
)
tool_use = next(b for b in response.content if b.type == "tool_use")
invoice = tool_use.input
With tool_choice forced to one tool, the model must call exactly that tool, and input is the parsed object matching your schema. For one-shot extraction you can ignore tool_use.id; you only echo it back in a tool_result block if you keep the conversation going.
Step 3: The validation sandwich
Structured outputs guarantee the JSON parses and matches your schema. They do not guarantee the data is right. The total can parse as a number and still disagree with the line items, and the customer ID can point to nobody.
So you layer checks. The schema defines shape, business rules refine it, and your application verifies against real state. With Pydantic:
from pydantic import BaseModel, field_validator
class Order(BaseModel):
product_id: str
quantity: int
unit_price: float
total: float
@field_validator("total")
@classmethod
def total_matches(cls, v, info):
qty = info.data["quantity"]
price = info.data["unit_price"]
if abs(v - qty * price) > 0.01:
raise ValueError("total must equal quantity * unit_price")
return v
Trust no single layer. The model guarantees the shape, your validator rejects the impossible numbers, and your code checks the product actually exists and is in stock.
Cost and schema tips
- Keep schemas small. Every field the model must reason about costs tokens and adds room for error. Extract only what you use.
- Prefer enums over open strings.
{"type": "string", "enum": ["low", "medium", "high"]}blocks "HIGH" and "high priority" before they reach your database. - Use the cheapest model that still parses reliably. Simple extraction rarely needs a frontier model.
- Cache your schemas. Prompt caching applies to the schema in your system prompt across repeated calls, which dominates the bill on a busy pipeline.
Where to go next
Wire extraction into an agent loop where a tool call is also a step in a workflow, and validate every tool output the same way. Then pair structured outputs with prompt caching so the guaranteed shape also stops costing you money.