Structured outputs & reliability

A chatbot can get away with prose. A feature can’t. The moment your code needs to act on what a model says (create a ticket, fill a form, route a request, call a tool), you need output that is machine-readable and a system that behaves sensibly when that output is wrong. Models are probabilistic components; reliability comes from the engineering around them.

Ask for structure, not prose

The first step is to stop parsing free text. Define the output you need as a schema and ask for it explicitly. JSON Schema is the common language here: it describes types, required fields, enums and nesting.

{
  "type": "object",
  "properties": {
    "category": { "type": "string", "enum": ["billing", "bug", "account", "other"] },
    "priority": { "type": "integer" },
    "summary": { "type": "string" }
  },
  "required": ["category", "priority", "summary"],
  "additionalProperties": false
}

There are roughly three levels of support, from weakest to strongest:

Prompt-only. You describe the format and hope. Works surprisingly often, fails in surprising ways.
JSON mode. The provider nudges output towards syntactically valid JSON, but not your particular shape.
Schema-constrained output. Major providers offer features that take your schema and constrain generation to match it, typically by compiling the schema into a grammar and disallowing tokens that would break it. Tool or function calling with strict schemas works the same way.

Use the strongest level your provider offers. It removes a whole class of parse errors and reduces retries.

A schema guarantees shape, not truth

Constrained decoding makes the structure right. It says nothing about whether the values are correct. A perfectly valid object can contain the wrong priority, a made-up order ID, or a summary of the wrong email.

There are also practical gaps even in the structural guarantee:

Providers support a subset of JSON Schema. Some constraints, such as numeric ranges or string lengths, may not be enforced during generation, so check the docs for your provider.
Output can be truncated when it hits the maximum token limit, leaving incomplete JSON. Always check the stop reason.
The model may refuse. Some APIs report refusals in a separate field rather than inside your schema.

Treat every model response as untrusted input, exactly as you would a form submitted by a stranger.

Validate everything

Validation happens in two layers:

Structural: parse it and validate it against the schema with a real validator (Pydantic, Zod, Ajv or similar), even if the provider promised compliance.
Semantic: check that referenced records exist, numbers add up, dates are plausible, quoted text actually appears in the source, values fall in allowed ranges, and the current user is allowed to do what the output requests.

OWASP lists Improper Output Handling among its top risks for LLM applications for a reason: model output passed unchecked into SQL, shell commands, HTML or downstream APIs becomes an injection vector.

Retries and fallbacks

Not every failure deserves a retry. Separate them:

Transient (timeouts, rate limits, server overload): retry with exponential backoff and jitter.
Validation failures: retry a small, fixed number of times, and include the specific error in the follow-up so the model can fix it.
Refusals and invalid requests: usually repeat on retry. Handle them as their own outcome.

When retries run out, have a planned fallback rather than an exception bubbling up to the user:

A different model or provider.
A simpler version of the task (extract two fields instead of ten).
A safe default or a “we couldn’t process this” state.
A human review queue.

Always cap total attempts and total time, so a bad input can’t burn money in a loop.

Idempotency and partial failure

Retries create a subtle problem: the same step might run twice. A timeout doesn’t tell you whether the action succeeded before the connection dropped. If a model-driven step has side effects (sending an email, charging a card, writing a record), make the side effect idempotent. Pass an idempotency key, or deduplicate on a stable request ID, so repeating the call is harmless.

For batch work, track status per item. Keep the successes, retry or quarantine the failures, and report clearly what happened. One malformed response in a thousand shouldn’t throw away the other 999, nor should it slip silently into your database.

Design so wrong outputs fail safely

Assume the model will sometimes be confidently wrong, and limit what that can cost:

Narrow the output space. Enums beat free text for decisions. Add an explicit other or unknown value, or constrained decoding will force a bad fit into a real category.
Least privilege. Give tools only the permissions the task needs. A summariser doesn’t need write access.
Confirm irreversible actions. Deletions, payments and external messages deserve a human or rules-based check.
Safe defaults. If validation fails, do nothing harmful: leave the ticket unrouted, not routed to the wrong team.
Log inputs, outputs and validation results so you can find patterns in failures and build evals from them.