Structured output modes fix most of the first five. Nothing fixes the last two except validation, business checks, and a safe fallback. That is why we treat the model as an untrusted input source, the same as a form submitted by a stranger.

What is the difference between JSON mode and structured outputs?

These two names get mixed up, and they give very different guarantees.

OpenAI, Anthropic, and Google Gemini all offer some form of schema-constrained output in 2026, and the Vercel AI SDK wraps them behind one function. Each provider supports a slightly different subset of JSON Schema, so keep your schemas simple: objects, strings, numbers, booleans, arrays, and enums. Avoid deep unions and clever regex patterns in the schema you send to the model. Put those checks in your own validation instead.

How do you define the schema once and use it everywhere?

Write one zod schema and derive everything from it: the TypeScript type, the schema you send to the model, and the server-side validator. Here is an example for a support inbox that classifies incoming tickets.

// lib/ai/ticket-schema.ts
import { z } from "zod";

export const TicketTriage = z.object({
  category: z.enum(["billing", "bug", "feature_request", "account", "other"]),
  priority: z.enum(["low", "normal", "urgent"]),
  summary: z.string().min(1).max(280),
  customerEmail: z.string().email().nullable(),
  orderId: z.string().nullable(),
  needsHuman: z.boolean(),
});

export type TicketTriage = z.infer<typeof TicketTriage>;

Three habits that save time later:

1. Use nullable instead of optional for "might be missing" fields. Many structured output modes require every property to be present. A field that is always there, but can be null, is easier for the model and for your code.
2. Use enums for anything your code branches on. If a switch statement depends on it, it should be an enum, not free text.
3. Put length limits in zod. A summary that is 4,000 characters long is valid JSON and still breaks your UI.

How do you call the model with structured output in Next.js?

Keep the call on the server, in a route handler, Server Action, or background job. Never call the model from the browser with your API key. This example uses the Vercel AI SDK, which accepts the zod schema directly.

// lib/ai/triage.ts
import "server-only";
import { generateObject } from "ai";
import { openai } from "@ai-sdk/openai";
import { TicketTriage } from "./ticket-schema";

export async function callTriageModel(ticketText: string) {
  const { object } = await generateObject({
    model: openai("gpt-4o-mini"), // pick the model your evals support
    schema: TicketTriage,
    system:
      "You triage support tickets for a SaaS product. " +
      "Use only facts present in the ticket. " +
      "If a value is not in the ticket, return null. Never guess an order id or email.",
    prompt: ticketText,
    maxOutputTokens: 400,
  });
  return object;
}

If you use a provider SDK directly, the idea is the same: convert the zod schema to JSON Schema (the OpenAI Node SDK ships a zod helper for this, and zod 4 has a built-in JSON Schema export), send it as the response format, and parse the reply. Function names differ between SDK versions, so check the docs for the version in your package.json.

How do you validate, retry, and fall back safely?

Even with structured outputs, validate on your side. The SDK may already throw on a schema mismatch, but your own safeParse also runs the checks the provider could not enforce, and it gives you one place to handle errors. Then retry once with the error message, and if that fails too, fall back to something safe.

// lib/ai/triage-safe.ts
import "server-only";
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";
import { TicketTriage } from "./ticket-schema";

const FALLBACK: TicketTriage = {
  category: "other",
  priority: "normal",
  summary: "Needs manual review",
  customerEmail: null,
  orderId: null,
  needsHuman: true,
};

function extractJson(text: string): unknown {
  // Strip code fences or chatty prefixes, then parse the first JSON object.
  const start = text.indexOf("{");
  const end = text.lastIndexOf("}");
  if (start === -1 || end <= start) throw new Error("No JSON object found");
  return JSON.parse(text.slice(start, end + 1));
}

export async function triageTicket(ticketText: string): Promise<{
  result: TicketTriage;
  source: "model" | "retry" | "fallback";
}> {
  let lastError = "";

  for (let attempt = 0; attempt < 2; attempt++) {
    const repairNote = attempt === 0 ? "" :
      `\n\nYour previous reply was invalid: ${lastError}\nReturn only corrected JSON.`;

    try {
      const { text, finishReason } = await generateText({
        model: openai("gpt-4o-mini"),
        system:
          "Return only a JSON object with keys category, priority, summary, " +
          "customerEmail, orderId, needsHuman. Use null for unknown values.",
        prompt: ticketText + repairNote,
        maxOutputTokens: 400,
      });

      if (finishReason === "length") {
        lastError = "the reply was cut off; keep the summary under 280 characters";
        continue;
      }

      const parsed = TicketTriage.safeParse(extractJson(text));
      if (parsed.success) {
        return { result: parsed.data, source: attempt === 0 ? "model" : "retry" };
      }
      lastError = parsed.error.issues
        .map((i) => `${i.path.join(".")}: ${i.message}`)
        .join("; ");
    } catch (err) {
      lastError = err instanceof Error ? err.message : "unknown error";
    }
  }

  console.warn("triage fallback", { lastError });
  return { result: FALLBACK, source: "fallback" };
}

This version uses plain text generation so you can see every step. In production you would usually combine both: generateObject with the schema for the first attempt, and this safeParse, retry, and fallback wrapper around it. The rules we follow:

1. Retry once, not forever. One repair attempt with the exact validation error fixes most bad replies. A loop of five retries just multiplies cost and latency.
2. Treat a cut-off reply as its own error. If the finish reason says the output hit the token limit, the fix is a shorter output or a higher limit, not a JSON repair.
3. Make the fallback boring and safe. Here it routes the ticket to a person. It never invents a category the business acts on.
4. Log the source. Recording "model", "retry", or "fallback" per call tells you how often validation saves you, which is a number you can track weekly.

How do you catch output that is valid but wrong?

Schema validation proves the shape. It does not prove the facts. Add business checks after safeParse for anything the model could invent:

// After a successful parse
if (result.orderId) {
  const order = await db.order.findFirst({
    where: { id: result.orderId, accountId: session.accountId },
  });
  if (!order) result = { ...result, orderId: null, needsHuman: true };
}

if (result.customerEmail && !ticketText.includes(result.customerEmail)) {
  // The model "found" an email that is not in the ticket. Drop it.
  result = { ...result, customerEmail: null };
}

The pattern is simple: any id, email, amount, or date that your code will act on must be checked against your database or the original input. If it does not match, drop it and send the item to a person.

How do you test structured output before launch?

Put 30 to 100 real examples in a JSONL file in the repo, with personal data removed, and run them through the same triageTicket function on every prompt or model change. Track three numbers:

We covered a lightweight version of this setup in our guide to adding evals to an LLM feature before launch. Structured output is the easiest place to start, because most checks are plain code: did it parse, is the enum valid, is the id real.

What is the short checklist?

1. One zod schema per AI output, with enums, nullable fields, and length limits.
2. Structured output mode on, using the simplest schema the provider supports.
3. Model calls only on the server, with a max output token limit.
4. safeParse on every response, even when the SDK claims a guarantee.
5. One retry with the validation error, then a safe fallback that routes to a person.
6. Business checks for every id, email, amount, or date your code acts on.
7. A small JSONL test set run on every prompt or model change, with first-try validity and fallback rate logged.

If you want help building AI features like this into a web or mobile product, Luminix Studio builds apps and AI automations for startups worldwide. You can reach us at luminixstudio.com.