Validating model output before it reaches your database, and handling rejects
A legal-tech client asked us to add a clause-extraction feature to their existing contract review tool. The model reads a PDF, pulls out termination clauses, and returns structured JSON with a clause type, a severity score, and the extracted text. The first version wrote directly to Postgres. Within a week, the table had rows where `clause_type` was "N/A", `severity` was -1, and `extracted_text` contained the entire document because the model gave up on isolating the clause.
The model is not your user. It does not fill out a form with client-side validation. It produces text that looks like JSON until you look closely. Treating its output as untrusted input is the only sane posture.
We put a validation layer between the model response and the database. The layer runs three checks: schema validation, value-range validation, and content sanity. Schema catches missing keys and wrong types. Value-range catches the -1 severities. Content sanity catches the cases where `extracted_text` is longer than the source paragraph or where `clause_type` doesn't match any value in our controlled vocabulary.
We use a small set of validators built on Zod schemas. Here is the core of it:
import { z } from 'zod';
const ClauseSchema = z.object({
clause_type: z.enum(['termination', 'renewal', 'indemnity', 'confidentiality']),
severity: z.number().min(0).max(10),
extracted_text: z.string().min(10).max(2000),
page_number: z.number().int().positive(),
});
export function validateClauses(raw: unknown) {
const parsed = z.array(ClauseSchema).safeParse(raw);
if (!parsed.success) {
return { ok: false, errors: parsed.error.issues, raw };
}
return { ok: true, data: parsed.data };
}This runs before anything touches the database. The function returns the raw output alongside the errors, which matters more than you might expect.
The rejects go to a `review_queue` table with the same schema plus a few columns: the raw model output, the validation errors, the source document ID, and a status column that starts at `pending`. A human reviewer opens the queue, sees what the model produced, and either corrects the output manually or marks it as unparseable. The corrected row then goes into the main table. Nothing is lost.
We learned a few things the hard way. First, do not retry failed validations automatically. If the model produced `-1` for severity, calling it again with the same prompt usually gives you the same result, and now you have paid twice. Log the failure, surface it, move on.
Second, track reject rates per document type. When rejects spike for a specific contract template, that is usually a prompt problem, not a model problem. We found one template where every clause had a header line the model was interpreting as part of the clause type. A one-line instruction in the prompt fixed it.
Third, do not store raw model output in your main table. The review queue is where raw output belongs. Your main table should only hold validated, clean rows. If you mix them, you will eventually write a query that assumes the schema is correct and breaks on the rows that are not.
The cost question comes up. Running validation is cheap. Running the model is not. If your reject rate is 15%, you are paying for 100% of the inference and using 85% of it. That is not a reason to skip validation — it is a reason to improve your prompt and your schema until the reject rate drops. We aim for under 5%. Above that, the prompt needs work.
One thing we do not do is try to fix bad output programmatically. If the model returns `clause_type: "termination_clause"` instead of `"termination"`, you could strip the suffix and move on. But now you are maintaining a transformation layer that silently rewrites model output, and it will grow. Reject it, send it to the queue, and fix the prompt so the model stops doing it.
The validation layer is about 40 lines of code. The review queue UI is another 80. Together they are the difference between a feature you can trust and one you have to apologize for.