Define the grade before you pick the model

The demo is easy.

Someone pastes a messy ticket into a chat box. The model returns a clean summary. The room nods. A ticket appears: “ship AI for support.”

Six weeks later the same team is arguing about tone, missing context, wrong policy quotes, and whether the bot should ever send without a human. The model was never the hard part. The grade was missing.

We treat production AI as a workflow with an explicit pass/fail definition — not as a chat surface that “feels smart.” Model choice comes after the grade, the fail path, and the measurement plan exist in writing.

If you cannot measure the output, you cannot operate the feature.

What a grade actually is

A grade is not a star rating from a stakeholder who liked the demo.

A grade is a repeatable rule for a single task:

• Input class — what events enter the workflow (inbound email, PDF, CRM note, voice transcript).
• Allowed actions — summarize, classify, extract fields, draft a reply, propose a next step.
• Success criteria — what must be true for the output to ship or act without human edit.
• Hard fails — missing citation, out-of-policy claim, schema break, low confidence, empty retrieval.
• Owner — who changes the grade when the business changes.

Example for support triage:

Pass if: (1) category is one of the eight allowed labels, (2) urgency is P1–P4, (3) customer-visible summary is under 400 characters, (4) no refund promise language, (5) every factual claim either comes from the ticket body or is marked “needs agent.” Fail otherwise → human queue.

That paragraph is more valuable than three model bake-offs without it.

Why demos lie

Demos pick friendly inputs. Production is long-tail: angry customers, half-empty fields, PDFs that are scans, languages nobody tested, and policies that changed last Tuesday.

Demos also hide operating cost:

• Who reviews fails?
• How long can a case sit in the queue?
• What is logged for audit?
• What is the abort path when the provider is down?

If those answers are “we’ll see,” you do not have a feature. You have a prototype with a public URL.

Major platform guidance keeps circling the same point from different angles: tool use, agents, and production setups only hold when you separate task success from “the model said something fluent.” Anthropic’s eval and agent materials emphasize measuring performance on defined tasks and designing human-in-the-loop paths for production — not hoping free-form chat is enough. Google’s generative media guidance is blunt about post-processing and human evaluation when outputs can be wrong. The vendor details change; the need for a grade does not.

The minimum production shape

For internal automation and product features, we design five stations. Skip one and the system becomes a chat toy again.

1. Intake

Name the event. One workflow, one input contract. If marketing, support, and legal all dump into the same prompt, you will never know which grade failed.

2. Context

Retrieval, tools, or structured fields. If the task needs policy, tickets, or product data, context quality usually dominates model brand. Bad retrieval makes a strong model confidently wrong.

3. Model step

Generate under constraints: schema, length, tone, tool calls. Prefer structured outputs for anything that will be stored or acted on.

4. Grade gate

Automated checks first: schema validation, banned phrases, required fields, citation presence, confidence thresholds, simple rubrics, or a second-pass critic with a narrower job. The gate is code and policy, not a feeling.

5. Pass / fail destinations

• Pass → ship, store, notify, or act.
• Fail → human queue with the draft, the sources, and the failed checks visible.

No silent fail. No “best effort send.”

Build a tiny eval set before the UI

Before design polish, collect 30–50 real examples (anonymized) that match production messiness. Label the expected outcome by hand.

For each example, record:

• Expected category / fields / decision
• Known hard fails (poisoned or incomplete cases)
• Notes a human would care about

Run the workflow on that set every time you change prompt, model, retrieval, or tools. Track:

You do not need a research lab. You need a spreadsheet that refuses to lie.

When the set is too clean, the metrics stay green while production burns. Refresh the set with real fails from the queue every month.

Human-in-the-loop is a product surface

Review is not a temporary shame mode until the model “gets good.”

For anything customer-facing, financial, medical-adjacent, legal-adjacent, or reputation-sensitive, editable drafts with sources are the feature. Agents should see:

• What the model produced
• What context it used
• Which checks passed or failed
• One-click accept, edit, or reject with a reason

Those reject reasons become the next eval cases. That loop is how quality compounds.

Chat UIs that hide sources and checks train users to either over-trust or ignore the system. Neither is operable.

Intake → gate → archive. The middle station is where most demos pretend nothing can go wrong.

Model choice comes late on purpose

Once the grade and eval set exist, model swaps become boring experiments:

• Same workflow, same checks, new model or prompt
• Compare success, false passes, latency, cost
• Keep the winner only if the grade improves or cost drops without safety loss

Buying a “smarter” model without a grade is fashion. Swapping models under a fixed grade is engineering.

The same applies to build vs buy: off-the-shelf copilots are fine when the grade is light (draft email, summarize a meeting you already trust). Custom workflows earn their keep when your process, data, and fail paths are specific.

Where teams usually cut the wrong corner

• Skipping retrieval evaluation — they only grade the final paragraph, not whether the right policy was in context.
• Autonomous send on day one — no queue, no ownership, no rollback story.
• One mega-prompt for five jobs — triage, tone, refunds, and escalation all fighting in one blob.
• No schema — free text that cannot be validated, stored, or routed.
• Metrics that only track usage — “messages sent” is not success.

A discovery checklist we actually use

1. What single job gets a grade in v1?
2. What does pass look like in one paragraph?
3. What are five hard fails that must never auto-ship?
4. Who reviews fails, and in what tool?
5. What is logged for audit and debugging?
6. What is the offline / provider-down path?
7. Where do the first 40 eval cases come from?
8. What number makes us stop the rollout?

If step 2 and 4 are blank, stop model shopping.

Closing

Fluent text is cheap. Operable judgment is not.

Define the grade. Wire the fail path. Measure task success on ugly examples. Then pick a model — and keep the right to swap it when the grade says so.

If your AI initiative still starts with a chat widget and a hope, you are optimizing for the demo room. Build for the review queue. That is where the product either becomes infrastructure or becomes a slide that aged badly.

Scoping an AI workflow with real review and measurement? Start a project inquiry with the job you want graded and two examples of a hard fail. That is enough for a first architecture pass.