Skip to content

Applied AI · September 2026

“Correct” is a product decision.

If a team cannot describe good behavior before launch, it will discover bad behavior through a real person.

In short

An AI system is only as good as the definition of “correct” someone wrote down before launch; without one, a team discovers bad behavior through a real person. When I built a conversational agent for a small business, the work that mattered was not the prompt. It was collecting real example conversations and deciding, for each awkward, skeptical, personal or unclear case, whether the system should ask a clarifying question, decline, or hand the conversation to a person. More than 100 automated checks and ten end-to-end scenarios made those choices visible before any customer met them. That evaluation is not paperwork: it turns an opinion into a shared standard, and it catches changes that look harmless on their own but shift the whole experience over a few messages. The model matters. The definition of “good enough to reach someone” matters more.

When I built a conversational agent for a small business, the tempting path was obvious: write a clever prompt, open the channel, and keep tweaking when something felt off. That is how a demo becomes a liability.

The more useful question was not “Does it sound good?” It was “What would correct behavior look like when the conversation becomes awkward, skeptical, personal, or unclear?”

Make the behavior inspectable.

We created examples of the conversations the system would face and defined what should happen: when it should ask a clarifying question, when it should decline, and when it should hand the conversation to a person. More than 100 automated checks and ten end-to-end scenarios made those choices visible before a customer had to discover the mistake.

The evaluation is not paperwork.

It is the place where a team turns an opinion into a shared standard. It catches changes that look harmless in isolation but change the whole experience over a few messages.

What I would do again

Start with real examples, write down the boundaries, test the uncomfortable cases, and give the system an exit to a person. The model matters. The definition of “good enough to reach someone” matters more.