What I actually think about AI products.

I build with these systems every day and I run a fleet of them unattended for small businesses. That vantage point produces opinions that fit neither of the available camps, so here they are plainly.

The model is the easy part now.

Capability has stopped being the constraint for most products. Almost anything a small business needs from an AI feature is already within reach of a competent general model.

What is not solved, and nobody can buy it, is four questions:

• What the system does when it is unsure.
• Who it hands the case to.
• What it is forbidden from saying.
• How anyone finds out when it goes wrong.

That is product work, not machine learning. Teams that treat it as a model problem ship demos. Teams that treat it as a behaviour-design problem ship things people keep using.

“Correct” is a decision somebody has to make.

Every AI feature assumes a definition of good behaviour. If you have not written it down, the model has chosen one for you from the general shape of the internet.

That default is friendly, agreeable and confident. Three properties that are actively dangerous in a product where being wrong costs a person money or time.

In one system I built, most of the work was defining the refusals: what counts as a question already answered, what counts as somebody declining, what the system must never bring up again. The replies were straightforward. The boundaries were the product.

Confidence is a user-interface property, and we ship it badly.

What I find objectionable in most AI products is not that they are wrong. Every tool is sometimes wrong.

It is that they are equally fluent when right and when wrong. A human expert signals uncertainty constantly, in hedges and tone and pauses, and we calibrate against those signals without noticing.

Strip that signal out and you have built something that hands its risk to the user while sounding like it is doing them a favour.

A system that says “I am not sure, here is why, here is who would know” is a better product than one that is right slightly more often and never flinches.

Autonomy should be earned in stages, not granted at launch.

I do not think automation should be handed the keys on day one. And I do not think that is caution: I think it is a better experience.

• Let it observe.
• Let it propose somewhere safe.
• Let it do a bounded piece of the work, with somebody close by.
• Only then, let it act alone.

And even then, leave a clear road back to a person.

Users will give an automated system far more responsibility than you would expect, as long as it has shown them its work first. What they will not forgive is being surprised by how much authority it already had.

Most AI monitoring is theatre.

This is the lesson that cost me the most. I had dashboards full of green checks over systems that were producing nothing.

Jobs writing empty files. A gate no output could ever pass. A monitor recording a confident zero where the honest answer was “no data.” Every check was passing, and every check was measuring whether the process ran, not whether anything happened.

So I now hold two rules. A check you cannot make fail on purpose is decoration, not verification. And shipped is not working, one real person getting a real outcome on the other side is working, and nothing short of that counts, however green the pipeline.

Where I land

I am not worried about AI being too capable for the products we put it in. I am worried about it being too confident, too eager, and too poorly supervised inside products where somebody is trusting it with something that matters to them. Those are all design decisions. We can make better ones.