Activity is not progress

AI makes it very easy to look productive. More work gets generated, more activity becomes visible, and everything appears to be moving.

Did the business move faster?

Two numbers, and they are not close

In one part of our work we measured the same thing two ways.

Measured as raw processing, the step was dramatically quicker than the way we had done it before. That is the number people quote. It is real, it is verifiable, and it is the one that appears in presentations.

Measured against the calendar, from the point where the work started to the point where it became useful to somebody, the improvement was a fraction of that.

The gap between the two numbers is not measurement error. It is the organisation.

Reviews, approvals and the queue

Reviews still happen, and they should. Somebody has to read the thing.

Decisions still require a person who is accountable for them, and that person has their own week.

And a recommendation only gets acted on once it has been right a few times in a row with somebody watching, which takes as long as it takes.

Work still queues behind other work. The step you made four times faster sits between two steps you did not, and the process moves at the pace of what you did not change.

Most of that is the control environment doing its job, not friction to be engineered away. An organisation that removed all of it to realise the larger number would have removed the reason anyone trusts the output.

Why we report the smaller number

It is less impressive and far more useful, for a simple reason. It describes what the organisation can actually do now that it could not do before. The larger number describes what one step in a process can do in isolation, which nobody experiences.

There is also a commercial argument. A supplier quoting a raw-processing multiple is quoting something that will not reproduce in your environment, because your environment has its own reviews, its own approvals and its own queues. When the promised improvement fails to appear, the supplier has not technically been wrong and the client has still been misled. That is a bad way to build a relationship that has to last years.

We would rather quote the number that survives contact with a real organisation.

We stopped optimising the step

Attention moved from the step to the sequence. Once you can see that the calendar number is held down by who is free to review and how long the queue is, those become the things worth working on, and neither is an AI problem.

In client work the calendar number is the one we manage. What a client experiences is the elapsed time from the first conversation to something they can use, and from each milestone demo to acceptance. So the engagement is built around the parts that usually queue. Requirements are confirmed and signed before the build starts, and our QA reviews each milestone before the client sees the demo, so acceptance is a decision the client can take in the room. How that runs is set out on AI-assisted engagement and delivery.

It also changed what we will say publicly. We are documenting the measurement behind both findings in this series, and we will publish the figures when we can state exactly what was measured, over what period, against what baseline. A number without that is decoration.

Where the four months leave us

We started this series by admitting that we could describe everything we had built and almost nothing about what it had changed. Working through it in order, the corrections were these. Fix the knowledge before choosing a model. Know where the processing physically happens. Extend what already works instead of starting over. Keep accountability exactly where it was. Then measure outcomes, not activity, and be willing to believe the result.

We will not be the most advanced shop in this market. We would like to be the one whose numbers still hold when a client measures them at their own end. We will publish ours when the method is written up, and it is worth asking your suppliers for theirs.