Measuring personal growth honestly

Earlier this year a series on this site argued that AI programmes should be judged on outcomes, and that a supplier quoting a number should be able to say what was measured, over what period and against what baseline. We hold our own product to the same standard, and a wellbeing product is where that standard is hardest to meet.

UMOYA Zone, the personal development app we co-own with its founders, grew out of a coaching programme that asks people to rate themselves on a set of competencies when they start and again after twelve weeks. The difference between the two is meaningful to the people who take part. It is also a self-assessment, and it is worth being precise about what that means.

What self-assessment measures

A self-assessment tells you how a person sees themselves on the day they answer. That matters, because how people see their own direction and ability is much of what keeps them going, and a higher rating after twelve weeks often reflects something real.

It does not tell you, on its own, that behaviour changed. People answer differently at the end of a programme they chose to join. They have learned the vocabulary, they want the effort to have been worth it, and the second questionnaire is answered by someone who now knows what a good answer looks like. None of that makes the result false. It makes it one kind of evidence.

Say what the number is

The wellbeing market is full of figures that do not survive a second question: return-on-investment multiples drawn from studies of other programmes, productivity gains that were modelled rather than observed, testimonials standing in for measurement.

Our rule for UMOYA Zone is the same one we apply to delivery claims. Any figure we publish states what it measures, how it was collected, over what period and from how many people. A self-assessed improvement is described as self-assessed. A benchmark from someone else's research is labelled as someone else's. Where we have not measured something, we say so.

Better evidence is a design problem

The more useful question is how a product can produce stronger evidence without compromising privacy. A few approaches help.

Pair what people report with what they do. Goals set, goals completed and how often someone returns to the app are behaviours, and they can be counted on the device.

Measure more than twice. Short, frequent check-ins show a trend. Two measurements twelve weeks apart show a difference, which is easier to inflate.

For organisations, agree the outcome first. An employer offering the programme should decide in advance what would count as success, whether that is absence, retention or engagement survey results, and measure it independently of the app.

Earning the buyer's trust

Buyers are right to be sceptical of wellbeing claims, because so many of them are marketing. The way to earn trust is to be the supplier who explains exactly what a number means, even when the explanation makes it smaller.

It is the same position we have taken on AI delivery figures. More about the product is on the UMOYA Zone page.