NovAsia

A team needs more than success stories

Nova Asia AI assistant

Let’s explore this page together

A useful product review should examine decisions that did not work as expected, separate intent from outcome and leave behind a concrete change to the decision-making process.

This article reflects the named expert’s practical perspective. See NovAsia’s editorial policy for how material is prepared and reviewed.

Success stories are easy to keep. They have a clean shape: there was a problem, the team made a change and the result improved. Those stories can be useful, but they create a distorted internal memory if they are the only decisions worth documenting.

A team also needs to examine choices that looked sensible at the time and produced a weaker result than expected. The exercise is about improving the process, not assigning blame. The team needs to understand which assumption failed, which signal was missing and what should change before the next similar decision.

Start with the expectation that made the decision reasonable

Consider a hypothetical product change. A platform decides to reduce the number of properties shown first and surface a smaller set of highly relevant options. The reasoning is plausible: buyers should spend less time opening near-duplicate cards and more time comparing meaningful alternatives.

After launch, an awkward pattern appears. Some users ask why properties with a particular feature have disappeared. Others reopen the full catalogue and remove settings because they cannot see how the shortlist was created. The new experience is shorter, but not necessarily clearer.

That result does not prove that curated shortlists are a bad idea. It does tell the team to return to the expectation. What was the feature supposed to improve? How would a buyer notice the benefit? What evidence would indicate that the selection had become too aggressive?

If those questions were never answered before launch, the first process improvement is already visible: future changes need an explicit expectation and a failure condition.

Separate a good intention from a good implementation

The original intention may still be sound. Reducing repetitive work can help buyers. But the intention does not validate every mechanism used to achieve it.

Perhaps the problem was weak source data. Perhaps the shortlist logic was reasonable but unexplained. Perhaps an important hard requirement was treated as a soft preference. Perhaps the feature worked well for common searches and poorly for unusual ones.

This is where post-hoc storytelling becomes dangerous. A team can either say “personalisation failed” or “users do not like shorter lists” and move on. Both conclusions may be much broader than the evidence supports.

A useful review tries to preserve the limitation. It asks what population, what part of the catalogue and what behaviour were actually observed. That makes the lesson smaller, but far more reusable.

Do not turn the review into a trial

When a weak result becomes a search for the person who approved it, the team learns the wrong lesson: avoid ownership of uncertain decisions.

Product work always contains uncertainty. If people are punished for decisions that were reasonable on the evidence available, they will respond by delaying, over-documenting or presenting every choice as somebody else's instruction. None of that improves the product.

Accountability should mean being able to reconstruct the reasoning: what was known, what was assumed, what outcome was expected and what new evidence changed the view. Pretending every unsuccessful result was foreseeable is something else.

A calm review can still be demanding. It can identify missing checks, weak data, an ignored warning or an unjustified assumption. It simply keeps the criticism attached to the decision process rather than turning it into a character judgement.

The review should change something concrete

“Be more careful next time” is not an outcome. A good review leaves behind a modification to the way future decisions are made.

For the hypothetical shortlist feature, the team might decide that any mechanism which hides options must also explain the exclusion logic. Or it might require a visible way to return to the unfiltered set. Or it might define a rollback condition when hard requirements are being lost.

The exact rule depends on the product. The lesson only matters once it changes a later decision. Otherwise the next team will rediscover the same concern from scratch.

Sometimes the right conclusion is not removal. A feature may work well for one class of searches and poorly for another. Narrowing the scope can be a stronger response than declaring the whole idea either a success or a failure.

Success deserves the same scepticism

Looking seriously at unsuccessful decisions improves the way a team reads wins as well.

A favourable metric is never the whole product. More property-card opens can mean better discovery, or it can mean people are struggling to understand the list. Fewer support questions can mean clearer information, or it can mean the route to ask a question became harder to find.

Success stories also need a stated expectation, a relevant population and known limitations. Otherwise the team risks copying a result without copying the conditions that made it work.

I would rather maintain a history of decisions than a collection of victories. The useful record is simple: what we were trying to change, what we believed, what happened, what we could not conclude and what changed afterwards.

Some entries will describe strong ideas. Some will describe temporary compromises. Some will describe changes that should not be repeated in the same form. A product history is stronger when it includes decisions that did not work. It is evidence that the team is capable of learning without rewriting every past decision as either genius or negligence.

A team that records only success gradually becomes good at defending its previous choices. A team that can examine weaker decisions without turning the review into blame gets better at something more valuable: changing course while the cost of doing so is still manageable.

Choose the right specialist