Hypothesis-based roadmaps

If feature roadmaps are lies we tell ourselves, what’s better? Let’s talk about hypothesis-based roadmaps.

A hypothesis-based roadmap shows spans of time where our teams will be pursuing hypotheses in order to achieve outcomes, which map to our OKRs.

The hypotheses are in line with the strategies described in the Objectives, and the outcomes are the same outcomes represented as Key Results.

For example, let’s say that increasing customer activation by 10% is one of our KRs, and we have a hypothesis that we could achieve that if we added an onboarding experience.

We aren’t sure exactly how long it will take us to hit our +10% activation goal but based on our experience, we’ll guess 8 weeks: 5 weeks for the initial design → build → launch and 3 weeks for post-launch iteration.¹

I like hypothesis-based roadmaps for 5 reasons:

1. We’re focused on outcomes, not outputs. We are being clear on what the success metric is and the OKR it is connected to.

2. Culturally, as an organization, we’re admitting we have hypotheses about what we think will work and debates about our level of confidence in them, but we can never know for sure that any single feature will work.

3. We’re giving ourselves time to try different solutions to achieve our goal instead of overpromising that one perfect feature will get it right on the first try.

4. Just like with feature roadmaps, we’re still getting stakeholder buy-in on prioritization and resourcing, but now with clearer connection to OKRs.

5. And, we still have an artifact we can use to communicate to the rest of the organization and externally about what outcomes we’re focused on and the ideas we’re pursuing to achieve them.²

No roadmap approach is perfect — if it was possible to always know what will work, everyone would just do that and no startup would ever fail. With hypothesis-based roadmaps paired with OKRs, we can embrace the uncertainty while still providing direction on product strategy, prioritization, resourcing, and timing.

The Workshop

This is a newsletter-only section where I share a half-baked idea in hopes that y’all who are smarter than me can work it out with me.

An Amazon feature caught my eye yesterday when I was re-ordering shampoo.

A little history about me: I was briefly an engineer at Amazon on the Community team, which built user-generated content features. I worked on features that allowed customers to build lists of their favorite products. I then moved over to product management and was PM of Personalized Recommendations.

One of the areas where Amazon’s recommendations approach struggled was in fashion, because trends would come and go quickly, and new products would hit the market every season. There often wasn’t enough time for the “customers who bought this also bought” data to accumulate before the next new thing came along.

My theory for the last 15 years has been that influencers are the missing link. Influencers can do the work of discovering the next hot new trend, and an Amazon-style recommendations algorithm can focus on recommending you influencers you might like. So more like “Customers who bought this also bought from these influencers”.

Back to the Amazon feature. So I clicked on the second result and saw this:

Notice the “Follow” CTA in the top right. It gets at this hypothesis of mine from all those years ago, and I’m curious to know how well it’s doing. Good enough to earn real estate on the order confirmation page, at least!

1  I’m oversimplifying the product development process, of course. Obviously the hypotheses themselves are the result of customer discovery efforts — looking at data, talking to customers. And throughout this design → build → launch → iterate process, we’re using our validation toolkit (interviews, user testing of design prototypes, A/B testing, etc) reduce how long it takes us to realize we need to correct course, to fit more cycles of iteration into the same window of time.

2  Sometimes, the reality is that Big Important Customer demands feature X in order to renew. We’re still going to plop that on our roadmap, no different from a feature roadmap. But hopefully we can still do good Product work and uncover the underlying problem they are trying to solve when they demand feature X, to give us room to convert it into a hypothesis and therefore find alternative solutions.