Outcomes-based roadmapping

You can use an eraser on the drafting table or a sledgehammer on the construction site.

The challenge

In many organizations, roadmaps are feature-based; rigid six-month plans that focus explicitly on what by when. In effect, they’re just presentation slides to roll up to senior management—not something actionable or really, remotely based in reality, when it comes down to actually doing the work.

This is where organizational change becomes really necessary—because software is never really done and customer expectations aren’t static, either. The thinking is understandable when you fund projects instead of teams. The people writing the checks want to know what they’re going to get for their investment, and by when.

In this moment, as companies like Amazon release code every 11.7 seconds and Netflix literally thousands of times per day, taking months to build a business case to fund a project, more months (if not a year) to approve the funding, and then boxing that project into a specific set of deliverables by an immovable launch date is a recipe for irrelevance in the marketplace.

The old way: the feature factory. What by when. And also: what’s that other company doing and what do we need to do to “catch up?”

Featured-based roadmap
Feature-based roadmap
Feature parity competitive analysis

The problems with feature-based roadmaps are many. They assume that all of the answers exist already, for starters and that there’s nothing to be gained from research with actual people/customers. What inevitably happens, of course, is that we create these “roadmaps” that are little more than Waterfall-based feature factories. They’re often aimed at “feature parity” or “at-market parity” with competition, without consideration for rule number one: your business’s customers may not care what your competition is doing. They may represent fundamentally different kinds of people with entirely different needs and expectations. The business could “get to parity” with the competition only to find out that they’ve delivered exactly no value, because customers don’t care.

What’s also missing from the “feature parity” approach is any context into what research—if any—was done by said competition to arrive at the decision to implement X feature by Y date. Were they just following the herd when they did it too? Who knows…

By instituting research as a non-negotiable, upfront but continuous part of the process, we get to a place where we’re identifying and understanding the right problem to solve, evaluating the usefulness of the problem-solution fit and then getting to a place where we’ve “done it right”—and can measure it.

What we’re also doing at that point is not focusing so much on filling a leaky marketing bucket with new leads, but instead paying attention to the captive audience that’s already paying us money. The cost to acquire a new customer is always going to be higher than the cost to keep a current customer happy.

“It's time to redesign design.”

Alex Schleifer, VP of Design at AirBnB

The new approach

Companies are becoming very proficient at designing and building software and the field of UX is becoming more about setting processes, what we can bring to a company and the ability to distill priorities down to a really clear vision. This is now a competitive advantage.

I started advocating—loudly—for new thinking. Top down and bottom up. The organization had been calling some things “agile” for some time, but I started by pointing out the obvious: what we were doing was really waterfall in sprints.

Running the business document

I worked to craft a “running the business” document to share with senior leadership. This document highlighted current state + existing gaps and impact as well as desired future state + expected benefit. I focused our efforts on the operational issues we were experiencing, including not really behaving like an agile organization; staffing shortages; project instead of team funding; process inefficiencies. I worked to highlight what we needed in order to achieve our target state: a continuous, integrated stream of delivery. It was here that I first introduced the concept of outcomes over output.

I got pressure to create yet another what by when roadmap for the Experience Design team, focused on features to be implemented or improved in 2021. For this small team, four agile processes plus support for digital marketing initiatives all roll into one spot.

The established way of doing things needed to change. A big pain point with the old way was really that it also didn’t leave any buffer for anything to go sideways or—if we’re doing our jobs—to learn new things along the way and adapt and change course as necessary. We need to embrace learning as part of the process.

With a big project in motion and another pre-ordained delivery date in place, I sent out a bit of a manifesto, leaning again on the concept of “outcomes over output,” highlighting key passages and tying them back to how the organization could improve the way things were done for future projects. This information made its way to senior leadership and ultimately led to the CIO organization buying a copy of the book “Outcomes Over Output” for every employee. Progress!

Outcomes over output - email
Outcomes over output - email 2

For my team’s roadmap, I continued this thinking, creating an artifact to be shared as preamble to the actual roadmap, setting up what it is/isn’t, why I was doing it, and the approach.

Outcomes over output
Outcomes-based roadmap - definition and ground rules
5 signs of a bad UX roadmap

It’s not a train schedule, it’s a roadmap. It’s where we’re going, not when we’re going to get there.

The new roadmap was organized by high-level themes, broad time horizons and a consistent structure. This was also an opportunity to start to align research and relative confidence to each card and prioritize or reprioritize accordingly.

Understanding that organizational shifts of this magnitude aren’t going to happen overnight, the roadmap I created bridged the gap between where we had been and where I wanted to head via subthemes within each card (which is as close as this would get to what by when).

Out of a desire for this document to stand on its own, I also led with a disclaimer, notes on the structure of each card, and the design principles that guide the work of my team.

Experience design roadmap - full, now, next, future - outcomes over output

1. This section answers:

  • What is this
  • What are the high-level strategic goals?
  • Who owns it?
  • When it was last updated

2. This sets up the roadmap—what it is, how it’s structured and that it’s a living document, subject to evolution. Of note: we’re defining each theme here and how to create a new one. Beneficiary + need, business objectives, focus and confidence are clearly explained.

3. This is why we do what we do and the filter through which we run all of our work.

It’s a living reminder that this process works and provides value.

4. The roadmap is organized by broad time horizons: Now, Next, Future, and Future++.

5. The only time we get more specific with timing in a roadmap like this is in the “Now” column, because we have more clearly defined, likely in-progress work.

6. Each card is organized the same.

  • Beneficiary defines the recipient of the value our works provides
  • The need is the purpose of the UX work.
  • Business objective is what we’re trying to achieve with this work. This could be more knowledge, hitting revenue targets, new market insights, etc.
  • Who & What identifies the teams or people that will be responsible for the work, as well as anticipated deliverables.
  • Focus is how we align the work to broader objectives. In our case, there are operational aspects of this (process, tools, UX maturity) as well as key customer journeys that we’re working to improve upon
  • Confidence – how confident are we in this theme of work? Research-based or assumption-driven? The former is high confidence; the latter is not.

Results

  • Nielsen Norman Group asked to use this roadmap as an example of a great roadmap in their upcoming trainings on roadmapping.
  • The roadmap sparked further thoughtful conversation internally
  • Team vs. project funding acknowledged as a blocker at executive level
  • A copy of “Outcomes Over Output” was purchased and distributed to every team member in the CIO organization
  • Concerted efforts were made to move the organization more toward Agile methodologies
  • Tiger teams were created to solve specific challenges; QA embedded into continuous stream of delivery
  • Perception of UX began shifting to more of a critical need vs. UI layer