Hypothesis driven design: workshop part 1

Hypothesis driven design is a way of applying user research and experimentation to validate design choices. We, as a matter of course, user test all our digital products, but do we really know what we are testing and measuring?

The approach helps teams focus on finding solutions that are meaningful to users and to take calculated risks to move a product forward. Making sure design solutions deliver the outcomes we think they will, by using performance data to measure success or failure. And failure is fine, because we learn.

The key principles:

• • Outcome focused – not output focused
    The approach is not to create a deliverable or output (eg, we will create a sign-on form), but to create an outcome (eg, we want to increase the number of sign-ups).

• • Evidence based design – informed thinking
    The hypothesis statement is designed to help prove or disprove an assumption. Through user research and experimentation each hypothesis is tested to see whether we’ve achieved our desired outcomes.

• Collaborative design – shared understanding and joint ownership
  Collaborative design allows teams to create design concepts together. It helps build shared understanding and joint ownership of the design problem and solutions.

Workshop part 1

We are aware there are problems with the main navigation menu (mega-menu)  on The National Archives’ website. We’ve had feedback from users that the menu is often missed and Google analytics even suggests the click rate is low. For the workshop we decided to use the mega-menu to test run hypothesis driven design.

Declaring assumptions

The first step was to declare our assumptions as a group exercise. In preparation, we had Google analytics reports, user feedback, past attempts to address the issue and our own experiences of using the mega-menu. These helped use form our assumptions.

We broke up into smaller groups and we came up with three assumptions:

• • The mega-menu is hidden, if we make the top level items visible users will be more likely to use it.

• • The mega-menu is overwhelming, if we make it easier to digest then users will be more likely to find what they are looking for.

• The mega-menu content is not user focused, if we design the content to reflect user journeys, then users will be more likely to find what they are looking for.

How to measure

Next we needed to work out how we would validate each assumption. In each case we decided a combination of Google analytics (click rate) with user feedback will give us the metrics to prove or disprove our assumptions. Obviously, the initial stages of prototyping (paper or wireframe based) wouldn’t have the Google analytic tools, but we felt user feedback would be sufficient.

We also discussed that a high-fidelity prototype will benefit from AB testing, which would give us the quantitative evidence via Google analytics.

The hypothesis statement

The last step of the workshop was to create hypotheses from our assumptions. A hypothesis is designed in this format so assumptions are easier to test.

We wrote our hypotheses by using the following formula:

Changing [ __________ ]
to [ __________ ]
will lead to [ __________ ]
because [ __________ ],
and we’ll know that this is true/false when we see [ __________ ].

This is what the first assumption transformed into a hypothesis statement would look like:

Changing the current mega-menu
to a mega-menu where the top level menu items are visible
will lead to more users interacting with menu
because it’s no longer hidden behind a button,
and we’ll know that this is true/false when we see a higher click rate.

What’s next?