Reliability vs. Validity
by Paula Thornton
While a recent post on intent was successful in the dialog that ensued, I’m still trying to fully appreciate (there’s a lot to appreciate) the significance of Roger Martin’s explanation of the tension between Reliability and Validity (June 2007, IIT-ID conference). It makes me consider if the pursuit of intent and all the design research that goes with it, is effectively a means to strike a balance between the two.
I’ve been listening to this piece over and over again — each time additional depth is gained (including evaluating Martin’s style as to ‘how’ he tells the stories to his audience, and where he chooses to focus). It’s a topic he’d written about previously for Business Week, but his talk makes it a lot more meaningful and significant. He illustrates the significance of the tension by differentiating them:
Reliability
- Consistent, Repeatable, Predictable
- Certainteed Outcomes
- Validated on Past Data
- Measurable, Avoids Bias
- Limited Variables
Validity
- Diverse Variables (possibilities)
- Embraces Bias
- Validated by Future Events
- Outcomes Vary by Context
- Relies on Heuristics and Analogies
The challenge is that they’re inverse concepts. Moving toward one, requires minimizing the other. Does it mean that there’s not a middle? Not at all — indeed that’s the real goal. While Roger (probably for great ‘making a point’ purposes) puts business on one side and designers on another, I fundamentally believe that optimal design is actually in the middle (middle, not being a spacial thing, but somewhere other than one of the ends). Design is simply asking business to shift away from the thing that it’s intent on driving toward: science. But reality suggests that there is no ‘ideal’: there are too many extenuating circumstances (context). Therefore, the only way to optimize is to add a good dose of art. Design is what happens when you successfully find the optimal blend between science (the observable facts) from the art (celebrating the ‘unseen’).
Martin notes in his Business Week piece:
If a corporation wants to enjoy the benefits of design in its products, services, processes, or business models, it must go considerably beyond simply hiring designers or declaring itself design-oriented. The CEO must take responsibility for safeguarding validity. If the CEO doesn’t, the corporation’s natural inclination toward reliability will win out.
…certain corporate divisions — including powerful ones like finance — are more insulated from direct market pressures and can more easily slide into deep reliability.
Every CEO needs…to understand that he can’t let finance or any other division run roughshod over validity, or he’ll unknowingly drive design thinking completely out of his corporation. That’s why an additional task for the CEO is to act as the CVO — chief validity officer — in order to protect and nurture a design culture.
P&G’s CEO A.G. Lafley in a discussion with Roger Martin in 2008 is a living testament of this vigilence. He shares the tremendous effort it took to shift their business to a design culture. Between the many insightful examples of ‘how’ P&G shifted their business, by Claudia Kotchka, you can hear her repeatedly give credit to Lafley for being responsible for initiating and supporting the shift to design thinking at P&G.
Enterprise 2.0 is a shift to validity over reliability — not to replace one over the other, but to move toward a balance — bringing the yin to balance the yang, while celebrating the significance of both. Trying to implement a shift to validity while trying to hang onto the ways of reliability (without changing them radically) will lead to continued failure. As well, abandoning reliability will also fail.
Enterprise 2.0 is specifically set up to fail faster — make the future turn into the past sooner, but do so with smaller risk, smaller investments, smaller bits of focus. Or to use another 2.0 term — it’s the mashup between the two, making it fundamentally different than either one.
Circling back to the opening statement, I truly do belived that one means by which to bridge this gap is to bring the reliability artifacts of validity to the table — create the corresponding collection that was described as design research or customer insight, in the former piece.
















