MIT Sloan Management Review

the manufactured evidence supports the organization’s best guesses about a complex and unpredictable decision environment.

What Can Be Done to Lessen
the Negative Impact of Decision-
Based Evidence Making?

One way for organizations to avoid negative decision-based evidence making is for decision makers to have a clear understanding of the different roles evidence can and should play in a decision process. This implies that decision makers within the organization have the flexibility to determine what constitutes legitimate justification of a particular decision. The difficulty that arises in practice is that organizations may have trouble differentiating between situations in which disconfirming evidence should be heeded and situations in which disconfirming evidence can safely be dismissed. Clearly, Enron was wrong to systematically ignore the formal analysis of its risk management team. However, the Aeron chair was an enormous commercial success, thereby vindicating Herman Miller’s decision to discount the negative reaction to the chair by focus groups.

Ultimately, the leadership of the organization must take responsibility for such judgments. To help, we provide the following guidelines:

1Understand the nature of the decision problem and assess the potential contribution of formal evidence to the quality of the decision process. There are many different types of problems — ranging from new product development to adoption of emerging technology standards — in which evidence based on historical data provides little insight. Decision makers should have the courage and the organizational support in such environments to make and justify a decision based on intuition, experience and consultation with others.

2Weigh the risks, costs and benefits of evidence when advocating an evidence-based approach to decision making. The costs should include not only the time required to collect evidence but any negative signals created by decision-based evidence making. For example, the decision by the architectural and engineering company to replace its conventional monitors with flat monitors was minor

in comparison to the company’s capital and operating budgets. The incremental benefit of a formal business case was outweighed by the losses in prestige and legitimacy caused by management’s initial insistence on bogus evidence.

3Differentiate between internal and external au- diences when engaging in decision-based evidence making. As noted above, there are situations in which evidence has significant ceremonial and signaling value. However, internal stakeholders (such as employees) typically have much better access to information than those outside of the organization. Consequently, internal audiences are seldom fooled by decision-based evidence making.

4Ensure that the objective evidence painstakingly gathered by your analysts is reflected more often than not in the decisions of the organization. If you must feed manufactured evidence to internal audiences, do so only rarely and sparingly. Enron provides an example of an organization in which a disregard for evidence and analysis became endemic.

There is mounting evidence in favor of evidence-based decision making in a wide range of organizational decision environments. The resistance of many managers to rational and analytical decision-making techniques is thus surprising. But what is troubling is that many managers who believe they have committed their organizations to evidence-based decision making (and have made hefty investments to back up this commitment) have committed instead to decision-based evidence making. Methodology alone cannot and should not replace managerial discretion or judgment. But, in much the same way that a streetlight can be used for illumination or support depending upon the need, greater understanding of the multiplicity of ways that evidence is used within organizations can lead to better decision making.

Peter M. Tingling is founder and CEO of Octothorpe Software Corp. Michael J. Brydon is an associate professor of management at Simon Fraser University. Comment on this article or contact the authors at smrfeedback@mit.edu.