Not all decisions incorporate evidence in the same way, or intend to marshal it toward the same end. This chart shows three roles that
evidence can play, depending on whether the aim is to make, inform or support a decision.
ROLE OF EVIDENCE IN DECISION MAKING DESCRIP TION ARCHE TYPAL DECISION
Make decision Evidence forms the basis of
the decision
Facilities location
RISKS
Poor decisions due to
misspecified models
Evidence
Decision
Process
Decision
Inform decision
Evidence
Intuition,
Experience,
Bargaining, etc.
Evidence is one of several
inputs to the decision
process
Diagnosis, strategic
planning
Decision
Process
Decision
Mismatch between
evidence and other inputs
requires shift to “make”
or “support” roles
Support decision
Intuition,
Experience,
Bargaining, etc.
Evidence is created to
support a decision made
using other inputs
New product development,
technology adoption
Decision
Process
Decision
Evidence
Demoralization of analysts;
poor decisions due to
decision biases and false
consensus
techniques are evaluated against impressionistic
and intuitive judgments of experts show that algorithmic techniques often provide better results, even
in unstructured decision contexts such as granting
parole or predicting job satisfaction.
The primary risk of making decisions by relying
exclusively on hard evidence is that the algorithms and
models used to transform evidence into a decision
provide an incomplete or misleading representation of
reality. There are many examples in finance, for example, where the models used by traders misspecified
important real-world dependencies and risks. As the
collapse of companies such as Long-Term Capital
Management LP and The Bear Stearns Companies Inc.
attests, the organizational downside created by a commitment to misspecified models can be significant.
Inform a Decision
Evidence is used to inform a decision whenever the
decision process combines hard, objective facts with
qualitative inputs, such as intuition or bargaining
with other stakeholders. The role of evidence in informing decisions is thus akin to due diligence. For
example, in a succession planning decision, objective
evidence about the candidates’ past performance in
managerial roles is often an important input to the
decision process. However, subjective, impressionistic information is typically combined with hard
evidence when making the final decision. Bargaining,
expressions of power and other organizational elements that do not fit within the orthodox model of
rational choice may also enter into such decisions.
The evidence-based inputs to the decision
process either confirm or disconfirm the decision
makers’ initial subjective beliefs and preferences. If
the evidence is confirmatory, decision makers can
move forward, confident that they have “the numbers” required to support their choice. However, a
dilemma arises if the evidence disconfirms the initial subjective decision. The decision makers must
either trust the evidence (in which case they have
implicitly switched to the make mode described
above) or side with their gut.
When asked, managers report that they routinely
grant evidence priority over their impressionistic
assessments. For example, in a survey of the capital
budgeting practices of large American companies,
45% of respondents said they would reject a capital
investment opportunity that had a favorable “
strategic analysis” if the net present value (NPV) of the
opportunity was negative. However, as illustrated in
the example of the enterprise e-mail system, executives often provide analysts with subtle (and, in
some cases, unsubtle) signals regarding the desired
outcome of a formal, evidence-based analysis. Our
research shows that senior decision makers are
often unaware that evidence has been shaped by
subordinates to conform to the perceptions of
