Decision-Making: Weighing the Evidence

“There is no evidence that one can become expert in something as broad as decision-making, policy, or strategy…or perhaps even management. … Large groups of diverse individuals will make more intelligent decisions than even the most skilled decision-maker.”

The implication of this is that individual business executives, expert consultants, investment analysts, learned doctors and heads of state are not competent to make important decisions related to cognitive, coordination or cooperation problems, and should always defer to the collective wisdom of large diverse groups when such problems arise.

Surowiecki identifies five types of decisions that qualified (reasonably informed, diverse, independent) ‘crowds’ are especially competent at:

• ascertaining (all the) pertinent facts surrounding an issue
• predicting outcomes
• making a decision among a discrete set or finite range of alternatives
• determining an optimal process to follow (in simple or complicated situations, but not complex ones)
• assessing causality (in simple or complicated situations, but not complex ones)

Crowds are not particularly good at imagining solutions to problems, or knowing which tools and methods to use to solve them ñ creative groups and individuals are better at these elements of decision-making.

Most decisions involve some aspects that are best done by a substantial diverse crowd, and other aspects that are best handled by small creative groups or individuals. The chart above right shows how these aspects could be combined to make an overall decision.

We make decisions based on a judgemental synthesis of what we ‘know’ intellectually, perceptually, emotionally, and intuitively. That knowledge may be direct, from personal experience, or indirect, from what we’ve read or been told by someone whose judgement we trust.

Indigenous peoples tend to make decisions more holistically, rather than biasing their decisions in favour of intellectual knowledge alone. They are more tentative in their judgements and try to allow more time for all knowledge, including that which is subconscious, to be considered and integrated. They will place great weight on the judgements of those they trust, but ultimately each individual will be trusted (given the authority) and expected (given the responsibility) to make any decision that affects them alone, without having to justify it to others. When the decision affects others, they will make the decision-making process a collective one, and will allow those who disagree with the decision to opt out of it (provided that does not adversely affect the welfare of others).

In today’s crowded and massively interdependent world, nearly every decision affects many people, and we rarely have either the responsibility or the authority for making decisions alone. And, as Surowiecki points out, when we do have the personal authority to make decisions for others (because of our position atop the hierarchy), we are likely to do so badly. So many important decisions either are, or should be, collective decisions.

If you have watched decisions being made by a collective, you can see how this process can go terribly wrong. A particular vulnerability of collectives  is the all-too-human propensity to be grateful that someone else is taking on the difficult work of running the group and making the tough decisions. The politics of collective decision-making often comes down to the grabbing of authority and the shrugging off of responsibility, until the decisions end up being made by a small faction (or even an individual) willing to accept (most of) the responsibility for the decision as long as they have (substantially all of) the authority. Itís a copping-out process that allows the power-hungry and indifferent to collude and bully the remainder, and this invariably leads to sub-optimal decisions.

Collective decisions also tend to give greater weight to intellectual knowledge than perceptual, emotional, and intuitive knowledge, because of its perceived ‘objectivity’ (and hence simpler process of achieving understanding and agreement on its veracity). As a result, collective decisions, even those we have acceded to, often leave us feeling uneasy, since we feel (emotionally or intuitively, but in ways we can’t readily articulate) that the decision-making process was incomplete and flawed.

Nowhere is this prejudice for intellectual knowledge more evident than in the new field of “evidence-based decision-making’. Evidence (=what can be seen or understood easily) will not allow for the introduction of emotional or instinctive judgement, no matter how valid it may be. It can be applied tyrannically to overrule experience with the greater weight of ’empirical data’ and so-called ‘best practices’, even when the result may be catastrophic. It can also be applied helpfully to overrule pigheadedness and short-sightedness.

So, for example, the egomaniacal doctor who ‘knows’ that prescribing x is always the best solution for every patient can be reined in and made more responsible when ‘the evidence shows’ that y is usually a better prescription. But so too can the professional whose insight into individual differences may cause him/her to occasionally prescribe z because in a rare few other cases in his/her experience, with a certain combination of symptoms (too few to constitute substantive ‘evidence’), z proved to be a better answer.

You can be sure the lawyers will weigh in consistently on the side of treatment y, to the advantage of the patients of the egomaniacal doctor doling out x, and to the detriment of the patients of the wise doctor, patients for whom z is a better prescription. The result is that every patient with the general symptoms will get prescribed y, and will be unable to sue even if the doctor knew z or x would have been a better prescription. And every patient prescribed z or x will be able to sue their doctor, even if that was the best prescription in their individual case. We should all have learned by now that in complex (human and ecological) systems there are no best practices ñ every situation is different, and the ‘best practice’ for dealing with it is unique.

Evidence is, after all, a loaded word. What we call evidence is the data that we personally find useful in a particular context, and in complex systems we all have different contexts and perspectives, so we will never agree on what is appropriate evidence, or on what the evidence ‘means’. For example, when a conservative politician reads crime news, he sees ‘evidence’ supporting a decision for more law and order. The liberal considering the same data will see ‘evidence’ for a decision to improve social welfare, improve education and strengthen gun control.

The best sort of evidence is first-hand observation, since the context is harder to omit or misconstrue, but even it must be filtered through our personal worldviews, worldviews that are inherently subjective and biased. And sometimes that’s a good thing: unemotional, insensitive, unintuitive decisions can be colossally bad ones.

Stories are the best second-hand evidence, but we all know how subversive they can be, by selective omission and emphasis. As for lesser evidence, lies, damned lies and statistics, as they say. You can make the numbers say whatever you want them to say.

So there are six major obstacles that interfere with our ability to make good collective decisions:

1. Exclusion: Not involving all of the necessary or appropriate people in the decision-making process (e.g. excluding the patient from medical decisions).
2. Political Interference: Precluding fair and equal participation of all members of the collective group (e.g. by bullying, back-room deals, power games and abrogation of responsibility).
3. Personal Bias: In how our worldview filters and interprets the ëevidenceí.
4. Bias Towards Intellectual Knowledge: Because what we know holistically is subtler and more complex than we can ever express in language and hence communicate persuasively to others.
5. Context-Free Evidence: Some of the best ‘evidence’ can only be properly internalized when it is observed directly, or at least understood through exceptionally-crafted stories; without such context, mere data loses much of its decision-making value.
6. Consensus-Building Incompetence: Few of us have learned the difficult skill of engaging other points of view, synthesizing, working through differences instead of glossing them over, trivializing them, or over-compromising ñ work that is necessary to achieve true consensus. Most of us have little patience for this process, which can take enormous time and energy.

The solution, as with most complex problems, is to discover and follow good working models. Open Space and its invitation process can help address the problem of exclusion. So can simple humility: find the politician and the doctor and the planner who consult genuinely with those affected by their decisions before they are made. My experience has been that informed groups with good facilitators can minimize political interference.

Only good critical thinking skills, and patience, can begin to overcome our most dangerous personal biases: Things are the way they are for a reason, and we are oh so quick to judge and oversimplify what that reason is. Knowing people who do think holistically (hard to find in the corporate world, alas) can help you acquire this capacity yourself, reducing your bias toward ‘objective’, rational knowledge.

Getting out of your house and office and seeing things first hand will increase your appreciation of the complexity of issues and your insistence on creating a context of understanding before jumping to conclusions (and decisions). So will learning to tell better stories. And the only solution for improving your consensus-building competence is practice. I know a dozen people who seem to be able to achieve remarkable consensus, but only two of them (both women) do so genuinely, and it’s a skill that did not come easily (the other ten are just great at sweeping differences under the rug, which inevitably resurface later with greater virulence).

So, like motherhood and apple pie, Evidence-Based Decision-Making is a great idea. But what evidence, according to whose interpretation, arrived at how, in what context, assessed through what decision-making process,and including whom?

The devil, as always, is in the details.