THE WISDOM OF CROWDS

Posted on June 5, 2004 by Dave Pollard

James Surowiecki, the financial reporter for The New Yorker, has a new book out called The Wisdom of Crowds. It is essentially about decision-making, and the two most fatal (and ironically, almost opposite) judgement errors we make when making decisions:

Groupthink — the tendency of people’s judgement to coalesce around a single point of view (and not necessarily the most logical or wise one). You can witness this with a group that wants to believe something — the members tend to attach too much credibility to assertions that support that belief, and deny, or not even hear, assertions that contradict it.
Arrogance — the tendency of those with power and influence to believe their experience or gut feeling is more reliable, more credible, than the consensus of advisors, focus groups, or even customers. Some CEOs even pride themselves on their ability to make critical decisions with imperfect information, and feel that asking for such information is unnecessary, or even a sign of weak leadership.

The consequences of both types of flawed decision-making can be catastrophic, and constitute what I have called the cost of not knowing. I have cited Surowiecki before in this blog, and I have enormous respect for his sharp, challenging mind. His is no conventional wisdom — he’s imaginative, brilliantly logical and profoundly skeptical. Groupthink can be prevented, he says, by ensuring the group has intellectual diversity, independence (from each other) and is neither too centralized nor too decentralized. A group with these qualities is inherently more knowledgeable and its judgement more sophisticated, informed and reliable than any CEO or ‘subject matter expert’ that business, with its cult of leadership, tends to rely on for making critical decisions.

The greatest knowledge failures in recent history are preventative failures, sins of omission — notably 9/11, SARS, bird flu, BSE, the Great Blackout of ’03, Enron and other corporate rip-offs, and, in Canada, the Walkerton e coli deaths, not to mention many murders and other crimes. To what extent did our failure to prevent such disasters result from simply not having critical information, and to what extent were these failures aggravated by groupthink or individual arrogance? Was groupthink on a very large scale behind the dot com bubble, and is it happening again now? And is the dearth of genuine innovation in large organizations also partly attributable to these judgement errors?

Surowiecki tackles these questions thoroughly and provocatively, and proffers some steps we can take to employ the wisdom of crowds (or, more precisely, substantial, informed, well-designed decision-making and decision-influencing groups) to prevent groupthink or arrogance from producing knowledge failures. But Surowiecki’s most unsetlling lesson is that much of the time, these failures arise not (or at least not just) because we don’t know, but because we don’t want to know, or don’t care to know.

Painting above is Van Gogh’s ‘Pollard Birches’. Pollarding is the process of harvestiing the tops off trees without killing the tree, so that they grow back. Its earliest practitioners gave me my surname.

This entry was posted in Our Culture / Ourselves, Working Smarter. Bookmark the permalink.

5 Responses to THE WISDOM OF CROWDS

Piers Young says:

June 7, 2004 at 05:27

Hope you don’t mind me cluttering up your comments section, but EO Wilson put it quite well: “The ability of the brain to generate novel scenarios and settle on the most effective among them is called creativity. The persistent production of scenarios lacking reality and survival value is called insanity”So maybe we’re trying to balance the two here? Groupthink stifles creativity, but without groupthink, (and maybe with a little authority) creativity becomes insanity.
Michael says:

June 8, 2004 at 07:02

I loved Edward O Wilson’s book Consilience: The Unity of Knowledge. But I’m very interested in James Surowiecki’s book. I think they might be a great companion for one another.Consilience was written in what I call a quiet time, when the world seemed to be progressing toward commonality. Sadly, the US war on terror, which grew to include Iraq, seems to refute Wilson’s vision that the world will arrive at conscilience. Surowiecki, on the other hand, writes with a post 9/11 experience.Reading your review, I was reminded of a quote from Wilson’s thoughtful and astute book that ties in well with your thoughts:”Thanks to science and technology, access to factual knowledge of all kinds is rising exponentially while dropping in unit cost. It is destined to become global and democratic. Soon it will be available everywhere on television and computer screens. What then? The answer is clear: synthesis. We are drowning in information, while starving for wisdom. The world henceforth will be run by synthesizers, people able to put together the right information at the right time, think critically about it, and make important choices wisely.”If there is something lacking right now, and maybe Wilson can be credited for seeing this need back in 2000, it is the need for synthesizers who can analyze all the information and “make important choices wisely.” I have more quotes from Consilience in an August 28 post of last year if you are interested– http://blogs.salon.com/0002090/2003/08/28.htmlThanks for the review, Dave.
Dave Pollard says:

June 8, 2004 at 19:36

Piers: I think there’s a difference between groupthink (a herd mentality) and wild idealism — speaking as someone prone to both. I think groupthink is often a sign of ignorance or intellectual lazyness. What roots innovation in reality (where it belongs if it is to be of any value) is not that it can be successfully sold or marketed to lots of people, but that the idea stemmed from a known customer need in the first place. The difference between a wild and crazy product idea and a wildly successful one, is that the inventor of the latter listened carefully to someone other than himself.Michael: I’ve read Consilience and found it had a few brilliant turns of phrase but otherwise needed ‘more matter and less art’, as Shakespeare said. It’s interesting — I made a presentation to a conference of journalists a few years ago on The Future of The Media, and I said exactly what Wilson does: The media need to commoditize the facts and make them available to everyone free, and demonstrate their value (and get their revenue) by hiring subject matter experts to do analysis, investigative reporting, ground-breaking research, intelligence-gathering and other ‘synthesis’ activities, based on these facts, to tell us ‘what it all means’. At the time they said ‘that’s not our job’.
Michael says:

June 8, 2004 at 21:00

I find it fascinating that the media believed it was not their job to commoditize the information. I think of blogging and often think that there is more similation and analyzing (synthesis activities) going on between bloggers, which has progressively widened its reach from the mere blog community to the general public to the media community, which initially discredited it. Thanks for the information regarding Consilience.
Emily says:

June 9, 2004 at 22:43

Greetings from a fellow Pollard. Love your weblog, thanks!