Recently I reported some corporate backlash against James Surowiecki’s ideas in The Wisdom of Crowds and its message that, if organizations were smart, they could dump a lot of expensive senior executive and consultant/expert baggage and get better decisions by putting critical questions collectively to employees, customers and appropriately ‘qualified’ elements of the ‘general public’. The objections point out that ‘crowds’ are not great at doing everything. But that’s exactly Surowiecki’s point: The very things that crowds are good at are precisely those things that executives, consultants and experts pride themselves on doing. A couple of readers asked me if I could distill Surowiecki’s arguments into some kind of decision tree to decide who is best to make decisions. This is my response. We need to start by looking at who the alternative decision-makers are, and what knowledge, skills and talents they offer that are relevant to the decision-making process:
These would be, I think, Surowiecki’s assessments, based on the research in his book. They are also mine, based on thirty years of varied business experience. The reason why executives, consultants, senior managers and other experts don’t rate ‘high’ in any of the six categories of relevant capacities for decision making is (a) they are usually individuals, and can only know as much as any busy individual can know, (b) in the case of outside experts, they lack experience/context actually working for the organization, and (c) in large organizations executives are paradoxically sheltered from awareness of problems due to the “bad news doesn’t travel upwards” (because “they shoot the messenger”) information behaviour that is endemic to our society. For those who haven’t read The Wisdom of Crowds, a ‘qualified crowd’ is one that is (i) intellectually diverse, (ii) independent and objective, (iii) each member has access to unique knowledge, (iv) each member is basically informed, and (v) each member is appreciative of (cares about) the problem or decision at hand. Surowiecki identifies five things that qualified crowds can — if asked appropriately — be very good at:
In all except the first type, the crowd must be given a set or range of alternatives to choose from, and, when they are, Surowiecki says, the ‘errors’ in judgement tend to cancel each other out, so that the crowd’s consensus tends to be consistently better than that of executives, consultants and other experts. If you don’t buy this, you’ll have to read the book — his argument is compelling and well-substantiated (it’s also intuitively sensible). In situations of the first type, ascertaining (all the) pertinent facts surrounding an issue, the crowd is contributing more collective knowledge than any small group of ‘experts’ could hope to have, and are ‘better’ at doing this by sheer dint of numbers. So what happens in the real world when important decisions must be made? In my experience, this is the typical process:
If you accept the capacities in the chart above, the result of this ‘business as usual’ process is clearly sub-optimal. Consultants and other outside experts bring precisely the capacities that the executives already have, and none of the ones they lack. Involving researchers and creatives will improve the quality of the decision somewhat, but not as much as involving the crowd. And that assumes that nothing gets lost in the ‘translation’ of knowledge between the researchers, creative people and executives. What’s worse, many researchers and creative people will tell the executives what they want to hear, not necessarily the truth — they lack the independence and objectivity that ‘qualify’ a crowd. Here by contrast is the optimal process, for complicated (not complex) problems:
This learn-analyze-imagine-assess-decide-on-action process involves each group of stakeholders doing what they do best. If there are appropriate incentives for the crowd (and sometimes that’s as simple as recognition and thanks), this process need not be cumbersome, and to some extent it can be automated (members of the ‘crowd’ can to some extent self-qualify by going through an online qualification survey, and step 3 can also be done entirely online). It is course frightening to executives, because it reveals their true, limited value in the decision-making process. In fact just about anyone can perform the three steps above (they are mostly administrative and facilitative), bringing into question the need for highly-paid executives, and a hierarchical decision-making organizational structure, at all. So this approach is clearly more amenable to egalitarian, non-hierarchical organizations. It’s also bad news for the consultants and outside experts — they aren’t needed in the process at all. Here, from an earlier article, are 25 business problems that such an approach might solve:
I said that the above process is optimal for complicated problems. What about complex problems, like these?:
The process for such problems must of necessity be emergent, rather than prescriptive as for merely complicated problems. Such problems do not lend themselves to (anywhere near) ‘complete’ knowledge, rigorous analysis, determination of clear causality, or predictability. In fact, such problems don’t have ‘solutions’ per se at all. What can emerge is a collective understanding sufficient to allow all of the participants in the process to contribute knowledgeably, positively and responsibly to addressing the problem in self-organized adaptive ways, individually and collectively, in the context of their own lives and work. This process is essentially the same process that indigenous cultures have used for millennia to address such problems, and the same process used by ‘complex system’ methodologies like Open Space:
This learn-explore-imagine-converse-emerge-let-self-decide-on-action process is structurally similar but significantly different in methodology and responsibility than that outlined above for complicated problems. Each process respects the different characteristics of the problem/issue and appreciates the need for a different approach to it. What I have observed over the past few years is encouraging: Organizations with enlightened leadership (and leaders with modest egos) appear to intuitively appreciate the limitations of the ‘boss-decides-in-a-vacuum’ approach to management, and are starting to involve line staff and customers more in at least the information-gathering (step 1) part of the decision-making process. This isn’t tapping the wisdom of crowds but it’s a big step in the right direction. Some organizations are even beginning to realize that prescriptive ‘solutions’ to complex problems (and generally all problems that involve human behaviour and interaction are complex) don’t work, and are starting to devolve authority and responsibility to individuals on the front line to make more tactical decisions. I’ve seen less willingness to involve creative minds in organizations in imagining alternative solutions, to actually devolve decision-making authority to crowds, or to give individuals decentralized authority and responsibility to make strategic decisions. But perhaps as some brave organizations start to do this, successfully, others will follow. Laterally-thinking readers will probably have realized that these processes aren’t limited to business or even organizational contexts. Think about its application to problems in a family context, where the larger community is the ‘crowd’ (if you’re lucky enough to live in a community whose members know and care enough about each other to qualify as a crowd under Surowiecki’s five criteria) — and you’ll understand what ‘ittakes a village to raise a child’ could really mean. |
Navigation
Collapsniks
Albert Bates (US)
Andrew Nikiforuk (CA)
Brutus (US)
Carolyn Baker (US)*
Catherine Ingram (US)
Chris Hedges (US)
Dahr Jamail (US)
Dean Spillane-Walker (US)*
Derrick Jensen (US)
Dougald & Paul (IE/SE)*
Erik Michaels (US)
Gail Tverberg (US)
Guy McPherson (US)
Honest Sorcerer
Janaia & Robin (US)*
Jem Bendell (UK)
Mari Werner
Michael Dowd (US)*
Nate Hagens (US)
Paul Heft (US)*
Post Carbon Inst. (US)
Resilience (US)
Richard Heinberg (US)
Robert Jensen (US)
Roy Scranton (US)
Sam Mitchell (US)
Tim Morgan (UK)
Tim Watkins (UK)
Umair Haque (UK)
William Rees (CA)
XrayMike (AU)
Radical Non-Duality
Tony Parsons
Jim Newman
Tim Cliss
Andreas Müller
Kenneth Madden
Emerson Lim
Nancy Neithercut
Rosemarijn Roes
Frank McCaughey
Clare Cherikoff
Ere Parek, Izzy Cloke, Zabi AmaniEssential Reading
Archive by Category
My Bio, Contact Info, Signature Posts
About the Author (2023)
My Circles
E-mail me
--- My Best 200 Posts, 2003-22 by category, from newest to oldest ---
Collapse Watch:
Hope — On the Balance of Probabilities
The Caste War for the Dregs
Recuperation, Accommodation, Resilience
How Do We Teach the Critical Skills
Collapse Not Apocalypse
Effective Activism
'Making Sense of the World' Reading List
Notes From the Rising Dark
What is Exponential Decay
Collapse: Slowly Then Suddenly
Slouching Towards Bethlehem
Making Sense of Who We Are
What Would Net-Zero Emissions Look Like?
Post Collapse with Michael Dowd (video)
Why Economic Collapse Will Precede Climate Collapse
Being Adaptable: A Reminder List
A Culture of Fear
What Will It Take?
A Future Without Us
Dean Walker Interview (video)
The Mushroom at the End of the World
What Would It Take To Live Sustainably?
The New Political Map (Poster)
Beyond Belief
Complexity and Collapse
Requiem for a Species
Civilization Disease
What a Desolated Earth Looks Like
If We Had a Better Story...
Giving Up on Environmentalism
The Hard Part is Finding People Who Care
Going Vegan
The Dark & Gathering Sameness of the World
The End of Philosophy
A Short History of Progress
The Boiling Frog
Our Culture / Ourselves:
A CoVid-19 Recap
What It Means to be Human
A Culture Built on Wrong Models
Understanding Conservatives
Our Unique Capacity for Hatred
Not Meant to Govern Each Other
The Humanist Trap
Credulous
Amazing What People Get Used To
My Reluctant Misanthropy
The Dawn of Everything
Species Shame
Why Misinformation Doesn't Work
The Lab-Leak Hypothesis
The Right to Die
CoVid-19: Go for Zero
Pollard's Laws
On Caste
The Process of Self-Organization
The Tragic Spread of Misinformation
A Better Way to Work
The Needs of the Moment
Ask Yourself This
What to Believe Now?
Rogue Primate
Conversation & Silence
The Language of Our Eyes
True Story
May I Ask a Question?
Cultural Acedia: When We Can No Longer Care
Useless Advice
Several Short Sentences About Learning
Why I Don't Want to Hear Your Story
A Harvest of Myths
The Qualities of a Great Story
The Trouble With Stories
A Model of Identity & Community
Not Ready to Do What's Needed
A Culture of Dependence
So What's Next
Ten Things to Do When You're Feeling Hopeless
No Use to the World Broken
Living in Another World
Does Language Restrict What We Can Think?
The Value of Conversation Manifesto Nobody Knows Anything
If I Only Had 37 Days
The Only Life We Know
A Long Way Down
No Noble Savages
Figments of Reality
Too Far Ahead
Learning From Nature
The Rogue Animal
How the World Really Works:
Making Sense of Scents
An Age of Wonder
The Truth About Ukraine
Navigating Complexity
The Supply Chain Problem
The Promise of Dialogue
Too Dumb to Take Care of Ourselves
Extinction Capitalism
Homeless
Republicans Slide Into Fascism
All the Things I Was Wrong About
Several Short Sentences About Sharks
How Change Happens
What's the Best Possible Outcome?
The Perpetual Growth Machine
We Make Zero
How Long We've Been Around (graphic)
If You Wanted to Sabotage the Elections
Collective Intelligence & Complexity
Ten Things I Wish I'd Learned Earlier
The Problem With Systems
Against Hope (Video)
The Admission of Necessary Ignorance
Several Short Sentences About Jellyfish
Loren Eiseley, in Verse
A Synopsis of 'Finding the Sweet Spot'
Learning from Indigenous Cultures
The Gift Economy
The Job of the Media
The Wal-Mart Dilemma
The Illusion of the Separate Self, and Free Will:
No Free Will, No Freedom
The Other Side of 'No Me'
This Body Takes Me For a Walk
The Only One Who Really Knew Me
No Free Will — Fightin' Words
The Paradox of the Self
A Radical Non-Duality FAQ
What We Think We Know
Bark Bark Bark Bark Bark Bark Bark
Healing From Ourselves
The Entanglement Hypothesis
Nothing Needs to Happen
Nothing to Say About This
What I Wanted to Believe
A Continuous Reassemblage of Meaning
No Choice But to Misbehave
What's Apparently Happening
A Different Kind of Animal
Happy Now?
This Creature
Did Early Humans Have Selves?
Nothing On Offer Here
Even Simpler and More Hopeless Than That
Glimpses
How Our Bodies Sense the World
Fragments
What Happens in Vagus
We Have No Choice
Never Comfortable in the Skin of Self
Letting Go of the Story of Me
All There Is, Is This
A Theory of No Mind
Creative Works:
Mindful Wanderings (Reflections) (Archive)
A Prayer to No One
Frogs' Hollow (Short Story)
We Do What We Do (Poem)
Negative Assertions (Poem)
Reminder (Short Story)
A Canadian Sorry (Satire)
Under No Illusions (Short Story)
The Ever-Stranger (Poem)
The Fortune Teller (Short Story)
Non-Duality Dude (Play)
Your Self: An Owner's Manual (Satire)
All the Things I Thought I Knew (Short Story)
On the Shoulders of Giants (Short Story)
Improv (Poem)
Calling the Cage Freedom (Short Story)
Rune (Poem)
Only This (Poem)
The Other Extinction (Short Story)
Invisible (Poem)
Disruption (Short Story)
A Thought-Less Experiment (Poem)
Speaking Grosbeak (Short Story)
The Only Way There (Short Story)
The Wild Man (Short Story)
Flywheel (Short Story)
The Opposite of Presence (Satire)
How to Make Love Last (Poem)
The Horses' Bodies (Poem)
Enough (Lament)
Distracted (Short Story)
Worse, Still (Poem)
Conjurer (Satire)
A Conversation (Short Story)
Farewell to Albion (Poem)
My Other Sites
…and you’ll understand what ‘ it takes a village to raise a child’ could really mean. —that’s a brilliant summation of a new wave of thinking, doing and feeling. andrew
Thanks for this post Dave, now I understand what this specific kind of “crowd” is about….. Crowd, in the way I understand its meaning, …. is, most of the time….., something very different.
Superb post Dave ! I’m a big proponent of collective intelligence and this is a crucial part of the offering at our software startup.
At least some organisations are beginning to wake up to the fact that a very large amount of knowledge embedded throughout different parts of the entire structure is not being used. We bring together diverse teams to work on very complex, high value problems/challenges faced by organisations that they typically have tried to resolve themselves but have failed to do so. However one needs to provide some useful framework so that the conversations that get started are productive and actually help to deal with the issue of concern in a holistic way. I have spent 17 years experimenting with and evolving a methodology that takes account of the key concepts that have arisen from complexity research and that works in organisations facing real world issues (strategy/competition in a global economy, supply chain redesign, cost reduction, integration of companies/divisions, organisation redesign, etc, etc.). The good news is that organisations such as Rolls-Royce, the UK Ministry of Defence and others in investment banking, government delivery of social services, etc, are recognising the results they can get by involving diverse groups in an effective way i.e. not just asembling “crowds”. Other consultants don’t generally like us much because our approach usually only takes about six days of workshops using people from within an organisation to first of all develop a deep and wide understanding of the issue (a holistic approach) that exposes the key drivers then design a plan to resolve the issue that is optimal for the particular organisation. The participants implement the plan they have prepared in the process. So they don’t have outside consultants living in their organisation for months, usually telling them what they should be doing and racking up billable days.In some organisations there is a need for outside expertise in particular, mainly technical, areas where the organisation is lacking that technical knowledge e.g particular knowledge of certain types of market researchor financial expertise around raising investment, etc. But when it comes to making decisions about the “business” (I include the delivery of government social services in my use of the word “business”) then it is the people within the organisation who should be assembling the whole picture and making the decisions.To put the current situation in context, Drucker once pointed out that it takes about 20 years from the time new ideas first gain acceptance in the academic world to the time when those ideas are in general use in business. So looking at the new ideas that are surfacing from fairly recent research in complexity and other areas, we should not be surprised that people running organisations today who came up through the linear, reductionist approach of ‘scientific management’are not aware of better ways of managing organisations. Our challenge is to raise their awareness!!