Collective Wisdom Model
For the last few weeks I’ve been bouncing ideas on how to implement the principles in James Surowiecki’s book The Wisdom of Crowds off a variety of people in the public and private spheres. I think I’ve finally got a model that works.

For those unfamiliar with the concept of the Wisdom of Crowds, here is just a bit of what I’ve written to get you up to speed:

  • A summary of the book, and some possible applications
  • Using the Wisdom of Crowds in business — some early thoughts
  • Using the Wisdom of Crowds to power a global think-tank
  • How the Iowa Electronic Market used the Wisdom of Crowds to correctly predict the Republican election sweep
  • Why the Wisdom of Crowds doesn’t work well to find partners

Tell me what you think of this approach, which is designed to integrate a lot of diverse applications: business, social, political, economic, artistic, scientific, or technological. You have a problem to solve or a decision to make, this model will help you do it better.

Just to restate the basic principle: Many cognitive, coordination and cooperation problems are best solved by canvassing groups (the larger the better) of reasonably informed, unbiased, engaged people. The group’s answer is almost invariably much better than any individual expert’s answer, even better than the best answer of the experts in the group.

The reason for this superiority is that each individual brings to the problem some valuable unique knowledge or perspective, and any errors in that knowledge or perspective are balanced off against those of others in the group, so the collective wisdom of the group is likely to be extremely accurate, reliable, knowledgeable, and predictive. If you’re skeptical, please read the book — Surowiecki presents dozens of examples to support this thesis. The average prediction of one such group, the Iowa Electronic Market, over the several months before the election, was that Bush would win by a comfortable 3% margin and that Republicans would make gains in both houses of Congress. They were exactly right.

My ‘Collective Intelligence’ model realizes (a) that there are some things that crowds can’t do (they need to be given a problem with a discrete or quantifiable set of possible answers from which to choose), (b) that care must be taken in the ‘qualification’ of the crowd to meet Surowiecki’s conditions of nonbias (they must understand the problem, be diverse in their perspectives, independent of groupthink tendencies and each able to bring a bit of unique knowledge to the problem, and (c) that there needs to be some incentive for people to participate in the crowd (those guessing correctly the number of jelly beans in the large jar at least win the jelly beans). The diagram above reflects these three constraints. Here’s how it works, using, as an example, a company’s problem of unsatisfactory productivity:

  1. This problem cannot be put to the crowd as is. If you simply asked “How do we best solve Unproductive Inc.’s productivity problem?”, you’d get at least as many answers as you had crowd members. There’d likely be few discernible patterns, and most of the crowd, not being familiar with Unproductive Inc.’s processes and operations, wouldn’t know what they were talking about and would therefore probably be biased by hearsay they’d read in the papers or on the Internet. The problem needs to be deconstructed into its component parts, a three-stage process: (i) what are the constituent issues that comprise the problem, (ii) what are the root causes underlying each constituent issue, and (iii) what is the range of possible solutions to each root cause. For example, Unproductive’s productivity problem could break down into excessive time spent looking for information, too many people handling a single customer problem, and a host of other issues. The root causes for the ‘excessive time spent looking for information’ issue could include poor tools, poor understanding of how to use the tools, failure or inability to delegate information searches to knowledgeable intermediaries etc. These might even have further root causes underlying them. And the possible solutions for ‘poor understanding of how to use the tools’ could include online help screens, classroom training, personal productivity improvement sessions, and others. If there is doubt as to which are the most important or relevant constituent issues, or underlying root causes, the ‘crowd’ can be canvassed to help make those assessments. But you need a small intermediary body of people with good analytical and creative skills, and an understanding of the problem, to develop the alternatives, to craft the question for the crowd. I call that body the Solution Team.
  2. Suppose now the Solution Team has broken Unproductive Inc.’s problem into eight issues, which between them have fourteen root causes (some overlapping), and for which 22 alternative solutions (again, some overlapping) have been identified. The next step is to ‘qualify’ the crowd that will rank these alternative solutions. In this case, it requires some knowledge of the processes and operations of Unproductive Inc., so the crowd would be limited to (i) employees of Unproductive Inc. and (ii) customers and advisors who have a basic understanding of Unproductive’s processes and operations. I call this body the Qualified Crowd.
  3. The solution alternatives crafted by the Solution Team are presented to the Qualified Crowd. Unproductive Inc. offers a Reward Pool of $X to be distributed equally among members of the Qualified Crowd who select the Collective Answer (the one selected by the largest number of members of the Qualified Crowd) as a motivation for participation. Alternatively, they could announce that they would draw one name from the Qualified Crowd members who select the Collective Answer, and reward the entire Reward Pool to that person. They could decide to hold back a portion of the Reward Pool that would be only distributed if and when the Board of Unproductive Inc. agrees that the implemented Collective Answer actually worked. Unlike most models where the fees are set by the vendor, under this model the customer gets to decide how much they’re willing to invest in getting the Collective Answer. The greater the investment, the greater the crowd it will attract, and the more likely the Collective Answer is to be accurate.
  4. The model could even be extended to allow Unproductive Inc., if they don’t have the resources to implement the Collective Answer themselves, to ask members of the Qualified Crowd to invest in the implementation of the Collective Answer, either as members of the implementation team (who could be remunerated on an hourly basis, or, if the implementation project is set up as a separate business entity, in the form of shares in the new entity) or as passive cash investors (who could receive a predetermined return on their investment, or shares if it’s in a new entity). The key again is flexibility. The model not only solves problems, it also helps assemble the resources to implement the solution.
  5. I see Collective Wisdom, the enterprise that oversees the entire process, and which helps the customer to select the Solution Team, to construct the Qualified Crowd, and to identify and contract with implementation resource, as a very small entity, perhaps just me and a couple of very creative partners with good analytical skills (I’d subcontract the website management — the core competency of this enterprise is the management of the Solution Teams, Crowd qualification and solution implementation. And I wouldn’t propose to try to ‘sell’ Collective Wisdom’s services by advertising or through sales calls — initially, customers will be drawn by the ideas of the book to at least give it a try. Thereafter, it will all be viral marketing: Customers who get better answers at lower cost will spread the word, and members of the ‘crowd’ who make money, or perhaps even earn a living, by contributing to the Collective Wisdom solution process will spread the word faster.

Let’s look at another, simpler example. Imagineers Inc. wants to know which of a new series of thirty possible products and services they should release to the market, and how to price them.

  1. No need for a Solution Team for this problem. Imagineers Inc. already knows the solution alternatives — thirty alternative products and services, each with a possible price of from $1 to sky’s-the-limit.
  2. Almost no restrictions to the crowd in this example. If it’s a consumer product, anyone who’s motivated is Qualified. If it’s a business-to-business product, you could either limit the Qualified Crowd to those who understand the function of Imagineers’ products and services to their customers, or you could provide a website where those that wanted to participate could go to learn more about the products and services. You could have an online ‘Qualification Test’. You could even segment the Qualified Crowd into Customer and Non-Customer segments and get Collective Answers from each, using the Customer segment as a control group.
  3. The Reward Pool works the same way as in the previous example. If the pool is $50,000, you could distribute it to all the members of the Qualified Crowd who picked the Collective Answer (the product or service of the thirty that was selected by the most crowd members), or you could give the entire Reward Pool to the one person who not only picked the Collective Answer but got closest to the second Collective Answer — the optimal selling price for this product or service (the median of all the Qualified Crowd’s responses).
  4. Implementation resources (people and money) could be solicited the same way as in the first example. Imagineers Inc. could even decide to spin off an entirely new entity to sell the Collective Answer, and use the crowd to finance and staff it.
  5. The cost of mispricing a new product offering is enormous. Price too high and nobody buys. Price too low and you leave profits on the table — and customers really resent price hikes after they’ve started buying something. One happy customer who’s invested a few thousand dollars in Collective Wisdom and saved millions as a result, and you’ll have other customers lining up, just from word-of-mouth.

The model will even work on what Surowiecki calls co-operation problems, where there is some constraint that requires compromise by all, such as “What is the optimal salary distribution for our company?” All that’s needed to include such problems is to state clearly the constraints (e.g. total organization salary cannot exceed $X) and to not allow Qualified Crowd members to ‘vote’ on issues where they have a conflict of interest (e.g. each person can vote on everyone else’s salary but not on their own).

This model isn’t limited to business applications. Non-profits have productivity problems too. In fact, of the 25 problems that most often keep executives awake at night, which I reported on in an earlier post*, most of them are challenges for organizations of every type, from government departments to NGOs to charities. Although the bigger your organization the more likely the Wisdom of Crowds is to benefit your problem-solving and decision-making, even small organizations could benefit from tapping into Collective Wisdom. Here’s just a few questions that could be posed to the ‘crowd’, just to show the diversity of applications for this model:

  • Should our central bank raise interest rates next month, and by how much?
  • What is the $US going to be worth, relative to the Euro or a global currency basket, this time next year?
  • Which alternative voting system is the best (the BC government used The Wisdom of Crowds to answer this, and it worked brilliantly)?
  • What’s the best way to motivate people in the third world to have fewer children?
  • How could we break our dependence on fossil fuels within the next decade?
  • What’s the answer to eliminating popular support in many countries for terrorist attacks?
  • What solution set should our organization use for our desktop, Intranet and connectivity technology?
  • What features and functionalities should we include (and not include) in our next software or hardware release?
  • What major unforeseen risks threaten our organization and/or its customers?
  • What kind of music, literature, film, TV programming or art does the public want and think it’s not getting enough of?
  • How can we fairly reduce global disparities between rich and poor, and improve distribution mechanisms to get resources desperately needed by the poor to their destinations?

I’m pragmatic enough to be willing to have Collective Wisdom work initially on business problems, and continue to use such fee-paying applications to fund the enterprise. But in my heart what I really want this enterprise to do is the think-tank work, solving some of the world’s most intractable and pressing problems. I’m optimistic for two reasons: I think when it comes to solving global problems, a lot of people will be motivated to join the ‘crowd’ for altruistic reasons, and donate their time just to help make the world a better place. We don’t need to offer them a Reward Pool for such problems. And I don’t think the infrastructure of Collective Wisdom needs to be all that large and expensive. I think a few profitable, successful business pilots will be enough to get the process streamlined, the infrastructure paid for and the crowd assembled, so that we can start spending a chunk of time on solving global problems, for free. And although there may be some first-mover advantage here, I think the real value of Collective Wisdom will be in the creativity and analytical skill of its core staff and Solution Teams, and in the quality of its Qualified Crowds and implementation teams. It’s the people, not the technology, that will make or break it.

There are some kinds of questions that I’m ambivalent about throwing to the crowd. In the above model, it’s the Solution Team and the customer whose imagination is tapped, not the crowd’s. I’ve assumed that giving a crowd, even a Qualified Crowd, an open-ended question like “What features would you like to see in a car, which you can’t find in any car today?” would be an invitation to anarchy. You could be reading replies to such a question for months, and end up with a completely unmanageable number of ideas, so many that you’d never be able to identify the needle in the haystack that might actually pay off in a big way. But maybe I’m a pessimist. What do you think? Could we tap in not only to the Wisdom of Crowds, but to it’s creativity as well? How could we do so in a manageable way?

I think the Value Proposition for this model is compelling: Tapping into the Wisdom of Crowds with a disciplined process will reduce or eliminate the need for (and cost of) ‘expert’ consultants, academics and focus groups, while producing better decisions and solutions than those experts can offer. In the process, it can even provide manpower and investment for implementing the solutions, reducing the need for RFPs and venture capital. Business is always looking for ways to reduce cost. Not-for-profit organizations are always strapped for cash. This model works for both.

That’s what I have so far. I’m getting a lot of expressions of curious interest, including some business organizations that would be willing to test the waters. I’m copying James Surowiecki on this post to see what he thinks. I’d love to know what you think. You are my wise and qualified crowd.


* Here are the 25 business problems, any of which could be addressed using this model:

  1. How can we improve employee productivity?
  2. How can we reduce business/credit/security risk?
  3. How can we become more innovative?
  4. Should we outsource IT, KM, HR and/or marketing?
  5. Which of these new product ideas will be successful?
  6. What price should we sell this new or old product for?
  7. How will sales/prices be affected by future innovations?
  8. How will sales be affected by inflation, int. rates etc?
  9. How will material & labour costs change in the future?
  10. How can we reduce our fixed costs & overhead?
  11. How can we increase our market or customer share?
  12. How can we (a) find or (b) keep the best people?
  13. Which acquisitions should we make, at what price?
  14. How much is our company worth?
  15. What service/community wraparounds would work?
  16. How much should we be paying staff, management?
  17. Which companies should we partner with?
  18. Which functions should we centralize, decentralize?
  19. How should we penetrate a new market/demographic?
  20. How can we increase customer satisfaction/retention?
  21. Which suppliers should we use?
  22. How can we reduce employee theft, fraud, error?
  23. Where are we paying more taxes than we have to?
  24. How should we protect our intellectual property?
  25. What new businesses should we start, or spin off?
This entry was posted in Working Smarter. Bookmark the permalink.


  1. marty says:

    great article (again). Here’s a couple examples of collective wisdom at work:http://www.fastcompany.com/magazine/57/lilly.htmlthe company mentioned at the end (Quovix) is my company (only now we have 1,500 in our network). InnoCentive has about 60,000 scientists who can help solve problems).The model works but not without challenges.

  2. Dale Asberry says:

    One very concerning problem is that The Wisdom of the Crowds is easily subverted by a single person with a strong ego looking to gain power. That person will manipulate and organize pockets of power to support his own.Why is this so?”All this was inspired by the principle – which is quite true in itself – that in the big lie there is always a certain force of credibility; because the broad masses of a nation are always more easily corrupted in the deeper strata of their emotional nature than consciously or voluntarily; and thus in the primitive simplicity of their minds they more readily fall victims to the big lie than the small lie, since they themselves often tell small lies in little matters but would be ashamed to resort to large-scale falsehoods. It would never come into their heads to fabricate colossal untruths, and they would not believe that others could have the impudence to distort the truth so infamously. Even though the facts which prove this to be so may be brought clearly to their minds, they will still doubt and waver and will continue to think that there may be some other explanation. For the grossly impudent lie always leaves traces behind it, even after it has been nailed down, a fact which is known to all expert liars in this world and to all who conspire together in the art of lying. These people know only too well how to use falsehood for the basest purposes.”- Adolf Hitler

  3. Dale Asberry says:

    Marty,Thanks for the link to Quovix. My home is not a mile and a half from the office (Fall Creek and Shadeland). Looking for a top notch distributed computing architect?

  4. Marty says:

    Dale,Hey – Fall Creek And Shadeland? That’s where I live! How about a “Save The World” coffee session at Barton’s friday morning. We can debate Pollard’s ideas and maybe generate a few of our own… time to go offline for further discussion… – email is mmorrow@quovix.com

  5. T. V. says:

    Ideology doesn’t proceed by suppression but by framing; as any pollster knows, you can strongly determine a poll’s outcome by phrasing the question a little differently. How are the specific solutions presented to the Qualified Crowd for voting going to be secured from this kind of framing? Sitting here I can imagine about twenty different ways that I could tweak outcomes by simple choice of vocabulary. That’s just the first of the who-watches-the-watchmen questions you’d face (another is how you decide who is “qualified,” and who gets to make that decision–and by the way, there doesn’t seem to be any Suroweckian diversity-and-knowledge screening for the IEM traders at all, so how does IEM count as evidence for or against his thesis? Not to mention the spotty track record your challenging commenters cite…even the linked essay about IEM says it only has a “slight edge” over polls). Dave, your enthusiasm for Wisdom-of-Crowds solutions strikes me as very “Canadian”–it doesn’t assume the degree of nihilistic corruption that’s the necessary pragmatic beginning point for thinking about any decisionmaking in U.S. business or politics.

  6. Dave Pollard says:

    Marty — thanks for the link, an interesting and needed ‘case study’.Dale — yes, it can easily happen, which is why Surowiecki introduces criteria for a ‘wise’ crowd, one of which is independence and freedom from the potential of ‘groupthink’TV — absolutely, which is why the Solution Team’s makeup, modus operandi, and commitment to a fair and truthful result is so critical; any kind of poll can be twisted to produce the desired result, which is why I think this will work best on problems and decisions where politics (including business politics) is not an important factor.

  7. erwin spriet says:

    Good article Dave. Let me first say that I’m a (European) consultant that tries to make a living out of innovation and change in organizations. (At least that makes me one of your crowd, which in turn of course is the evidence of my wisdom.) In my daily practice I’ve been using a self-developed “technology” for over 15 years now. In retrospect, it is a mix of action research, Socratic method and -I must admit- wisdom of groups. Since my goal (or mission if you prefer) is to obtain profit for my client while improving satisfaction for the workforce (both!), my work involves a lot of surveying opinions of employees and other stakeholders. In doing so I must be very careful when formulating questions because there is an important risk of influencing the employee. So, T.V.’s remark of tweaking outcomes by use of vocabulary is very realistic and to the point. On the other hand, a company is not a democratic organisation: someone has the power based on his capital investment. So, even if my first step is to get ‘the truth on the table’ before doing anything, it is far from uncommon that the one who holds the power tries to “adjust” that truth. It is my experience that telling or selling “wisdom-of-crowds” as the ultimate procedure for sound decision-making to a CEO, is not that easy. First, because the crowd-decision may differ from his; second, because the involvement of more people invariably results in higher cost/more time. If those don’t work, there’s always the possibility of ‘not enough information’, ‘we tried that before’, or ‘priorities have changed’ as excuses. Besides, if in a company the crowds fell apart in a Bush-Kerry ratio, you’d have 48% of people that do not support the decision, and there’s nothing more dangerous than intelligent people whose opinions are discarded.Glad to hear though that you get reactions from businesses willing to “test the waters”. I think I know for a fact that -at least in Europe- it is not that easy.As for the kind of questions onecan ask the crowd, again a lot depends on how you phrase them. The rule of thumb I use in my consulting assignments is to formulate the question in such a way that it looks for the needs of the customer. The real needs. Not wants, demands or likes. (These can easily be added later on.) In this way I wouldn’t ask for what features people would like in a car that aren’t there today. Instead I’d explore what a man, a woman needs from a car. One ultimate answer could even be “I don’t need a car, I need something to get from A to B”. This information stretches the original question to a domain where the notion “car” is irrelevant. Which means that the whole question is irrelevant.See, I think a company can beat another by better understanding its customer. Christensen has described convincingly that it is not always a good strategy to keep on adding features to a product. That is because these extra features (logically) seem to make sense for the developer or marketer but they do not make sense for the customer because those features do not respond to a real need of his. That is why the average MS Excel-user apparently uses only 5 to 10% of its features. And that is why many users simply copy the thing instead of paying for it.Look at the facts: when does a successful product get obsolete? When another takes its place that clearly better matches the real need than the former. Not because it is stuffed with twice as much features the customer will not (be able to) use anyway.Finally I hope wisdom-of-crowds will not “eliminate” my firm (eliminating consultants being one of the advantages as you say). I not only hope, I don’t think so. I remain confident because I’m not exactly the ‘expert’ consultant you mean, since my expertise is in fact the expertise of facilitating and guiding a series of processes that use collective wisdom in order to result in our mission as stated above.Erwin Spriet, Belgium

  8. Robin Hanson says:

    It is great to see one more person realize just how general the potential for these things are. I’m most enthusiastic about the clearest cases, with clear alternatives and a clear outcome measure, but hopefully all of them will be tried eventually.

Comments are closed.