spy vs spyLast week’s New Yorker featured an article by Elsa Walsh entitled “Learning to Spy”. It was about Maureen Baginski, the woman who’s been brought in to clean up and modernize key functions of the FBI. She puts the FBI’s intelligence mandate simply:

“You need something, you go get raw material and you add value to it. You put out a product and you keep adjusting, based on the feedback that you get. That’s really all it is.”

So why isn’t the FBI doing this? According to Ms. Baginski, most FBI agents were trained and instructed to take ‘intelligence gathering’ too literally. They aggregated data just in case it was useful or needed, often without doing anything with it. Her motto for reform: Hunt, Don’t Gather; Disseminate, Don’t Just Aggregate.

Because they focused on gathering intelligence, the FBI ended up with mountains of information, so much that they didn’t have the time or resources to transcribe it all, so some of it just got erased or thrown out without even being examined. And this overload of information wasn’t shared or coordinated with other intelligence groups, so there was no context for assessing its meaning, its importance. That was one of the reasons for the failure to prevent 9/11. The information was all there, but no one distilled and organized and analyzed and integrated it, the various intelligence groups didn’t distill and share what they knew, and no one put two (sharp increase in enrolment in US commercial flight training schools by people from Arab countries) and two (increase in frequency of visits to the US by known associates of CIA-operative-turned-radical Osama bin Laden) together.

But there was a second behavioural problem. Too many of the agency’s employees were content to just gather intelligence, not “go get” it. In business circles, the gathering of information from readily available sources — the Internet, published documents, recordings, files etc. — is called secondary research. It’s what librarians used to do, and what in many organizations can now be done by anyone. Going and getting information, by interviewing people directly or obtaining it from physical observation, is called primary research. It’s harder, and takes what I call ‘know-who’ (awareness of who to interview or get permission from, and how to contact them, and a ‘great Rolodex’) plus persuasive and interviewing skills (in business, the more extreme method of conducting primary research — interrogation — is thankfully not called for).

In today’s world, there is so much information out there that settling for secondary sources is tempting. You could spend hours studying Bin Laden’s latest tape, for example, to get clues as to his whereabouts, his intentions, his motivations. Without leaving your easy chair.

Hunting for intelligence on these things from primary sources is harder, more dangerous, and more time-consuming. Contrast the armchair/laptop intelligence gathering with the intelligence hunting of some of the New Yorker‘s own fine journalists: Lawrence Wright, who talked with Saudi princes, journalists, religious leaders, opposition forces and the man on the street in Riyadh; Jeffrey Goldberg, who criss-crossed the new wall between Israel and the West Bank of Palestine to speak with settlers, pacifists, leaders and angry adversaries on both sides of the intractable conflict; and Seymour Hersh, still digging into inconsistencies, interviewing Iraqis from all factions face-to-face and using contacts, guile and dogged persistence to unearth the truth about what has happened in Iraq.

When you read the first-hand accounts of these investigative reporters, you really begin to understand what is happening in the Mideast, and why. These reporters, courageous and competent and connected, but no more so than what we would expect from professional intelligence agents, surely have a deep appreciation for what is happening and how to resolve it. But no matter how much technology and psychological study and fact-checking you apply to Bin Laden’s tape, you really can’t do more than guess what’s happening in his world from studying it. That’s the difference between primary and secondary research.

I’m not suggesting that today the FBI and the CIA (unlike the National Security Agency, whose mandate appears more specifically focused on analysis of secondary source information) just sit around reading stuff on the Internet, listening to wiretaps and watching surveillance videos all day. But Elsa Walsh isn’t the first to suggest that a disproportionate amount of intelligence agencies’ time is spent just gathering intelligence, and an insufficient amount hunting for the really valuable intelligence that you can only get directly from the horse’s mouth, adding value to it by distillation, analysis, providing insight and context, and then sharing, coordinating and disseminating it to others who can help add meaning to it and make it actionable. Knowledge is information that you can do something with.

The same could be said for intelligence-gathering in business. Ms. Baginski’s quote at the top of this post could just as easily apply to the job of business researchers and analysts. In fact, the discipline of ‘competitive intelligence’ prides itself on tapping many more primary sources than most other types of business research. But ‘CI’ is usually narrowly focused, as its name applies, on getting information about the employer’s competitors. Likewise, the ‘primary research’ done by investment brokers is conducted through their target’s PR department and other selected executives coached on what to say and what not to say, rarely much more informative than reading the company’s press releases. Overall, business is pretty lousy at primary research, so it’s not surprising that they’re usually out of touch with both employees and customers, and hence make so many bad decisions.

The real truth about what is going on in a business is best obtained by talking to front-line employees and to customers, using a technique that Knowledge Management guru Dave Snowden calls ‘cultural anthropology’. This involves employing a series of methods to earn, legitimately, the trust of the people you are interviewing or observing, so you hear and see what is really happening, not what the employees or customers think you want to hear. That’s knowledge you’ll never find in any database.

Tapping into that knowledge can be done in two ways: By conducting one-on-one interviews with front-line employees, perhaps during the process of providing those employees with Personal Productivity Improvement assistance, or, by canvassing the employees or customers for their suggested answers to business problems to aggregate The Wisdom of Crowds.

Whether the focus is on solving global political or security problems, or solving business problems, the lessons are the same: Hunt, don’t just gather; Share, collaborate and disseminate, don’t just aggregate; and “go get” intelligence direct from the people who know what’s happening and why, and who can provide context that is missing from secondary sources.

The cost of doing so is higher, and it’s harder work that requires considerable ‘know who’ and ‘know how’ skill. But it’s the only way to effectively reduce the cost of not knowing — a cost that, as evidenced by the failure to prevent the 9/11 attacks or to detect the fraud that led to the collapse of Enron, can be catastrophic.

This entry was posted in Working Smarter. Bookmark the permalink.


  1. Life Tenant says:

    Interesting post. But we shouldn’t discount too much the value of “open source” information. I have heard it argued that a substantial amount of information available through public sources, such as stories published by investigative journalists, offered valuable clues to terrorist plans and tactics in the 1990s, but was overlooked by intelligence agencies fixated on gathering information themselves through clandestine or high tech means. Intelligence agencies should take advantage of the fact that a huge number of talented and dedicated people, in a wide range of institutions, are gathering and analyzing information.

Comments are closed.