It appeared that the White House was blindsided by events in Egypt at first. The traditional intel sources failed to spot the undercurrents that suggested a revolt. NPR report today that this oversight is being corected – the Intellignce Community is going to learn how to scan the web for “smoke”.

Traditionally, intelligence agencies have relied on top-secret information to track changes in other countries. But wiretaps and secret intercepts didn’t help U.S. officials predict the Arab Spring that has brought revolution across the Middle East and North Africa.

In hindsight, officials say there could have found some clues about what was about to happen if they had read open sources more closely. Now they are searching for systematic ways to do that.

The uprisings in the region have shown intelligence officials that they need new ways to understand what motivates people around the world. While traditional intelligence tools can help, they are limited in their ability to put their fingers on the pulse of society or anticipate fickle human behavior.

“The traditional intelligence community is absolutely biased toward classified information,” said Lt. Col. Reid Sawyer, an Army intelligence officer and head of West Point’s Combating Terrorism Center. “I think that open source provides a critical lens into understanding the world around us in a much more dynamic way than traditional intelligence sources can provide.”

Isn’t this how all “intelligence” will work now?

In the past we have asked questions? Used artificial groups like focus groups. Our choices have influenced what we heard back. But now, we can listen and see patterns emerge. This is how we will also present the data – not in a linear report but as a pattern. Too weird an idea? Check this brilliant example of patterning and the Middle East designed by the Guardian. Here by taking a pattern perspective, we can see how momentum builds and broadens. We can see the dynamics!

We can all do this for any issue that we care about.

Discusses the countless uses for integrated search in intelligence agencies.

Bio: Robert Coxe is SVP, chief information officer for Criterion Systems, a company specializing in providing information technology services to Intelligence agencies, the Federal Government, Department of Defense and Fortune 500 commercial enterprises.