The Intelligence Age
A doctoral candidate in Virginia developed a highly accurate (as far as we can tell) and probably one of a kind map of North Korea (Wall Street Journal article here). This may become important in light of other developments, including North Korea’s announcement of having done a second, successful underground test of an atomic bomb (see AP story here).
Earlier this year, researchers for the Open Security Foundation used seemingly unrelated newspaper articles to learn details of the Heartland Systems data breach, one of the biggest data hacking incidents yet known (Wired story here), before the breach was made public.
Both of these items reminded me of an old story about one of the first people to study serial murder. This was a detective (whose name I should be able to remember but can’t just now. Sorry!) who began studying newspapers from all over California in order to find similar murders that were not thought to be linked, as likely as not because they were in different jurisdictions so that the investigators involved did not even know about them. He discovered quite a few links no one else had noticed this way.
This sort of research to link up scattered, seemingly unrelated information is called open source intelligence gathering and we may not be far from the time when you can get a degree in it and (hopefully) lots of high-paying jobs. The term should not be confused with open source software or artificial intelligence. This intelligence is the kind that concerns intelligence agencies like the CIA. And the open just means not hidden.
Wikipedia defines open source intel as a “form of intelligence collection management that involves finding, selecting, and acquiring information from publicly available sources and analyzing it to produce actionable intelligence. In the Intelligence Community (IC), the term “open” refers to overt, publicly available sources (as opposed to covert or classified sources); it is not related to open-source software or public intelligence” (Open Source Intelligence). Or, more understandably: OSI is the art of finding things out that are not explicitly published, without spies or wiretaps or other such arcane and often illegal methods. It’s taking bits and pieces of information that are publicly available and combining them to discover something more interesting, something that might be non obvious or even secret. This is a form of data mining that the Internet has made much much easier than it used to be, though that is only part of the equation.
The interesting thing is that not only does the internet make it easier to access open sources (see for example how mandatory legal disclosures are more and more moving online, bypassing those impossible to read little newspaper pages where they’ve been for decades here), it also makes it easier to get distributed teams of people working on a problem, purely for the love of probing a mystery or solving a problem.
In the first example above, Curtis Melvin collates information provided by people from all over the world (sometimes known to him, sometimes anonymous), checks what he can and has used that information and Google Earth to make what may be the most accurate map of North Korea in existence. Agencies like the CIA may have more accurate ones but it seems at least possible that, if they do, it’s because they built on his work. There just aren’t enough people who’ve been to North Korea (and come back) for the pool of those really knowledgeable to be all that big.
This goes beyond concepts like Web Mining (which is similar but limited to web pages) and Crowd Sourcing (which is generally thought of more in a business context, though it doesn’t have to be) to create new information that certain people (Kim Jong who?) might prefer stayed hidden. Sometimes this is attributed to the power of the Internet but don’t forget the drive, curiosity and ingenuity of the people doing the work.
I think we’re at the beginning of seeing what this sort of distributed mania can do with all the info that’s shimmering out there in the matr in the Internet.
If I were a sneakier sort of person, I would say that what Melvin has built is the beginnings of a good escape map.
See what open source intel can do?
LinkedIn
Technorati Favorites