Open Source Intelligence for Newsrooms

So much has been made of Open Source Intelligence, OSINT, among both practitioners and the press, in the past few years that I was reluctant to use it without ironic quotes. You can find web-based toolkits and professional copy, and crowd-sourced pastebins on the subject, in abundance. You will be given long lists of bookmarks for websites for “People Finders” and “Deep Web Search” and the like, and some of these can be quite useful. Professionally produced literature will add methodology such as the traditional intelligence-gathering cycle to similar lists of sources. While all of these resources can be tapped for their selective value, I have yet to encounter any such guide or reference that can be read as an instruction manual and applied with reliable results (50%). One popular information-gathering method on the web, Google Dorks, seems to be particularly disappointing for most competent researchers attempting to expand their toolbox to include internet OSINT and ‘data-mining.’ It is quite possible for non-programmer reporters and researchers to apply the tools used in penetration-testing, data-mining, and by intelligence professionals to discover obscure documents targeted to their interest.

I apply a hybrid method I refer to casually as forensic indexing which produces results which enhance a story, and quite often merit publication in their own right.

I am offering my services to media organizations as a researcher and consultant, to teach journalists to apply Open Source Intelligence methods to their reporting with reliably publishable results. I will also be developing a workshop-style tutorial for presentation at relevant national conferences. My personal website (this one), another wordpress site dedicated to this project, as well as this description page for it are still under-construction, however the method is fully ready to be applied, and I am immediately available for hire.

Years of trial have gone into proofing this method, and I have retained uncounted gigabytes of sensitive and/or newsworthy items. A recent find was published in DigBoston (co-written by Chris Faraone, Jon Riley, and myself) regarding an intelligence video surveillance system tested at a concert last year in Boston, and has been widely covered in other media outlets, from Vice to Popular Science (and “…I know it’s the Mail, but….”).

Read the Boston story here:



A third part is coming very soon –

Chris Faraone, who edited the DigBoston piece as well as sharing the by-line, was kind enough to give me a recommendation to publish here:

It’s hard to think of a better lens than Boston through which to expose these questionable technologies. I’ve been covering Boston for ten years, and as someone who is not a native, I can say that a large part of the reason I stayed here after journalism school was because the people in this city are generally the type who are smart and informed enough to give a damn about their privacy.

On top of that, it’s not exactly a secret in this city that the police are incompetent, from their pathetic homicide clearance rate, to the department’s history of hardcore bigotry, to their failure to communicate effectively with federal authorities both before and after the bombing of the Boston Marathon. Since this story came out on Thursday, the Dig has received an inordinate amount of positive feedback and had virtually no one attempt the old argument of, “If you’re not doing anything wrong then you have nothing to worry about.”

I’m very much looking forward to the work Kenneth Lipp does with information he has gathered on other cities, but the Hub was by all measures — from the readers to the leaders — an ideal place to get started.

Contact me at for more information.

More links describing documents uncovered:


And here's an unrelated but awesome picture I took of Eric Holder
And here’s an unrelated but awesome picture I took of Eric Holder

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s