Thursday, May 18, 2006

previous entry | main | next entry | TrackBack (0)

Open Thinthread thread

Sorry for the post title -- couldn't resist.

Siobhan Gorman has a story in the Baltimore Sun that suggests that, in the late 1990s, the NSA ditched one kind of data collection program (Thinthread) in favor of another. A lot of NSA types apparently preferred Thinthread:

The National Security Agency developed a pilot program in the late 1990s that would have enabled it to gather and analyze massive amounts of communications data without running afoul of privacy laws. But after the Sept. 11 attacks, it shelved the project -- not because it failed to work -- but because of bureaucratic infighting and a sudden White House expansion of the agency's surveillance powers, according to several intelligence officials.

The agency opted instead to adopt only one component of the program, which produced a far less capable and rigorous program. It remains the backbone of the NSA's warrantless surveillance efforts, tracking domestic and overseas communications from a vast databank of information, and monitoring selected calls.

Four intelligence officials knowledgeable about the program agreed to discuss it with The Sun only if granted anonymity because of the sensitivity of the subject.

The program the NSA rejected, called ThinThread, was developed to handle greater volumes of information, partly in expectation of threats surrounding the millennium celebrations. Sources say it bundled together four cutting-edge surveillance tools. ThinThread would have:

* Used more sophisticated methods of sorting through massive phone and e-mail data to identify suspect communications.

* Identified U.S. phone numbers and other communications data and encrypted them to ensure caller privacy.

* Employed an automated auditing system to monitor how analysts handled the information, in order to prevent misuse and improve efficiency.

* Analyzed the data to identify relationships between callers and chronicle their contacts. Only when evidence of a potential threat had been developed would analysts be able to request decryption of the records.

An agency spokesman declined to discuss NSA operations....

In what intelligence experts describe as rigorous testing of ThinThread in 1998, the project succeeded at each task with high marks. For example, its ability to sort through massive amounts of data to find threat-related communications far surpassed the existing system, sources said. It also was able to rapidly separate and encrypt U.S.-related communications to ensure privacy.

But the NSA, then headed by Air Force Gen. Michael V. Hayden, opted against both of those tools, as well as the feature that monitored potential abuse of the records. Only the data analysis facet of the program survived and became the basis for the warrantless surveillance program.

The decision, which one official attributed to "turf protection and empire building," has undermined the agency's ability to zero in on potential threats, sources say. In the wake of revelations about the agency's wide gathering of U.S. phone records, they add, ThinThread could have provided a simple solution to privacy concerns.

My take is similar to Kevin Drum's -- I'm not sure if this is an example of dumb policymaking or an example of the losers of a policy decision leaking to the press at an opportune time.

I am sure that readers wil have their own opinions.

posted by Dan on 05.18.06 at 11:04 AM


How about both?

posted by: Rick Latshaw on 05.18.06 at 11:04 AM [permalink]

Is it just me, or is 'Spokesman for the NSA' just about the easiest job in existence. Reporters call you up, and you decline to discuss the agency's operations. Presumably they'll get around to automating the whole process, but until then it sounds like a cush gig.

posted by: Dave on 05.18.06 at 11:04 AM [permalink]

Issues like this are good illustrations of why a military man is ideal for the new head of the CIA (and even NSA). You will be hard pressed to see a military officer leak something to the media because his idea lost in the planning phase. The reason for this is that the military is, comparitavely-speaking, highly professional (using the Samuel Huntington definition) and grounded with a non-negotiable, uniform values system.

Military officers have and sense a duty to be candid and fight hard for their point of view, but to salute, move out smartly, and execute once the decision is made. And if that decision is so unacceptible in their professional opinion as to run counter to their conscience, then they resign in protest (rather than being a coward and a sneak who rights one perceived injustice by committing the crime of leaking).

A leader with that value system and that type of professional background is needed in our intelligence community, in my opinion.

posted by: Tim Mathews on 05.18.06 at 11:04 AM [permalink]

"Four intelligence officials knowledgeable about the program agreed to discuss it with The Sun only if granted anonymity because of the sensitivity of the subject."

That's the saddest thing I've read today. Even worse is the fact that it seems to happen everyday in our current environment where our political opponent is treated as an enemy. We actually do have real enemies, and we spend our time fighting ourselves.

posted by: bob on 05.18.06 at 11:04 AM [permalink]

This sounds like it would have required considerable software development--not the encryption, which should be pretty simple, but the "more sophisticated methods of sorting through massive phone and e-mail data" and the "automated auditing system." Big software projects don't have a great track record for timely completion--look what happened to the FBI's Virtual Case File system, which was conceptually pretty simple.

It's not irrational, when time is of the essence, to do something that can be done *now* rather than to do something that is dependent on completion of potentially-long development tasks.

posted by: David Foster on 05.18.06 at 11:04 AM [permalink]

Post a Comment:


Email Address:



Remember your info?