U.S. counter-terrorism agents still hamstrung by data-sharing failures

Counter-terrorism analysts still lack the data-search tools that might have kept a bomb-wearing Al Qaeda operative from boarding a Detroit-bound airliner nine months ago, and probably won’t have them any time soon, U.S. officials acknowledge.

At the same time, officials say the terrorist threat against the U.S. is becoming more complex, with a greater risk from home-grown militants whose low profiles make sophisticated intelligence analysis more important than ever.

“It frustrates me,” said former Republican New Jersey Gov. Thomas H. Kean, who co-chaired the Sept. 11 commission, which urged U.S. intelligence agencies to vastly improve information sharing. “The president’s got to make this a top priority,” and right now it doesn’t seem to be, Kean said.

Analysts at the National Counterterrorism Center, created after the Sept. 11 attacks to integrate intelligence gathered by dozens of spy agencies, still sit in front of multiple computers searching databases maintained by the different government departments. The director, Michael E. Leiter, has three computer monitors and nine hard drives in his office.


Lawmakers have been pushing for a capability to search across the government’s vast library of terrorism information, but intelligence officials say there are serious technical and policy hurdles. The databases are written in myriad computer languages; different legal standards are employed on how collected information can be used; and there is reluctance within some agencies to share data.

That makes it harder to connect disparate pieces of threat information, which is exactly what went wrong in the case of Umar Farouk Abdulmutallab, a Nigerian who on Christmas Day last year tried to blow up an airplane using explosives sewn into his underwear. The bomb failed to detonate, and a passenger jumped on him.

Abdulmutallab, whose father had contacted the U.S. Embassy in Nigeria to express concern about his son’s extremist views, was not placed on a watch list barring him from flying to the U.S. because analysts didn’t connect all the information the intelligence community had about him, a White House review in January found.

Officers at the CIA and the National Counterterrorism Center “did not search all available databases to uncover additional derogatory information that could have been correlated with Mr. Abdulmutallab,” the White House review found, and their software didn’t allow them to correlate data that would have made the threat more clear.

The counter-terrorism center has since formed “pursuit teams” to dig into data, and the CIA has committed to disseminating information on suspected extremists and terrorists within 48 hours. But the agencies still lack the capability to search across all of the hundreds of databases containing potentially relevant information, officials concede.

Sen. Susan Collins, the ranking Republican on the Senate Homeland Security Committee, said last month that she found it disturbing that the problem had persisted.

When asked by Collins at a Sept. 22 hearing why the data-search problem had not been solved, Leiter answered, “There are a multitude of challenges.”

Congress has since addressed one of those challenges. An intelligence bill that passed Thursday made clear that information from “operational files” of the CIA intelligence agencies passed on to the counter-terrorism center was not subject to the Freedom of Information Act. The CIA had been reluctant to share certain information out of fear that it could somehow become public, according to a U.S. official who did not want to speak on the record about sensitive intelligence matters.

Other hurdles remain -- some technical, some legal, some bureaucratic, Leiter said. For example, information about U.S. citizens or residents must be handled differently than information on foreigners. And data gathered under the Foreign Intelligence Surveillance Act — targeting foreigners but potentially including information on Americans — carry restrictions on their use, Leiter said.

There are no easy legislative or technical fixes, said Russell E. Travers, information sharing chief at the National Counterterrorism Center. The agency is pursuing solutions that will allow automated connection of related information across databases, but “my guess is that this will be a challenge into perpetuity, because we get more and more information every day,” he said. One intelligence agency alone gets 8,000 terrorism messages each day with 11,000 to 15,000 names, he said.

Not everyone thinks information sharing between agencies is warranted.

“The idea of creating a single, U.S. government-wide repository of information on all things related to terrorism isn’t feasible — at least in the near term — and probably isn’t desirable,” said one U.S. counter-terrorism official not authorized to speak publicly. “Even as we’ve greatly expanded information sharing since 9/11, you still have to think about security and the sensitivity of certain data.”

That view infuriates Fran Townsend, an assistant to the president for homeland security and counter-terrorism in the George W. Bush administration.

“This is one that makes me angry,” she said at a public event in April. “This is not a technology problem. It’s a failure of policy and a failure of leadership. I venture to say that if the president of the United States calls in his Cabinet and says a Cabinet member will be fired if his agency fails to share information, you betcha that information is going to get shared … and we haven’t seen that.”

Travers argued that there were indeed technical problems, as well as tough policy ones. The Herculean task of separating relevant information from background noise makes terrorism analysis an extraordinarily difficult art, he said, and there is no button to push to identify non-obvious relationships.

“What I think we can do,” he said, “is shrink the haystack and make it somewhat easier for the analysts.”