WASHINGTON— The National Security Agency acknowledged that it repeatedly violated its own privacy guidelines in a now-defunct program to collect "to and from" data in American email, according to newly released documents that paint a picture of incompetence but offer no evidence that the agency intentionally misused its surveillance powers.
A judge on the Foreign Intelligence Surveillance Court, John D. Bates, said in an opinion whose date was redacted that there had been "systemic over collection" in the email program and that "those responsible for conducting oversight at the NSA had failed to do so effectively."
The documents, released by the director of national intelligence in response to a Freedom of Information Act lawsuit by civil liberties groups, offer new details about that and other surveillance programs previously revealed through leaks by former NSA contractor Edward Snowden.
The latest disclosures come as some members of Congress are pushing to rein in the NSA. On Thursday, the spy agency's top civilian is to testify before a Senate committee that is considering a bill to strip it of its authority to collect American phone records, which it says it needs to detect domestic terrorist plots.
On Monday, three Somali immigrants — an imam, a cabdriver and an employee of a money-transmitting business — were sentenced to federal prison in San Diego for sending money to a terrorist group in their homeland. It is the only prosecution the government has said resulted exclusively from its collection of American telephone records beginning in late 2001.
The new documents, including once-secret court opinions and disclosures to congressional committees, show that the NSA was under strict judicial supervision when collecting information connected to Americans. But they also make clear that the agency repeatedly failed to comply with its guidelines.
The heavily redacted Bates opinion doesn't detail the violations but suggests that the NSA was collecting "to and from" information on email accounts well beyond those that analysts had a "reasonable suspicion" were connected to terrorists. The agency also allowed information about Americans to be viewed by intelligence officials who were not authorized to see it.
The government said in a document submitted to the court that its violations were caused by "poor management, lack of involvement by compliance officials and lack of internal verification procedures, not by bad faith."
Despite his criticisms, Bates allowed the program — which did not collect the content of emails — to continue. The effort was abandoned in 2011, U.S. intelligence officials have said, because it wasn't yielding valuable counter-terrorism intelligence. Sen. Ron Wyden (D-Ore.) has said the program was a wasteful intrusion on Americans' privacy.
In the original legal opinion that authorized the program, U.S. District Judge Colleen Kollar-Kotelly said Americans did not have a reasonable expectation of privacy for their email and telephone metadata — information that shows who they are in contact with but not the content of their communications. Intelligence officials have compared metadata to the address written on the outside of a mailed envelope.
The judge acknowledged, though, that she was validating a novel interpretation of a law that allows authorities to use pen registers, also known as trap-and-trace devices, to record numbers called and received by suspects.
"The court recognizes that … it is finding that these definitions encompass an exceptionally broad form of collection," Kollar-Kotelly wrote in an opinion whose date has been redacted.
In a statement, James R. Clapper, the director of national intelligence, said the new releases were consistent with President Obama's instructions to "make public as much information as possible about certain sensitive programs while being mindful of the need to protect sensitive classified intelligence activities and national security."
"Release of these documents reflects the executive branch's continued commitment to making information about this intelligence-collection program publicly available when appropriate and consistent with the national security of the United States," he said.