WASHINGTON — The National Security Agency improperly collected the emails of tens of thousands of Americans for three years before acknowledging the problem in 2011 and bringing it to the attention of the secret intelligence court, which ordered the program overhauled.
Officials disclosed the history of that unlawful surveillance Wednesday, releasing three partially redacted opinions of the Foreign Intelligence Surveillance Court that detailed the judges’ concerns about how the NSA had been siphoning data from the Internet in an effort to collect foreign intelligence.
The documents were released in response to a Freedom of Information Act lawsuit filed by the Electronic Frontier Foundation, an advocacy group based in San Francisco.
The court ordered the NSA to stop what it had been doing and impose a technical solution that separated emails involving foreigners, which the NSA legally can collect, from those strictly between Americans. Where that technical solution didn’t work, the court required the NSA to restrict the use of any domestic emails and destroy the records after two years, instead of the normal five years.
U.S. intelligence officials, who briefed reporters under ground rules that they not be identified, sought to portray the matter as a technical glitch that the intelligence agencies caught and fixed. But in the court opinion, judges said the NSA repeatedly had misled them about the scope of what it was doing.
“The court is troubled,” Judge John D. Bates wrote in a footnote, that the email problem “marked the third instance in less than three years in which the government has disclosed a substantial misrepresentation regarding the scope of a major collection program.”
The footnote described one of the other violations, which involved a different NSA program: the bulk collection of telephone calling records in the U.S. The NSA has amassed a huge database of so-called metadata for most telephone calls made in the country. The data include which numbers called which other numbers, the dates and times of calls and their duration.
The existence of the massive collection of telephone records was the first in a series of disclosures about intelligence programs made this year by former NSA contractor Edward Snowden.
The violation the judge referred to was discovered in 2009 and apparently involved the process under which NSA analysts search the database looking for suspicious numbers.
“Contrary to the government’s repeated assurances, NSA had been running queries of the metadata using query terms that did not meet the required standard,” the judge wrote.
The opinion offered no further details on that assertion, but it appeared to contradict assurances by U.S. intelligence officials that there have been no major compliance issues with the telephone records database.
In a 2009 letter to Congress declassified several weeks ago, the Justice Department described “technical compliance problems” with the telephone data-collection program that the department said did not amount to “bad faith violations.”
In the ruling released Wednesday, Bates said the standards had been “frequently and systemically violated.”
The third violation, according to a statement by James R. Clapper, director of national intelligence, involved the bulk collection of U.S. Internet metadata. That data-collection program was discontinued in 2011. The portions of the court opinion describing that program were blacked out.
Clapper said the problems stemmed from “gaps in technical understanding among various NSA components,” which led to “unintended misrepresentations in the way the collections were described to the FISA court.”
The NSA realized in 2009 that “its compliance oversight infrastructure had not kept pace with its operational momentum” and increased its internal oversight, Clapper said.
In a related disclosure Wednesday, intelligence officials acknowledged that the NSA passes along to domestic law enforcement agencies some intelligence reports on Americans that are based on information collected through its Internet surveillance targeting foreigners. Such reports would be created if analysts discovered evidence of a crime or if the information was deemed relevant for intelligence purposes.
The officials declined to say how many reports are issued each year, but said that number is reported to Congress.
The NSA has broad power to collect information about non-Americans, but the Constitution and federal law limit the information a government agency can collect about U.S. citizens without first obtaining a warrant from a court.
Because of the way communications systems work, however, efforts to amass information on foreigners often will sweep in information about Americans. One example of why that happens is what a person sees when logging into a Web-based email account: A screen full of emails, including a few lines of content from each.
Those screen shots are transmitted over the Web as a single chunk of data, U.S. officials said, so if the NSA were lawfully targeting a foreign email address that came up in one such screen shot, the agency would get the whole thing, including American-to-American communications it wasn’t supposed to have.
When that happens, the NSA is supposed to follow court-approved rules to minimize the intrusion on the privacy of U.S. citizens or lawful residents. NSA officials say they cannot estimate how much data on how many Americans the agency has collected. By one extrapolation, the unlawful collection referred to in the 2011 court order may have involved as many as 56,000 wholly domestic communications a year — all without individualized search warrants.