Science Casts Doubt on FBI’s Bullet Evidence

The body of coin dealer Robert Rose was discovered in his Main Street office in South River, N.J., on a steamy July evening in 1995. He had been shot four times in the head.

There were no witnesses, no fingerprints, no gun.

But a chemical analysis of bullets by the FBI seemed to conclusively link the rounds that killed Rose to a box of cartridges belonging to one of his customers, Michael Behn. Lead in the different bullets bore the same telltale pattern of impurities, an FBI expert told the jury. Behn was convicted of the murder.

The same technique has been used in thousands of criminal cases over the last 30 years. Testimony by FBI experts about chemical “matches” between bullets has helped put hundreds of defendants behind bars across the country. In one Texas case, such testimony contributed to the conviction of an accused murderer, who was put to death.

Now, emerging scientific evidence has called the technique into question.

A Times examination of technical studies and trial transcripts -- and interviews with former FBI technicians, independent scientists and legal scholars -- suggests that the bureau’s use of evidence derived from the lead in bullets may be based on faulty assumptions that greatly overstate the importance of matches.

The FBI likens its lead technique to fingerprint analysis. Bullets found at crime scenes are tested for minute amounts of arsenic, tin, silver and other contaminants or additives. Those findings are compared with results of similar analysis of bullets found in the possession of suspects. FBI examiners have claimed in court to be able to link one bullet to others from the same production run -- even from the same box.

The technique has proved especially important in cases in which prosecutors have little or no direct evidence, such as fingerprints or an eyewitness identification.

There is no dispute that trace elements of chemicals can be precisely measured in bullets. The controversy centers on how the FBI interprets the data.

For years, FBI laboratory examiners operated on the assumption that each batch of bullet lead was unique. So if the same trace elements were found in the same concentrations in two bullets, the reasoning went, those bullets must have been made at the same time and in the same place.

Premises Questioned

Recent scientific studies have concluded that this premise is wrong. Studying blocks of lead used in the manufacture of bullets, researchers have found the same chemical makeup in batches made at different times. They also have reported that the concentration of trace elements can vary significantly in the same casting of lead.

If the skeptics are right, the matches found by FBI lab technicians are meaningless.

“The many technical flaws in the FBI’s bullet-lead analysis could well call into question hundreds of convictions,” said William Tobin, a metallurgist, former FBI crime lab examiner and coauthor of one of the recent studies.

FBI officials who supervise bullet analysis and the examiners who carry it out declined repeated requests for interviews.

In a statement, the bureau said: “We’ve been employing these methods and techniques for over 20 years in the FBI crime lab. They’ve been routinely subjected to vigorous defense scrutiny in the courts and we feel very confident that all of our methods are fully supported by scientific data.”

Nevertheless, individual FBI experts have begun to acknowledge that the technique is not as accurate as bureau officials have long maintained.

In a recent technical publication, and in court testimony last fall, FBI lab professionals said the bureau’s research showed that bullets from different batches of lead could be chemically indistinguishable, and that lead from the same source could vary in its composition.

Further evidence of deficiencies in the FBI method came from a statistical study commissioned by the bureau itself.

A team of Iowa State University statisticians studied FBI data on 800 bullets. They concluded that even in that small sample, many chemical matches between bullets could be the result of chance.

The researchers said that a much more extensive study would be needed to determine whether accidental matches are so common as to render the FBI method useless.

The new information has stirred the beginnings of a legal assault on the once-undisputed forensic technique. Some defense lawyers are hiring experts to challenge FBI testimony about analysis of bullet lead. In some recent criminal trials, they have won acquittals. Other lawyers are seeking to overturn old convictions based on bullet analysis.

A variety of forensic techniques, including polygraph tests, fiber analysis and fingerprinting, have come under legal challenge in recent years. The problem, experts say, is that such methods evolved from hunches, trial and error, and anecdotal evidence rather than from accepted scientific practice, which requires controlled experiments and rigorous peer review of the results.

“Scientists measure success by notions of reliability and validity,” said David L. Faigman, a professor at UC’s Hastings College of the Law in San Francisco. The FBI, he said, “measures success by usefulness. Convicting people is useful.”

The bureau recently commissioned the National Research Council, an arm of the prestigious National Academies, to conduct a detailed evaluation of its bullet-analysis method. The council will begin hearings today in Washington. It is not expected to issue a report for at least seven months.

Few are awaiting its findings with greater anticipation than Michael Behn, who is serving a sentence of 34 years to life. Behn said that when he was tried in 1997, his defense team could not find an expert to contradict the FBI’s bullet evidence.

“Nobody understands it and there’s nobody to go up against them,” Behn said from behind bulletproof glass in a visiting booth at the maximum-security New Jersey State Prison in Trenton.

“What they try to do is blind you, throwing all the science and technology at you. Even the judges don’t know what [the FBI is] talking about.”

Seeking New Trial

Last June, Behn filed a petition for a new trial based on recent challenges to the FBI’s bullet analysis.

Behn became a suspect in Rose’s slaying because he had visited the coin shop a few hours before Rose was killed. Behn, then a 28-year-old auto-supply dealer, maintained that he had bought $40,000 in rare coins and left.

But police were suspicious because Behn initially did not reveal that he had visited Rose’s shop that day. Behn said he did not want police to know that he purchased the coins with cash income that he had hidden from tax authorities.

Behn had no criminal record. His mother and a friend provided alibis that placed him away from the crime scene for all but a few minutes of the night of the murder. Behn contended that the cartridges found in his office were for a gun that he had reported stolen a year earlier.

The FBI crime lab compared the .22-caliber bullets that killed Rose with those from the box that belonged to Behn.

During the trial in Middlesex County Superior Court, Charles Peters, an FBI lab examiner, testified that the two sets of bullets contained nearly identical amounts of arsenic, antimony, copper, silver, bismuth and tin.

Peters said that based on this information, and packing codes on Behn’s ammunition box, it was possible that the bullets came from the same box and it was certain that they were manufactured on the same day -- April 1, 1988.

The testimony was the only evidence that seemed to link Behn directly to the killing. Jurors found it persuasive. Several said in recent interviews that without it, they might have acquitted Behn.

Anthony Tan, a member of the jury, said the bullet analysis was an important factor in his decision to vote for conviction. “The trace elements seemed to match,” he said.

When he sentenced Behn, Judge Barnett E. Hoffman called the evidence “particularly significant.”

Chemical analysis of bullet lead differs from the better-known forensic method of examining the grooves carved in bullets as they travel through a gun barrel. Each gun is thought to leave a unique “rifling mark.” But bullets recovered at crime scenes are sometimes too damaged to preserve such gouges. And many criminals dispose of their guns.

Lead analysis offered a way to solve crimes involving gun violence even when no gun could be found. It has often strengthened prosecutors’ weakest cases -- those where evidence is scarce or circumstantial.

The technique was developed in the early 1960s by the late Vincent Guinn, a leader in analytical chemistry and a revered figure in forensic science.

In contrast to steel or gold, which are manufactured in precise mixtures, bullet lead contains trace impurities such as silver, or additives such as antimony, used to increase hardness.

Guinn, then a scientist at General Atomic in San Diego, a manufacturer of nuclear reactors, found that these trace elements could be precisely measured through “neutron activation analysis.” Lead is irradiated by bombardment with nuclear particles, and trace elements are measured by the amount of radiation they emit.

Neutron activation required a nuclear reactor -- too costly for local or state law enforcement agencies. The FBI crime lab, using a reactor owned by the National Bureau of Standards, refined the method and became the only law enforcement agency in the country to use it.

In 1990, neutron activation was replaced by “inductively coupled plasma optical emission spectroscopy,” a less costly approach.

In that technique, examiners remove a sliver of lead from a bullet and dissolve it in acid. The solution is converted into a fine spray, then heated to 14,000 degrees Fahrenheit. Under such conditions, each element emits a unique wavelength of light that can be measured with a spectrometer.

The readings are extremely accurate, detecting a fraction of a part of silver per million parts of lead. This is like finding a single silver ping-pong ball in a living room filled to the ceiling with gray ones.

A Tool in Tough Cases

FBI crime lab examiners have testified to the precision of the technique.

In the 1997 North Carolina murder trial of Herbert Wayne Chavis, Peters said that bullets recovered from the victim’s body and those collected from the defendant’s car were manufactured at the same plant on the same day. Chavis was convicted and sentenced to life in prison.

On the witness stand, Peters was asked how rare it would be for two matching bullets not to have been made at the same plant on the same day. He replied: “I can’t put any statistics, but I could spend a lifetime looking for that.”

Other examiners have tied bullets used in a killing to a specific box of cartridges.

At a 1988 Texas murder trial, John P. Riley, then an FBI lab examiner, said: “From my 21 years of experience doing bullet-lead analysis, I can determine if bullets came from the same box of ammunition.... That is the case that we have here.”

As in the Behn case, prosecutors had little physical evidence against the defendant, James Otto Earhart, and no eyewitnesses to the crime of which he was accused -- the murder of a 9-year-old girl. Earhart, who had a history of violent behavior, was convicted of the killing and executed.

Even as FBI experts testified about bullet analysis in courtrooms around the country, some within the bureau’s crime lab had doubts.

William Tobin, who spent 24 years as an FBI metallurgist until he retired in 1998, said in an interview that he had long been “concerned that the premises used to support the practice seemed to contradict basic principles of metallurgy.”

In 1999, he was contacted by Michael Behn’s sister, Jacqueline, a professor of criminology at Bergen Community College in Paramus, N.J. She was trying to help her brother overturn his murder conviction and was looking for an expert to assess the bullet-lead evidence.

After reviewing Behn’s case and the testimony of FBI examiners in other cases, Tobin concluded that his doubts about bullet analysis were well-founded. Tracing a bullet back to a single batch or box contradicted what he knew about the melting and casting of metals -- a mammoth industrial process that he believed defied such certainty.

Tobin approached Erik Randich, a metallurgist at Lawrence Livermore National Laboratory in Livermore, Calif., who had a particularly sophisticated knowledge of lead composition. Tobin knew Randich from their collaboration analyzing bomb fragments in the Unabomber investigation years before.

They decided to conduct a scientific evaluation of bullet-lead analysis. They enlisted the cooperation of two scientists from companies that are leading suppliers of lead to the ammunition industry. The resulting study, which they did without pay, was published last year in the journal Forensic Science International.

Tobin, Randich and the two company scientists evaluated data provided by the lead suppliers and concluded that a key FBI assumption -- that each batch of lead is unique -- was mistaken. Batches of bullet lead produced more than a decade apart had the same composition of trace elements, the researchers found.

FBI experts have testified repeatedly in court that each time lead is melted and cast, its composition changes just enough to create a unique chemical signature for each batch. But the four researchers found that there is no such signature. Trace elements are often distributed unevenly within each casting, they reported, so that entire batches rarely bear a uniform chemical profile.

As molten lead is cooled, the trace elements tend to concentrate, clumping silver, arsenic or tin toward the center, the study found. As a result, the researchers said, bullets created from the same block of lead can appear unrelated and bullets produced years apart can appear virtually identical.

Even if the FBI was correct in assuming that each batch has a distinctive chemical composition, the bureau’s conclusions would still be wrong because of a misunderstanding about bullet production, the authors said.

FBI examiners have testified that bullets are often smelted from batches of lead as small as 70 pounds -- enough to make about 10,000 .22-caliber bullets.

But according to the study, ammunition is usually created from lots of lead that weigh 50 tons or more and yield up to 17 million .22-caliber bullets -- a number that would dramatically dilute the significance of any match.

The size of lead castings was a crucial issue in a recent Kentucky case involving the shooting death of a University of Kentucky football player.

FBI lab examiner Kathleen M. Lundy compared a bullet recovered from the victim’s body with bullets found in the defendant’s possession. Lundy testified that they came from the same batch. Her description of the manufacturing process at the Winchester Co. in East Alton, Ill., suggested that the batch was no larger than 280,000 bullets.

Her testimony was contradicted by a Winchester employee who testified later in the trial. The defendant, Shane Ragland, now 28, was convicted and sentenced to 30 years in prison.

The conflict in testimony sparked the interest of the U.S. Justice Department’s inspector general, whose investigators questioned Lundy last spring. In a sworn affidavit, she admitted that her trial testimony was untruthful and that the manufacturing batch was many times larger than she had suggested. The number of bullets made from it was not around 280,000 but in the tens of millions.

Lundy blamed her conduct partly on a sense of crisis in her work, fed by “new and repeated challenges to the validity of the science associated with bullet lead comparison analysis.” She was indicted by a Kentucky grand jury last week on charges of false swearing, a misdemeanor.

Ragland’s lawyers have filed a motion for a new trial based on Lundy’s admission.

In a recent technical article, FBI researchers modified the bureau’s once-firm position on bullet analysis, conceding that different compositions of trace elements were discovered within a single source of lead.

Last October, in a hearing on the Ragland case, Lundy acknowledged under questioning that bullets that appear chemically identical can come from different batches of lead.

As criticism of the bullet technique has circulated in legal circles, a few defendants have hired experts to counter FBI testimony.

In the spring of 2001, Peters, of the FBI crime lab, testified at the trial of Jose Mateu Jr., 18, an Alaskan accused of killing his father. Peters said that bullets from the victim and those recovered from Mateu’s house “were made from the same molten source of lead.”

He added: “We have looked at different sources of lead and never found two to be exactly the same.”

Then Tobin and Randich each took the stand as paid defense witnesses. They said that Peters’ conclusions had no scientific foundation.

The trial ended in a hung jury, as did a retrial. Mateu’s third trial is scheduled to begin today.

Richard Saferstein, former chief forensic scientist for the New Jersey State Police, said in a recent interview that the problem with bullet-lead analysis is not with the lab work but with the conclusions drawn from it. The FBI, he said, has tended to overstate the significance of such evidence.

“It’s the image that the FBI projects, of course, more than the technology. That image, when you are talking to jurors, is impressive,” he said. “You can’t abuse that privilege. It comes back to haunt the agency and the profession.”

Charles Piller is a Times staff writer. Robin Mejia is an associate of the Center for Investigative Reporting in San Francisco.