A QUESTION OF JUSTICE: WHY ORANGE COUNTY DOESN’T MEASURE UP : Series Methodology: Who Compiled Data and How It Was Used

In examining prosecution records, The Times Orange County Edition used published criminal justice reports compiled by county, state and federal authorities, as well as computerized data compiled by the California Department of Justice.

Reporters compared the outcomes of 873,894 felony arrests in California’s five most populous counties--Los Angeles, San Diego, Orange, Santa Clara and San Bernardino.

All of the cases reached their final dispositions during the five-year period beginning Jan. 1, 1988, and ending Dec. 31, 1992, the latest year for which official data is available.

For the comparisons between counties, reporters applied the same methodology used by the state Department of Justice for its annual report, “Crime and Delinquency in California.”

Individual police agency reports of felony arrests, Orange County Superior Court reports of felony case filings, and felony conviction statistics compiled by the Judicial Council of California also were used.

The Times relied most heavily, however, on adult felony case information from the Offender-Based Transaction Statistics (OBTS), a state Department of Justice database that tracks what happens to felony arrests in California’s 58 counties.

Although the OBTS data is generally considered by researchers to be an enormous and statistically valid sample, it has limitations. The percentage of felony arrests accounted for varies from year to year, and from one county to another.

Law enforcement officials such as Orange County Dist. Atty. Michael R. Capizzi criticize the OBTS database as incomplete and therefore unreliable for comparison purposes, because the outcome of some arrests are never reported to Sacramento.

“We don’t put a lot of stock in” OBTS, Capizzi said. “The people who prepare the data say that it’s up to 40% [incomplete] and should not be used for comparative purposes. You [The Times] just throw caution to the wind.”

The Times and numerous experts consulted by the paper have found, however, that despite its limitations the OBTS data is sound and a reliable indicator of what is happening to felony arrests in a county’s criminal justice system.

At least 12 states rely on OBTS data like California’s for criminal justice planning. Many nationally known think tanks, including RAND Corp. and Abt Associates, which produce reports for the U.S. Department of Justice, also use the data for scholarly studies.

The reporting forms are initially filled out by police and then completed by the courts, if a suspected felon’s case gets that far. They are the basic documents used to create the state’s criminal history files--the “rap sheets” that chronicle arrests and convictions, as well as a convict’s terms of incarceration or probation.

Police and prosecutors regularly rely on these criminal history files, which they tap into electronically for criminal background checks.

Although Capizzi and others denounce the statistics, the district attorney has previously relied on the same OBTS data to compile a lengthy annual report comparing crime and punishment in Orange County with the other California counties that make up the top 10.

The most up-to-date report from Capizzi’s office (which was withheld from The Times until the newspaper filed a formal California Public Records Act request) notes that in 1992 Orange County was last among the 10 most populous counties in the percentage of felony arrests reaching Superior Court.

Although there is no way to determine exactly which arrests remain unaccounted for, the Justice Department arrived at a rough estimate in 1992 of 40% statewide--and 30% for Orange County--by dividing the number of adult felony arrests from one year into the total number of cases that received a final disposition in the same year.

Because many cases take more than a year to arrive at a final disposition in the courts, department officials acknowledge that this measurement can be misleading and might be statistically inaccurate.

Justice Department officials say using OBTS data to compare counties should be done with caution. But they temper that caution by saying that only comparisons involving “sparsely populated counties” may be invalid. To avoid that problem, The Times’ study compares only the state’s five most populous counties over five years.

When statistics compiled by California’s court system are compared to OBTS data, it shows that the Orange County OBTS data accounts for as much as 98% of the cases prosecuted as felonies.

Because the OBTS files record essentially three dispositions of a felony arrest (the person arrested was prosecuted on a felony charge, a misdemeanor charge or was released without prosecution), it appears that practically all of the “missing” cases are those of people prosecuted on misdemeanor charges or released without prosecution by police, who then didn’t bother filling out a complicated form reporting what amounts to a fruitless arrest to Sacramento. This failure to report releases appears to occur on average more than 900 times a year in Orange County alone.

To further test the validity of its statistical comparisons, The Times took its charts, as well as other data necessary to validate the underlying statistics, and presented them for analysis to a symposium of professors and postgraduate students in statistics at UCLA.

After that analysis was done, Richard A. Berk, a statistics professor who is also director of UCLA’s Center for Study of the Environment and Society, had this to say:

“Is OBTS good, and particularly with regard to serious crimes, the felonies? The answer is unquestionably yes. Unanimously, we’re saying yes. The comparisons you are making are therefore valid. And there’s nothing wrong with making such comparisons.”

Other statisticians and law enforcement researchers consulted by The Times also consider OBTS to be a statistically reliable sample of case dispositions in California.

Among those consulted were experts at the RAND Corp. in Santa Monica; Mark Cunniff, executive director of the National Assn. of Criminal Justice Planners in Washington, D.C.; Barbara Smith, a criminal justice researcher for the American Bar Assn.; UC Irvine Prof. Mark Baldassare; Robert Cushman, a justice expert and director of Santa Clara County’s Center for Urban Analysis; and Berk, who is also chairman of the Evaluation Advisory Committee for the U.S. Sentencing Commission.