The Times used the Panel Study of Income Dynamics for its analysis of income volatility among working families and for its examination of specific income-threatening events.
The panel study has followed a nationally representative sample of about 5,000 families and their offshoots for nearly 40 years and is the most comprehensive publicly available income and earnings database in the world. It is run by the University of Michigan and principally underwritten by the National Science Foundation.
The families' identities are kept confidential.
The Times based its analysis of income-threatening events on techniques developed during the 1980s by economists Richard V. Burkhauser of Cornell University and Greg J. Duncan of Northwestern University.
Their methods were modified for the newspaper by economists Ann Huff Stevens and Marianne E. Page at UC Davis. The paper employed a UC Davis graduate student, Sami T. Kitmitto, to help generate the findings. Stevens and Page supervised Kitmitto and advised the newspaper.
The Times chose to focus on seven types of events that people often experience in the ordinary course of a work life. They were: divorce or separation, a major decline in the work hours of a spouse, the death of a spouse, the birth of a child, a major decline in the work hours of the head of the household due to disability or retirement, a major spell of unemployment for the head of the household, and a major decline in the work hours of the head of the household because of illness.
Of families who experienced one or several of these events during a decade, the paper asked what fraction subsequently suffered a 50% or greater decline in annual income.
In examining these events, the paper faced certain constraints. The panel study was started during the 1960s, when most families had a single male earner. As a result, it has only limited information about the earnings, employment status and work hours of spouses during the early years, and to this day collects much more data about the head of the household than it does about any other family members.
Because of funding constraints, the study switched from annual data collection to an every-other-year schedule after 1996. This required the paper to examine events in two-year intervals in order to maintain a consistent approach both before and after 1996.
In addition, the paper could not make distinctions it would have liked to make between certain types of events - for example between drops in the work hours of the head of a household due to disability and due to retirement - because of the way that the panel study initially collected the information.
In cases of a decline in the work hours of the household head because of disability or retirement, as well as a decline in the work hours of a spouse, The Times defined "major" drops to be at least 520 hours - or the equivalent of three months of full-time work - in the course of a year.
In cases of unemployment of the household head, as well as a decrease in work hours of the head because of illness, "major" drops were defined to be at least 320 hours - or the equivalent of two months of full-time work - in the course of a year.
The newspaper took a number of steps to ensure that its examination was not distorted by data problems with the panel study.
For instance, it made certain adjustments to avoid double-counting events such as divorce. It also eliminated a sub-sample of Latino families that were added to the panel study in the early 1990s. Independent researchers have concluded that the sub-sample was poorly designed and that its inclusion could skew results.
In addition, the newspaper excluded the 2% of panel-study families with the biggest upward swings in incomes and the 2% with the biggest downward swings to ensure that those with extremely volatile situations did not throw off the results.
The paper also sought to test whether the declines in income of 50% or more truly threatened a families' living standards or - as with a drop from, say, $400,000 to $200,000 - were big but of comparatively little consequence. It did this by measuring the percentage of panel-study families that not only experienced income declines of 50% or more in each of the last three decades, but whose declines brought them to within 150% of the government's official poverty line ($28,215 today for a family of four).
The paper concluded that the percentage of families suffering this kind of life-altering hit jumped sharply and in tandem with the overall increase in families experiencing 50%-plus income declines.
To zero in on working families, The Times focused on men and women 25 to 64 years old whose households had some income.
The analysis looked at pretax income of all family members from all sources, including workplace earnings, investments, public transfers such as jobless benefits, food stamps and cash welfare, and private transfers such as inheritances.
All amounts were adjusted for inflation and expressed in 2003 dollars.
When it came to gauguing income volatility, The Times used techniques that were developed by economists Robert A. Moffitt of Johns Hopkins University and Peter Gottschalk of Boston College. The Times also consulted with Yale political scientist Jacob Hacker, who has conducted his own analysis of income volatility among panel-study households and has published results linking it to economic risk.
The newspaper employed two Johns Hopkins graduate students, Xiaoguo Hu and Anubha Dhasmana, to help generate results. Moffitt guided their efforts and advised the newspaper.
The Times analyzed the annual fluctuations of panel-study families' income in five-year increments from 1970 through 2000. It asked how each family's income made the journey from the beginning of a five-year period to the end.
For example, for a family whose income rose $5,000 over a five-year span, the paper examined whether the increase occurred in steady $1,000 annual increases or as a result of a big jump one year and a plunge the next.
The Times' basic finding is that the income fluctuations that individual families experience have grown substantially larger over the last three decades, and especially the last 25 years.
The newspaper interpreted this rising volatility as evidence that families are taking on more economic risk. In doing so, the paper took a page from the financial markets, where the chief measure of a stock's riskiness is how much its price bounces up and down relative to an overall market measure such as the Standard & Poor's 500 Index.
The newspaper described its volatility results in terms of upper and lower boundaries within which the annual income swings of two-thirds of families in the panel study fell. It included only two-thirds in order to avoid having its results distorted by a few extreme cases, such as the movie star whose career dries up overnight or the hourly worker who wins the lottery.
For a family to be included in its sample for any five-year period, The Times required there be financial records for at least three of those years. To avoid double-counting income in cases where families broke up, the paper added such payments as alimony and child support to households receiving them and subtracted the same amounts from those paying.
As with its examination of income-threatening events, the newspaper ran a series of tests to ensure that its volatility results were not the product of data problems but reflected real changes in the economy.
For instanct, the paper eliminated the Latino sub-sample. It also ran its numbers with and without adjustment for changes in the size of families over time. It ran them with and without transfer payments, which some researchers believe were poorly recorded for several years during the early 1990s. And it ran them with and without any data for 1993, the year about which reserachers have the greatest concern.
In every case where the volatility results with some families cut out from the sample were similar to results with those families included, the paper added back the removed families. The aim was to make the fewest possible adjustments to the database and to maintain the largest possible sample.