The Source of the Statistics and How They Were Analyzed

The Times employed the Panel Study of Income Dynamics for its analysis of income volatility among working poor families.

The panel study has followed a nationally representative sample of 5000 families and their offshoots for nearly 40 years, and is the most comprehensive publicly available income and earnings database in the world. It is underwritten by the National Science Foundation and run by the University of Michigan.

The Times used techniques for gauging volatility that were developed by economists Robert A. Moffitt of Johns Hopkins University and Peter Gottschalk of Boston College. The Times also consulted with Yale political scientist Jacob Hacker, who has conducted his own analysis of income volatility among panel-study households and has published results linking it to economic risk. The newspaper employed two Johns Hopkins graduate students, Xiaoguo Hu and Anubha Dhasmana, to help generate results. Moffitt guided their efforts and advised the newspaper.

The Times analyzed the annual fluctuations of panel-study families’ income in five-year increments from 1970 through 2000. It asked how each family’s income made the journey from the beginning of a five-year period to the end.

For example, for a family whose income rose $5,000 over a five-year span, the paper examined whether the increase occurred in steady $1,000 annual increases or as a result of a big jump in one year and a plunge in the another.

The Times’ basic finding is that incomes fluctuations that individual families experience have grown substantially larger over the last three decades, and especially the last 25 years.

The newspaper interpreted this rising volatility as evidence that families are taking on more economic risk. In using volatility to gauge risk, the paper took a page from the financial markets, where the chief measure of a stock’s riskiness is how much its price bounces up and down relative to an overall market measure such as the Standard & Poor’s 500 Index.

To focus on working families, the Times looked at panel study men and women ages 25 through 64 and their children whose households had some income. To focus on the working poor, the paper ranked families by their average income during each five-year period. It then zeroed in on those in the bottom one-fifth of income earners and especially those at 20th percentile or the borderline between the bottom fifth and the next-higher fifth. The average annual income of panel-study families at 20th percentile is close to the government’s official poverty line for a family of four most years.

The newspaper described its results in terms of upper and lower boundaries within which the annual income swings of two-thirds of families in its sample stayed. It included only two-thirds to avoid having its results distorted by a few extreme cases, such as the movie star whose career dries up overnight, or the hourly workers who wins the lottery.

The newspaper looked at income of all family members from all sources, including workplace earnings, investment income, public transfers such as jobless benefits, food stamps and cash welfare and private transfers such as inheritances.

Because of weaknesses in the panel study data, the newspaper did not take into account taxes. All amounts were adjusted for inflation and expressed in 2003 dollars.

For a family to be included in its sample for any five-year period, the Times required that there be records on that family for at least three of those years. To avoid double-counting income in cases where families broke up, the paper added such payments as alimony and child support to households receiving them and subtracted the same amounts from those paying.

The newspaper ran a series of tests to determine whether its results were the rpoduct of data problems or reflected real changes in the economy, and to address concerns about some aspects of the panel study that have been raised by specialists.

The paper was forced to eliminate a sub-sample of Latino families that were added to the panel study in the early 1990s. Panel study officials and outside researchers have concluded that the sub-sample was poorly designed and the newspaper was concerned that its inclusion would distort results.

The paper ran its numbers with and without a separate immigrant sub-sample, as well as with and without adjustment for changes in the size of families over time.

It ran them with and without transfer payments, which some researchers believe were poorly recorded for several years during the early 1990s. It also ran them with and without any data for 1993, the year about which reserachers have the greatest concern.

To double-check that its results were not the product of a few outliers whose incomes either rose wildly or fell precipitously, and instead reflected a broad trend among most families, the paper ran its numbers trimming the top and bottom 2% of families and, separately, the top and bottom 3% of families.

In every case where the results with some families cut from the sample were similar to those with those families included, the paper added back the removed families in order to make the fewest possible adjustments to the database and to maintain the largest possible sample.