On Jan. 31, alarms alerted the control room at the San Onofre Nuclear Generating Station that a radiation leak was occurring in one of the nearly 39,000 tubes that carry radioactive water in the steam generators.
That failure led to an unparalleled shutdown of one of California's two nuclear power plants and triggered more than three months of detective work by Southern California Edison officials and federal nuclear regulators that has yet to determine the problem's root cause or when San Onofre will reopen.
Since then, the outlook for the plant has gotten worse.
Now, about 1,300 tubes — more than 3% of all the plant's tubes — have been taken out of service because of unexpected wear. And Edison has said that even when San Onofre reopens, it probably won't run at full capacity.
Figuring out the exact cause of the wear is crucial.
Edison finished installation of the $671-million steam generators less than two years ago, promising customers they would create major energy savings. Now, officials estimate it will cost as much as $65 million to fix the problems and tens of millions more to replace the lost power.
Officials said the investigation has yielded a better understanding about why the tubes are wearing down. When the plant is running at full power, the rate of steam flow is causing the tubes to vibrate, much like a guitar string being plucked. Some tubes vibrated enough that they rubbed against each other, causing the tube walls to wear down much faster than expected.
This is a potential danger because tube ruptures could release radiation and in extreme cases compromise the reactors' cooling system.
But officials have not yet pinpointed what missteps led to the problem.
Investigators set out to determine whether the problem was in the design, the way the equipment was built or installed, or the way it was operated.
Tube wear is expected to happen over time, and other plants also have had issues in new steam generators. But the rate and way in which it has happened at San Onofre is an anomaly, according to Nuclear Regulatory Commission officials and industry experts. The NRC has forbidden Edison from firing the plant back up until it submits a plan to fix the problems.
Several experts said the problems suggest a design flaw.
"This is very unusual behavior in steam generators, and I think the most likely scenario was that there was some error in the design," said Per Peterson, chairman of UC Berkeley's Department of Nuclear Engineering.
Peterson suspects the problem involves the design of support structures meant to prevent excessive vibration. He added that the solution suggested by Edison — running the plant at a lower power level — would probably address the vibration problem but would result in less power, cutting into Edison's revenues.
Sen. Barbara Boxer (D-Calif.) this week demanded documentation from the NRC and Edison about design changes for the new generators. Boxer is trying to determine whether Edison fully informed federal regulators about the design changes.
Determining what caused the flaw leads to the question of who will end up paying for the debacle: Edison and its shareholders, steam generator manufacturer Mitsubishi Heavy Industries or Southern California ratepayers.
Edison's contract with Mitsubishi includes a $137-million warranty that covers repairs but not the cost of buying energy on the market to replace the lost power.
Replacement power costs had already reached $30 million by the end of March, Edison reported, and will continue to rise if the plant stays out of service, or even if it runs at a lower power level for an extended period.
When California Public Utility Commission officials approved the price tag for replacing the steam generators, they assumed that the plant would run at an average of 88% capacity until its license expires in 2022.
An analysis at the time showed that a one-year outage or a scenario in which the plant would run at lower capacity for an extended period could make the project a money loser. But the PUC found those scenarios to be unlikely and determined that the project would probably be a good deal for ratepayers.
"If the plant runs at 50 to 80 percent capacity for the rest of its life, the entire cost-effectiveness analysis is turned on its head," said Matthew Freedman, attorney for advocacy group The Utility Reform Network.
PUC President Michael Peevey said that it is too early to say whether the tube problem makes the San Onofre project less cost effective in the long run but that his agency will ultimately deal with the question of whether Edison or ratepayers will have to pay the cost of replacing the lost energy.
"If Edison is found to have acted improperly or irresponsibly or without due regard for all the costs, Edison could end up paying for some portion of the replacement power," he said.
Edison officials have said it is unclear whether a mechanical fix could be implemented to allow the plant to run at full capacity again. But a number of experts said they doubted that would make economic sense. Replacing the steam generators would be even more costly.
"It seems really unlikely that they are going to do that again. I think that they are going to be running at a lower rating and there will be lawsuits about that," said Meredith Angwin, a former steam generator project manager at the Electric Power Research Institute.
The extended shutdown has also sparked debate about whether Southern California's power grid relies too much on San Onofre.
The plant can power about 1.4 million homes — it also plays a crucial role in "voltage support," facilitating the importation of electricity from elsewhere.
Energy officials say that with contingency plans in place, including bringing some mothballed generators at a power plant in Huntington Beach temporarily back online, Southern California should be able to get through a San Onofre-free summer without blackouts. But those fixes are temporary.
State Sen. Alex Padilla (D-Pacoima), chairman of the Senate's energy committee, said Edison and state officials should have had a backup plan before problems struck.
State officials, he said, need to talk about long-term scenarios for California's energy grid, including a future without San Onofre.
But replacing the plant may be difficult and politically problematic, experts said. Property owners tend to balk at unsightly transmission lines, and new fossil fuel plants raise air quality concerns.
Some have argued that San Onofre should be replaced by renewables, but experts on California's grid say that is not immediately practical.
"Renewables aren't a steady stream of supply. You have to have power to back up the renewables, and you have to have it in the right place," said Fred Pickel, an energy industry consultant recently appointed ratepayer advocate for the city of Los Angeles.