Musk said the automaker is working through a number of issues with Gov. Jerry Brown's staff but fears the company would not get the various environmental and other permits in time to get the factory built and running quickly.
Tesla, which has been hampered by a constrained supply of batteries for its electric cars, needs to have the so-called gigafactory in production by 2017, about the time that it plans to launch a lower-priced model that will sell for around $40,000.
Its current car, the Model S sedan, sells for $70,000 to more than $100,000 depending on options.
"We can't afford to wait a year or more for permits to proceed," Musk said. "If we don't have the gigafactory online when we have the vehicle capacity online we would be in deep trouble."
Tesla previously said it had not considered California as a location for the factory, which it said was likely to be built in either Arizona, Nevada, New Mexico or Texas.
"Other states have a much more streamlined approach," Musk said.
Tesla plans to break ground on one factory in June and a second a month or two later, Musk said, adding that he wants to get the process started as quickly as possible. It's not clear if Tesla will wind up opening both facilities.
"It looks like they will start with two sites and make them compete on cost, execution and delivery and technology," said Craig Irwin, an analyst with Wedbush Securities.
A Brown spokesperson said: "The administration is working every day to bring companies to California and help them grow here. Tesla is certainly one of those companies."
Senate President Pro Tem Darrell Steinberg (D-Sacramento) noted that two years ago the Legislature passed and the governor signed a bill, AB 900, that streamlines the state's environmental review process. He speculated that this process could come into play if Tesla's proposed plant meets the criteria.
Also, a spokesman for Assemblyman Bob Wieckowski, a Democrat representing Fremont, where Tesla's auto factory is located, said his office heard about six weeks ago that Tesla was reconsidering staying in California and was in talks with Brown's office.
Tesla announced this year that it would spend $4 billion to $5 billion on one factory that could employ as many as 6,500 workers. Panasonic is a partner in the project.
Musk outlined the factory plans in a call with investment analysts after Tesla announced that it had a first-quarter loss and that sales of its Model X sport utility vehicle will be delayed until next year.
The Palo Alto automaker lost $49.8 million in the first quarter compared with a profit of $11.2 million in the same period a year earlier.
Last year's profit was helped by heavy sales of special California zero-emission-vehicle environmental credits to other automakers, a business that dried up in the latest period.
However, the company did earn $17 million in the latest quarter, compared with $15.4 million a year earlier when certain adjustments are made to its financial statement, such as factoring back in profit deferred for lease accounting.
Revenue increased to $620.5 million in the latest quarter compared with $561.8 million a year earlier.
Tesla shares fell $5.93, or 3%, in after-hours trading following the report.
The automaker produced 7,535 Model S sedans at its Fremont, Calif., factory during the latest quarter, a record for the company.
It delivered 6,457 vehicles to customers during the period and built up an inventory it will use to fill demand in Europe and China.
It is now manufacturing almost 700 vehicles a week, up 15% from the end of the fourth quarter and on the way to an expected 1,000 a week by year's end.
The electric-car company is demonstrating that it can competently ramp up production and operate consistently, said Thilo Koslowski, an analyst at Gartner Inc.
But Tesla should be concerned about the delay into next summer getting its second vehicle, the Model X, into showrooms, he said.
"Right now is a golden opportunity for Tesla to conquer the market, but they need more models," Koslowski said. "This ideal condition of not having competitors to worry about can't last forever."
The company might consider a face lift or refresh for Model S, a technique other automakers use to reignite interest in their models as they start to age, Koslowski said.
Musk said there were multiple reasons for the delays. The automaker spent more time than expected refining the Model S sedan, making incremental improvements to the vehicle and designing a right-hand-drive model that will sell in England, Japan and Hong Kong.
"It didn't make sense to focus on Model X if we didn't have our Model S house in order," he said.
During the call, Musk also said Tesla plans to launch production of electric cars in China and Europe in three to four years.
The cars built in those regions will be sold in those markets.
Times staff writer Marc Lifsher contributed to this report.