The Pentagon has sent special operations teams to gather intelligence and launched at least one airstrike. But the White House so far has resisted calls from some senior aides to escalate the U.S. military role in another Muslim country to counter the potential threat.
Spy satellites and reconnaissance drones have shown the militants building fortifications around Surt, on the central Mediterranean coast and on training bases for foreign fighters farther inland, the officials said.
A U.S. intelligence estimate this week concluded that Islamic State has attracted more than 5,000 fighters in Libya, double the official estimate last fall, making it the extremist group's largest and most potent affiliate outside Syria and Iraq.
Islamic State threatens to gain a "stranglehold" in Libya and potential "access to billions of dollars of oil revenue," Secretary of State John F. Kerry warned, one of several alarms the administration has raised in recent days.
He spoke at a Feb. 2 conference in Rome where the U.S. and 22 other nations agreed to support the formation of a unity government in Tripoli, the capital, in a tenuous effort to restore stability and take on the militants.
Libya has had no functioning central government since the North Atlantic Treaty Organization bombing campaign helped a popular uprising oust ruler Moammar Kadafi in 2011.
It has faced political chaos and a low-grade civil war nearly ever since, with two rival governments battling for control, and squabbling militias exploiting the power vacuum.
"In the absence of a true government, [militant groups] have grown unchecked," said a U.S. defense official, who was not authorized to speak publicly. "It's like Syria all over again."
Islamic State "has a bad habit of growing in places that are ungoverned," Tina Kaidanow, the State Department's counter-terrorism coordinator said last week at the Center for Strategic and International Studies, a Washington think tank.
She cited an urgency at the White House and among its allies to move quickly before the group expands beyond its current foothold, which extends about 100 miles east and west of Surt.
"We don't want to see the growth of [Islamic State] outpace what will be a long-term effort to build out a successful Libyan government," she said.
The group's rise comes as foreign fighters from Tunisia and elsewhere in Africa have moved to Surt and other strongholds in Libya, rather than to the war zones in Syria and Iraq, where the militants have suffered several military setbacks in recent months from the U.S.-led coalition.
The Pentagon wants to ensure that Libya doesn't "get on a glide slope" where it "gathers a piece of territory from which it's able to tyrannize people and plot operations elsewhere," Defense Secretary Ashton Carter told reporters Thursday at Nellis Air Force Base in Nevada.
"That's the situation we want to avoid in Libya," he said.
Any military intervention probably would be led by France and Italy, the former colonial power. Options include sending troops from Italy to help protect the new government, using U.S. and French advisors to train Libyan counter-terrorism forces, and launching airstrikes.
For now, British and American special operations teams and intelligence services have focused on identifying Islamic State leaders, assessing their networks and strongholds, and reaching out to local militias willing to fight them, officials said.
But intelligence officials say the militias are unreliable, poorly organized and divided by region and tribe, as well as by outside support, the same complex problem that has crippled U.S. attempts to unify opposition groups in Syria.
Islamist and Berber militias in the west, assisted by Qatar, for example, have engaged in brutal clashes with more secular forces in the east apparently led by former Kadafi loyalists and supported by Egypt and the United Arab Emirates.
Islamic State fighters have attacked oil fields and installations, hobbling a major source of income for the fledgling government. Oil production has fallen in half from a year ago, according to the state-run National Oil Corp.
The only known U.S. airstrike was in November, when F-15 fighter jets killed a senior Islamic State commander known as Abu Nabil near the eastern city of Derna. Officials said the Iraqi national narrated a video released last year that showed militants beheading 21 Egyptian workers, all Coptic Christians, in Libya.
Some national security experts warn that ISIS, a common acronym for Islamic State, is quietly taking root in Libya much as it did in Syria before the militants' blitz across Iraq in early 2014, seizing cities, oil fields, military bases and banks.
"I think it's increasingly a national security priority for us to limit the spread of ISIS in Libya given the expansion that's been seen recently," said Ben Fishman, former top National Security Council official on North Africa affairs. "It's obviously connected to an aggressive move from ISIS central in Syria to take advantage of the disorder in Libya."
Rep. Adam B. Schiff (D-Burbank), the top Democrat on the House Intelligence Committee, urged the White House to launch airstrikes against the militants' leaders. He warned that Islamic State could use Libya "as a gateway into Southern Europe."
Islamic State established a foothold in Libya in September 2014 when some militants who had fought in Syria returned home to join forces with extremist militias that had pledged alliance to the group's leader, Abu Bakr Baghdadi.
Several senior commanders also were sent from Syria and Iraq to help organize the expansion of the group's self-declared caliphate, a strict Islamic-ruled quasi-state.
It quickly became a breeding ground for exporting violence.
The gunmen who killed 20 people at a museum in Tunis, capital of Tunisia, in March were trained in Libya, local officials said. So were the gunmen who massacred 38 people at a Tunisian seaside resort in June.
To address the threat, Tunisia began building a security wall along parts of its border with Libya. U.S. officials have suggested that Egypt, Libya's eastern neighbor, take similar precautions.