Jeffries, who has led Abercrombie since 1992, is retiring immediately both as CEO and as a board director, the New Albany, Ohio, company said in a statement. Board chairman Arthur Martinez, along with three top-level Abercrombie executives, will manage daily operations until a new chief executive is appointed.
"I believe now is the right time for new leadership to take the company forward," Jeffries said in a statement.
A new CEO will face a daunting challenge to turn around the company's fortunes.
Abercrombie was once extremely popular among the teenage set, but it has recently suffered 11 straight quarters of same-store sales drops as its target customers have flocked to fast-fashion competitors such as H&M and Forever 21. Jeffries was removed as chairman in January after the company faced pressure from activist shareholders who wanted new leadership.
Shares of the company closed Tuesday up $2.11, or 8%, to $28.46.
Jeffries was a controversial chief executive who weathered intense criticism over the years for suggesting that Abercrombie's clothes were marketed only to thin and attractive people.
He is largely responsible for crafting the company's image of preppy clothing worn by beautiful, scantily clad young people -- a reputation bolstered by the models wearing swim shorts who have sometimes stood outside its stores.
In 2012, a manual that revealed the exacting and sometimes odd standards dictated by Jeffries was included as part of a discrimination lawsuit filed by a former Abercrombie pilot.
The "Aircraft Standards" manual mandated that male crew working on the retailer's Gulfstream jet, for example, are required to wear boxer briefs, flip flops and a spritz of the company's cologne. Servers must sport white gloves to lay a table and black gloves to move silverware. And the chief executive's dogs -- identified by name as Sammy, Ruby and Trouble -- are to be seated in different configurations depending on which of the three are flying.