Crown Prince Felipe, 46, will succeed his father, Rajoy said. The 76-year-old monarch will speak to the nation later Monday. Rajoy did not give a time frame for the new king's coronation.
"The king is convinced that this is the best moment for a change in leadership of the state, with complete normalcy," Rajoy told a hastily scheduled news conference.
Juan Carlos has ruled since 1975, after the death of the military dictator Francisco Franco. For most of his reign, he has been beloved by Spaniards for shepherding Spain into a democratic era and putting down a military coup in 1981.
The king has been in ill health in recent years, including a broken hip in 2012 and a series of surgeries.
His popularity has also plummeted amid corruption allegations against his daughter and her husband. In 2012, Juan Carlos sparked public outrage by going elephant hunting in Africa at a time of deep economic recession at home. The trip cost several times the average Spaniard's annual salary.
The Spanish king's abdication comes about a year after a similar move by the queen of the Netherlands, who stepped down to allow her son to take the Dutch throne.