"There are no trumpets" at his home in the family residence of the White House, he said, adding that wife Michelle and daughters Malia and Sasha "tease me mercilessly."
Still, in a taping of "The Colbert Report," Obama jokingly acknowledged that he's not exactly a wilting flower in another wing of the White House, as he expands his use of executive power in controversial moves on immigration and climate.
Colbert reminded Obama that in 2008, he criticized excessive executive power under President George W. Bush, and pointed out that now, "You seem to have a whole lot of power."
"Does that happen to every president, where you get into the office and you think, 'Oh, you know what, I might be the only one I trust with this much power so I'll hold onto it?'" Colbert asked, in character as the conservative news anchor he plays on his talk show.
Good point, replied Obama.
"For the first time, you're asking a sensible question," Obama said. He said his preference would be to act on issues through legislation passed by Congress.
The comments came in a multi-segment presidential appearance on the show, as Colbert winds down the run of his show and prepares to take over for the departing David Letterman on CBS' "The Late Show."
Obama sidestepped a pointed question about whether he will approve the Keystone XL pipeline project and reverted to talking points on the importance of healthcare and immigration reform.
He laughed off the dismal outcome of the midterm election for his fellow Democrats, and took credit for rising employment figures.
Colbert brought up a recent jobs report, joking that Obama should have increased jobs before election day, not after, according to a pool reporter's account. At one point, Colbert joked that job numbers have been increasing because Obama has hired so many people to run the Pentagon: The president nominated Ashton Carter as his fourth Defense secretary last week.
"That's boosted our numbers a little bit," Obama said.
Colbert also asked Obama whether American jobs will be taken by the immigrants he plans to shelter.
"Baracus Maximus I," Colbert pronounced him.