Both are seasoned, popular comedians. Both wear glasses, as Letterman noted.
And both have gotten into a bit of hot water when they've stepped from their familiar gigs.
Colbert created quite a stir when he performed as the main entertainment at the 2006 White House Correspondents Dinner. The "Colbert Report" host delivered a nearly 20-minute address in character as the Bill O'Reilly-like commentator he plays on the Comedy Central show, and the speech drew a decidedly mixed reception.
The routine largely targeted then-President George W. Bush and First Lady Laura Bush, who were seated just a few feet from the podium. Colbert made fun of Bush's 68% disapproval rating, and he made a caustic reference to Bush's "Mission Accomplished" appearance on the USS Abraham Lincoln, the site of the collapsed World Trade Center and in cities ruined by Hurricane Katrina.
"I stand by this man," said Colbert. "I stand by this man because he stands for things. Not only for things, he stands on things. Like aircraft carriers, and rubble, and recently flooded city squares. And that sends a strong message: that no matter what happens to America, she will always rebound, with the most powerfully staged photo ops in the world."
The routine drew a rather chilling reception from the audience, and an unsmiling Bush was shown in news footage. However, clips of Colbert's speech were hugely popular on the Web.
Letterman also divided audiences when he hosted the Academy Awards in 1995. Although much of his material drew laughs, his "Uma, Oprah" riff introducing Uma Thurman and Oprah Winfrey fell flat. Letterman later said he felt that he and his writing staff had an off night: "I don't think any of us felt like heroes afterwards."
Still, the show drew a large audience and was the highest rated Oscars since 1983.