Colbert and the network have reached a five-year agreement, the broadcaster said.
Letterman, who has hosted the show for 21 years, last week announced his retirement on the program.
CBS did not give any details on the creative direction of the show once Colbert takes over, nor did it say where the series will be produced.
"I'm thrilled and grateful that CBS chose me," Colbert said in a statement. "Now, if you'll excuse me, I have to go grind a gap in my front teeth."