In very candid fashion, Griffin admitted to Stern: "I probably shouldn't have taken the gig."
She also said: "I don't really feel it was my best work, I'll be honest. I stand by my specials and my series. I feel like the lane that that show was in was so specific that they weren't able to make the transition to the comedic lane that I work in."
Though she had only been host of the series for less than two months -- replacing Joan Rivers, who died last September -- her time on the series was chaotic.
Co-host Kelly Osbourne abruptly quit in February after the Oscars after fellow panelist Giuliana Rancic made a joke about singer-actress-dancer Zendaya's dreadlocks smelling of weed.
Rancic issued an on-air apology for the joke, but according to Griffin, who said she wasn't around for any of the drama surrounding Osbourne and Rancic, said she was "a little surprised" by Osbourne's departure.
Stern suggested that possibly Osbourne was already looking to leave the show and used her outrage over a joke about her friend as the reason to leave.
"I do kind of get that feeling," Griffin said. She also wondered if Osbourne was still missing the loss of her friend, Rivers.
As for the behind-the-scenes culture of the show, Griffin said that it was a production filled with people ready and willing to give info to gossip sites.
"From the jump, even before I started, I've never worked on a show with so many leaks," Griffin said. "I got a little paranoid."
Clips from Griffin's appearance with Stern can be heard here.