Robertson, who also played a real-life role as the whistle-blower in the check-forging scandal of then-Columbia Pictures President David Begelman that rocked Hollywood in the late 1970s, died at Stony Brook University Medical Center on New York's Long Island, according to Evelyn Christel, his longtime personal secretary. A family statement said he died of natural causes.
PHOTOS: Notable deaths of 2011
More recently, he played Uncle Ben Parker in the "Spider-Man" films.
Throughout his career, Robertson worked regularly in television, including delivering an Emmy Award-winning performance in "The Game," a 1965 drama on "Bob Hope Presents the Chrysler Theatre."
Robertson first came to filmgoers' attention playing Kim Novak's wealthy boyfriend in "Picnic," the 1955 romantic drama starring William Holden.
While continuing to work on stage and in television, Robertson starred in a string of movies over the next seven years.
He was Jane Powell's love interest in the 1957 musical comedy "The Girl Most Likely," an Army lieutenant in the 1958 World War II drama "The Naked and the Dead," a surf bum (the Kahuna) in the 1959 Sandra Dee surf-and-sand epic "Gidget" and a doctor in the 1962 drama "The Interns."
Despite his continued work in films, Robertson failed to reach the top level of stardom — or, as he put it in the early 1960s, he had yet to make it to "that golden circle of three or five [stars] in Hollywood who can pick and choose" their movie roles.
Robertson appeared to be on his way after President Kennedy personally approved him to star in "PT 109," the 1963 film about Kennedy's heroic World War II exploits in the Navy as a motor torpedo boat skipper in the South Pacific.
Although described in a Look magazine cover story as "The Big Epic that Robertson's career has always needed," "PT 109" ultimately didn't do much to advance the screen career of what the magazine called "one of the finest young actors in America today."
In contrast to most of the movies he was making at the time, the stage and television brought Robertson some of his choicest roles.
On Broadway in 1957, he played the lead role of the drifter in Tennessee Williams' "Orpheus Descending," the same role Marlon Brando played in "The Fugitive Kind," director Sidney Lumet's 1959 movie version of the play.
And on television, he played an alcoholic in the critically acclaimed 1958 "Playhouse 90" production of "The Days of Wine and Roses," only to see Jack Lemmon land the role in the 1962 movie version.
Frustrated by seeing roles he originated go to major movie stars for the film versions — "I had a reputation then for being the bridesmaid, never the bride" — Robertson wasn't about to let it happen again when he starred in the 1961 production of "The Two Worlds of Charlie Gordon" on "The United States Steel Hour."
While still in rehearsals for the TV role that would earn him an Emmy nomination, Robertson bought the movie rights to the story of a mentally disabled man who undergoes an experimental brain operation that increases his IQ but later discovers that the life-altering changes won't be permanent.
Robertson continued to appear in films, notably playing a ruthless, conservative presidential candidate in "The Best Man," a 1964 drama co-starring Henry Fonda. In the meantime, he continued to do research for the title role in what became the movie "Charly."
"I would visit shelter workshops," he recalled in a 2006 interview with The Times. "I spent a lot of time with retarded people on both coasts."