Patterson died Tuesday at her Brentwood home of complications from a broken hip, said her daughter, Megan Lee.
The actress was a veteran of Broadway when she was cast as Lois, the socialite who would not make it to the altar with Grant in "An Affair to Remember."
"She just loved the fact that she kissed Cary Grant the first day on the set," her daughter said. "She really loved Cary Grant."
The characters she played were often "brittle, overwrought ladies, notoriously glamorous, usually business-oriented, and more often than not, quite overbearing," according to an Internet Movie Database description that her family seconded.
After making her film debut in "Taxi" (1953), Patterson appeared in more than a dozen movies.
She played a worried mother in the well-reviewed "David and Lisa," a 1962 film about two teens with mental illness who fall in love. In the 1957 movie "Desk Set" she portrayed Spencer Tracy's prim, uptight computer expert and assistant.
On television, Patterson was the governor's secretary in the CBS sitcom "The Governor & J.J." that starred Dan Dailey and originally aired from 1969 to 1970. She also played a powerful matriarch opposite James Garner on "Nichols," a short-lived 1971 western on NBC.
Neva Louise Patterson was born Feb. 10, 1920, on a farm outside Nevada, Iowa, and was named for a friend of her mother's, not her hometown, she told the Des Moines Register in 2002.
Patterson, the daughter of a letter carrier and his seamstress wife, moved to New York City in 1938.
On Broadway, Patterson debuted in "The Druid Circle," one of 10 plays that she appeared in on the Great White Way between 1947 and 1959.
Originating the role of Helen Sherman in 1952 in "The Seven Year Itch" on Broadway was a highlight of Patterson's life, her daughter said.
During the play's run, the twice-divorced Patterson met James Lee, who worked in the prop department.
After they married in the late 1950s, the couple adopted their daughter as an infant and a 13-year-old Italian boy. Lee became a writer best known for contributing to the 1977 miniseries "Roots." He died at 79 in 2002.
She taught herself to speak six languages and always had a quip at the ready, her daughter said.
In the early 1980s, Patterson played an ambitious mother in the science-fiction NBC-TV movie "V" and its sequel, the miniseries "V: The Final Battle."
Fan mail and unexpected visitors continued to show up at her door, her family said. Last month, five "V" fans from France brought gifts and a request for her autograph.
In addition to her daughter, Megan Lee of Brentwood, Patterson is survived by her son, Filippo Quaretti-Lee of Florida; six grandchildren; and two great-grandchildren.