The Huntington Beach congressman said, "When you had hair."
Brown rejoined, "When I had hair, Methuselah was walking the streets," referring to the longest-living person in the Old Testament.
It was a lighthearted moment, and not Brown's first reference to his advancing age. Nor was it his first public reflection on years past as he has prepared to enter what is likely to be his final chapter of elected life.
Heading into a historic fourth term, Brown has been talking about both his mortality and his family's historic ties to California.
"Over the next four years, to the extent I have the ability, the physical and intellectual stamina and capacity, I'm going to do my utmost to live up to the promise of California that brought my great-grandfather, August Schuckman, here to Sacramento in 1852," Brown told reporters outside the governor's mansion on election night last month.
Brown said his family's story taught him about "grit, guts, imagination, being willing to sail into the unknown with the confidence you can not only survive but the confidence you can create something wonderful."
At 76, Brown is the nation's oldest governor. But that was not an issue in the campaign; he swamped his 41-year-old rival, Republican Neel Kashkari. In fact, Brown raised his longevity himself.
"We only are here for a while, and we pass on…. But what do we leave as we go?" he said to a pre-election crowd of hundreds in the rural enclave of Williams — a spot with zero electoral importance but one close to Brown's heart because it lies near Schuckman's mid-19th-century settlement.
"It's really important we know where we came from if we want to figure out where we're going," he said.
Brown is focused, at least in part, on endeavors that would outlast him: a bullet train linking the Bay Area with Los Angeles, a proposal for twin tunnels to move water around the state, bolder efforts to battle climate change. He ties these forward-looking projects to the pioneering drive that led his ancestors to California.
In a post-election news conference, Brown noted that Schuckman had fled Prussia, traveled by wagon train through hostile Native American lands and eventually settled outside Williams during the Gold Rush. He pointed to a picture of his great-grandfather feeding sheep.
"I put it on the wall up there because he's an inspiration — the pioneering spirit," Brown said. "It took courage to forge into the unknown.
"OK, today, it's not the open space, but we do have uncharted territory — of climate change, of bitter diversities of ideology, capitalism and a democracy that is … dispiriting to many people," Brown continued. "I think the opportunity for courage and pioneering and innovation is just as alive."
Brown's most famous relative was his father, former Gov. Pat Brown. His father's accomplishments weigh on the governor's mind as he thinks about how he would like to leave the state.
"I feel I have a lot to live up to," Brown said, "and I'm going to make sure that during these four years I maximize that opportunity."
Bruce Cain, a Stanford University political science professor, has observed Brown since his first terms as governor, from 1975 to 1983.
Busy running for president in some of those years, Brown "wasn't as interested in his legacy, other than he seemed to almost be the antithesis of his father," Cain said. Brown's hallmark was his ascetic tendencies, "trying to do good with efficient, small government."
The governor's current turn toward the past is partly "to cement the Brown legacy as the builders of California," Cain said.
Former Gov. Gray Davis, Brown's chief of staff from 1975 to 1981, noted that the governor is in a different phase of life now.
"He came on the scene as the new kid on the block, challenging conventional assumptions, some good, some not so good," Davis said. "Now he's a wise old oracle overseeing probably his last four years in elected office, definitely his last four years as governor."
Brown said in an interview that he has always been interested in his lineage — he recalled conversations with his late grandmother about the family's history — and cites his time in a Jesuit seminary to rebut the notion that he was not reflective when younger.
"I was pretty introspective in seminary. We could not talk for two years, not a lot of chatter. We could talk twice a day, for 20 minutes," Brown said. "But as you get older, your perspectives change. When you're 20 or 30, it seems more one way. Forty years later, it looks a little different, so that fosters some degree of introspection, that's true."
In recent years, he has traced his ancestors' descendants, discovering cousins and other relatives he didn't know he had. Several dozen gathered at the Schuckman ranch for lunch last year; some will attend Brown's inauguration Jan. 5.
The governor also recently asked state Librarian Greg Lucas to make state archives easier to search by genealogists.
"He's encouraged us here at the library to try and find some way so any Californian can go and research and discover their California story," Lucas said.
And Brown is spending more time at the Williams-area ranch, of which he owns 28%. The 2,500-acre property is home to cattle and honeybees. Wells have been drilled, and a house will be built.
Once retired, Brown said, he may spend the final years of his life on the land.
"The more I spend time on it, the more I like it. It's endlessly fascinating … the animals, the shapes of the mountains," he said, adding that he is researching the ranch's soils and grasses.
The method mirrors Brown's self-described approach to governing.
"I'm studying the land to understand how best to work with it — understanding what it was, what it could be."