Column: As Wall Street changed, one historic firm stood fast — for the better

Former President Truman and New York Gov. Averell Harriman toast with coffee cups
Then-New York Gov. Averell Harriman, with former President Truman in 1956, when Harriman was seeking the Democratic nomination for president. Principal in the private Wall Street firm Brown Bros. Harriman and heir to a railroad fortune, Harriman was nearing the peak of his national influence.
(Anthony Camerano / Associated Press)

It may be the most distinguished Wall Street firm you never heard of: Brown Bros. Harriman & Co. traces its lineage back to 1800, when Alexander Brown moved to Baltimore from Belfast and started a linen trading company.

In 1931, what was then Brown Bros. merged with two relatively new-money firms that had been founded by Averell and Roland Harriman, sons of the railroad tycoon E.H. Harriman.

The firm no longer boasts Browns or Harrimans among its partners, but it’s still a mainstay of the financial sector, one of the last surviving major private firms on Wall Street.


I think we need more firms with a culture that recognizes that, in a world where money is power and can unleash innovation and creativity, money can also be incredibly destructive.

— Zachary Karabell

It’s also the subject of “Inside Money: Brown Brothers Harriman and the American Way of Power,” a new book by financial commentator and historian Zachary Karabell. I talked with Karabell recently about the book, the firm, and the culture that guided Brown Bros. Harriman in the past and still does so today.

We talked about the firm as exemplifying the idea that serving clients can be the best means for a business to serve its own self-interest.

Our conversation has been edited for length and clarity.

Let’s start at the end. Brown Bros. Harriman is little talked about today. Where does it stand in the Wall Street hierarchy?

It’s true that if people are aware of the firm at all it’s because of a moment in the 20th century when it was part of the establishment of the establishment.

If you talk about them now, people either think they’re out of business or they’ve been bought or they’re a sad story of a firm that’s just eking out an existence.

The reality is they have 5,000 employees, they’re a global private partnership with maybe $2 billion in revenue, $500 million in profit, and if they were publicly traded they’d probably trade for between $10 [billion] and $20 billion. Compared to Morgan Stanley and Goldman Sachs, that’s modest. [Morgan Stanley is worth about $165 billion in market value and Goldman Sachs $125 billion.]


So what’s their main business?

They do a lot of the more obscure stuff of financial services, the workmanlike, low-margin custodial services for other people’s bonds and stocks, and foreign exchange and asset management. They do a lot of the stuff that needs doing but not a lot of the stuff that makes headlines.

How did the FDA got browbeaten into approving an Alzheimer’s drug that may not work?

June 10, 2021

You point out in your book that they’ve survived the ups and downs of Wall Street, even up to the housing crash of 2008, by doing the opposite of what the street is known for: They stayed focused on their clients, and avoided risky ventures instead of embracing them.

One of the positive insights we can learn from them is that risk is actually risky. Brown Bros. understood that to survive over the long term, it’s rarely a good idea to take outsized risks relative to what you can bear to lose. If your goal is to last over the generations and be in harmony with the world, then measured risk is what you want to look for, not outsized risk.

In many ways that’s the secret of their success — don’t try to have it all, because doing so usually leaves you exposed to having nothing.

So did they completely avoid railroads, the telegraph, and other major new technologies?

They did small investments in the railroad and the telegraph in the 19th century. But the emphasis was on small — they don’t take the lead, these investments are not the predominant basis of their business. They’re not betting the farm. From the 1800s to the present, this is a firm whose governing mantra was, if every night you go to bed, your books better be balanced so if you wake up in the morning and the world has changed for the worst, you’re prepared for it. It’s being ever-ready for a crisis. It’s the perfect pandemic firm.

What kept them away from promoters who were sucking in so many other big financiers?

A lot of these people they found culturally unsavory. Brown Bros. Harriman then and even today has a stiff culture of rectitude that’s chemically averse to the speculators or the more nakedly ambitious people. For them, the whole group of speculators in railroad bonds was kind of scuzzy.

One of your main themes is the firm’s culture, which was focused on serving their clientele with integrity. Yet they did do business in Germany between the wars, and their involvement in the cotton trade before the Civil War implicated them in slavery — though they were supporters of the Union cause.

There’s not a whole lot of hypocrisy — even in their involvement in the cotton trade, which depended on slavery, they were opposed to slavery and tried to get out of the cotton trade. That’s not an apologia for their involvement in slavery, which can have no apologia.

To win drivers’ support for Proposition 22, Uber gave drivers more control over their workdays. Now that the measure has passed, things are changing.

May 28, 2021

There’s not a trace or hint of scandal or black sheep or problematic business practices in their entire history, unless you believe some deep conspiracy theories about them in the 1930s, that they somehow funded everything that led to Nazi Germany. They had significant business dealings in pre-Nazi and early Nazi Germany, as did a zillion other American companies, but they were not part of some secret cabal to bring Hitler to power.

Everyone talks about the cliché that your word is your bond. Alexander Brown preached that in his letters to his sons, who became the Brown brothers.

You write in your book about the one plunge into a big speculative investment that went wrong, and perhaps taught them a lesson.

One moment in the 19th century when they kind of dip their toes into speculative waters was funding the first transatlantic steamship line to compete with Cunard. James Brown, a son of the founder, invested with this guy Edward Knight Collins, a sort of roly-poly self-made aggressive character — there was a reason why Collins and P.T. Barnum were friends; they were cut from the same cloth.

The other Brown partners were repulsed. They were saying, “We don’t want to be around that type of person with the mentality of I want to get mine, I want to get it all, I want to get it quickly and damn the consequences.”

And then ... ?

The line’s S.S. Arctic went down in 1854 in a collision. It was the Titanic of its era. Collins and James Brown lost multiple family members. The sinking of the ship sank the hopes of the line. The firm never really embraced that kind of speculative enterprise again.

Yet they ended up merging with the offspring of one of the great railroad tycoons of the late 1900s, E.H. Harriman. How did that happen?

They never would have merged with E. H. Harriman himself. But his sons, Averell and Roland, had started an investment firm in 1920, partially as an excuse for Averell to travel around the world and try to do deals. They were a new firm run by the two sons of a rich guy. And you had Brown Bros., which had been around more than a hundred years.

All its younger partners had all gone to Yale, and most of them had been in Skull and Bones — it was the clubbiest of the clubby establishment. Prescott Bush [father of President George H.W. Bush and grandfather of President George W. Bush] married the daughter of George Herbert Walker, who was the president of the Harriman company. and Robert Lovett, the son of Robert Lovett who managed the Harriman business, married the daughter of James Brown, so they all knew each other.

Wall Street is enticing investors with SPACs — funds that won’t say what they’re buying.

March 2, 2021

But they must have had a reason to merge.

Brown Bros. had bought a whole bunch of Argentine bonds at exactly the wrong time. And while Harriman had money, all the business was shrinking. You had a business without enough money and money without enough business. So they merged. The infusion of Harriman capital helped float the enterprise during the worst of the Great Depression.

This was also the period that led to their peak in political influence, right? Averell Harriman spent a term in the mid-1950s as governor of New York and years as a diplomat.

There was Prescott Bush. Robert Lovett [who became a partner of Brown Bros. Harriman in 1926] was probably one of the most important State and Defense officials from 1941 to 1952. JFK wanted Lovett to be his secretary of Defense or the Treasury, but Lovett recommended Robert McNamara and Douglas Dillon for those posts. They had huge political weight, but the world has moved on.

Do we need more firms on Wall Street like Brown Bros. Harriman?

We need more firms with a culture that recognizes that, in a world where money is power and can unleash innovation and creativity, money can also be incredibly destructive. At the center of a system that is controlling the flow of money, you do want more people culturally to be mindful of its capacity to destroy and not just tripping over themselves just to get more-more-more-more.

Some elements of speculative capital are really vital for the system, but you want that to be the spokes and not the hub. In the world today it’s the reverse. Too many firms at the center are trying to get more-more-more and the conservative, quieter parts are pushed to the periphery. That balance I don’t think is healthy.

You want there to be a belief that to those who’ve been given a lot, a lot is expected. The partners of this firm believe that, and I don’t think it’s hypocritical. It might have been self serving, but we should be able to believe that people can be attentive to their own good and also to the public good. The two are not inherently at odds. That culture is an important one for a more balanced capitalism and a more balanced society.