Ranking the factors that help or hurt your investment returns

Ranking the factors that help or hurt your investment returns
Nothing has a bigger effect on investment returns over the long run than the behavior of investors under duress, such as during stock market crashes. (Richard Drew / Associated Press)

What drives the returns of any investment portfolio?

Specifically, from the moment someone starts saving for retirement, until the day he or she begins to take the required minimum distribution at age 70½, what are the factors that determine just how successful that portfolio is in terms of net, inflation-adjusted returns?


This is a more challenging question than you might think. Ask professional investors, and the responses cover a gamut of inputs, ranging from corporate profits, the economy, risk, valuation, taxes, interest rates, sentiment, inflation and more.

An unexpected challenge in performing this exercise is a tendency for some elements to offset others. For example, changes in profits could be offset by widening or contracting price-earnings ratios; sentiment might offset valuation; returns tend to vary inversely with risk. Why does this matter? Because in the real world, one hand giveth while the other taketh away. This concept of cancellation matters a great deal to total portfolio returns.

And so we are left with an intricate and difficult question. This is why complex, multivariate systems are so hard to assess by traditional analysis. What follows is my attempt to identify seven broad elements that typically determine the total return of any portfolio. (Note that these elements progress from the least meaningful over a course of a lifetime to the most. Any given latter item can cancel out the effect of earlier ones.)

On to the list:

No. 7. Security selection: Stock picking is what many individual investors and much of the media like to focus on. It’s a rich vein to consider, with traditional elements of narrative and storytelling, winners and losers. No doubt, better stock pickers will see commensurate portfolio gains. But that is merely one element of many, and not surprisingly, subject to other factors.

No. 6. Costs and expenses: The overall cost of a portfolio, compounded over 20 or 30 years, can add up to (or subtract from) a substantial amount of the returns. One Vanguard Group study noted that a 110 basis-point expense ratio can cost as much as 25% of total returns after 30 years. That does not take into consideration other costs such as trading expenses, capital gains taxes or account location (i.e., using qualified or tax-deferred accounts).

The rise of indexing during the last decade is a tacit acknowledgment that on average, cost matters more than stock-picking prowess.

No. 5. Asset allocation: What is the optimal ratio of stocks, bonds, real estate investment trusts, alternates and cash in a portfolio? Academic studies have proved that allocation is much more important to returns than stock selection. You can imagine all sorts of scenarios in which allocation trumps selection. The greatest stock picker in the world with a 20% equity exposure won’t move the needle very much.

No. 4. Valuation and year of birth: Valuations will fluctuate over the life cycle of any bull or bear market. However, for long-term investors, valuations are less about expected returns of pricey stocks and more about when they a) start investing and b) start to withdraw in retirement.

Much of this is random and beyond your control. Imagine the market crashing just before your prime saving and investing years; that should have a positive effect on net returns over time. What about someone who retired in 2000 and began withdrawing capital after the market got shellacked? That will also have an effect.

Those people born in 1948 not only managed to have their peak earning and investing years (ages 35 to 65) coincide with multiple bull markets and interest rates dropping from more than 15% to less than 1%. They also lucked into a market that tripled in the decade before retirement.

No. 3. Longevity and starting early: Having a long investing horizon is determined by many factors, including your longevity. How long you live is going to be a function of genetics, lifestyle and dumb luck. But when you begin saving for retirement is not a function of genetics or health. The sooner you begin, the longer compounding can work its magic.

No. 2. Humility and learning: We all begin as novice investors. Everyone makes mistakes — even the greats like Warren Buffett and Vanguard founder Jack Bogle. The key question is how quickly you can figure out all of the things you are doing wrong. Self-awareness and ego are a significant thread in this context. The sooner we learn to learn from our mistakes, the better our investment portfolios.

No. 1. Behavior and discipline: Nothing has a bigger impact than the behavior of investors under duress. I stumbled upon this observation early in my career as a trader; everything I have learned since has served to confirm it.


We see this again and again in the data — just look at DALBAR’s Quantitative Analysis of Investor Behavior. Investors continue to be their own worst enemies when it comes to investment performance — buying as stocks hit greed-driven highs and selling on panic-inducing lows. On average, their actions lower their returns significantly, but in the worst cases they demolish them. Even worse, behavior is (or at least should be) within their own control.

Bonus: Luck and random chance: There is a lot of random chance in investing. We often cannot tell the difference between skill and luck in stock selection. And the moment when we each realize this can also be somewhat random.

This list might help you consider what changes you might wish to make in your portfolio. At the very least, you might recognize the areas you are over- and under-emphasizing. Your investment returns will thank you.

Ritholtz writes a column for Bloomberg.