Why Is My Performance Less Than the S&P 500?

Trent asked me recently, “Why is my investment performance worse than the S&P 500?” Specifically, he had heard that the S&P 500 returned 25%+ in 2024. His annual performance in his 401(k) was 18%. Why the discrepancy?

(Remember that you can submit your questions to the show by emailing us at Questions@RetireMentorship.com or by visiting RetireMentorship.com/Question.)

Several people have asked similar questions. I reference the S&P 500 a lot on this podcast as a proxy for the performance of equities. (Episode 153: The S&P 500 Cracks 5,000 is one recent example of an episode directly about the index.) Because it’s such a popular index, many people compare their results to it and speak up when the results are worse.

There are four reasons why your performance may be worse than the S&P 500. Next week, we’ll answer the listener’s Question, “Can We Beat the S&P 500?” Answering why performance may be better or worse than the index can be very helpful to many. Here are the reasons they may be worse.

Investing Involves Fees

There are several layers of fees involved in investing.

Indexes vs. Index Funds

Stock Market Indexes cannot be invested in directly. You cannot buy an index. You cannot own the S&P 500. Third parties create indexes.

The S&P 500 lists the largest U.S. companies by market capitalization as compiled by Standard and Poor. The MSCI World and Emerging Markets indexes are lists created by Morgan Stanley Capital International. The Russell 2000 is maintained by FTSE Russell (Financial Times Stock Exchange). These companies maintain the lists, but you cannot buy into these lists.

You don’t invest in an index; you invest in a mutual fund or exchange-traded fund (ETF) that copies the index (an index fund). They attempt to mirror the index by buying the individual stocks of the companies that comprise the list created by these index companies (S&P, Morgan Stanley, FTSE, etc.).

Fund Fees

Funds have fees. The fees may be really low on an index fund (0.01% to 0.20%), but they remain. All fees are a drag on performance. If the S&P 500 index does 25.00% in a year and you buy an S&P 500 index fund with a 0.05% fee, you will have a return of 24.95%. Thus, you will always do slightly less than the index you are invested in.

Platform Fees

The platform you’re investing with will often have fees. Most people’s money is in their 401(k)s. 401(k)s have fees.

If you belong to a small company, those fees are going to be higher per participant to cover the administration and custody costs of contributing, holding, and investing your money. These can easily be 1-1.5% of your account balance. If the S&P 500 does 25% in a year, you’re going to get 23-24%.

If you are part of a large company, you usually get a “volume discount.” The fixed cost of administration is spread across thousands of participants, not dozens. You may pay $50 per year, and this would have little to no impact on your returns.

Many IRA and brokerage custodians charge fees for holding and trading your funds. That’s why we use and recommend Schwab. They have no account fees and no trading fees on all ETFs and a large list of mutual funds.

Advisor Fees

No one works for free. If you’re getting professional advice, you’re paying a fee. There are a bunch of ways Financial Advisors can charge fees, which we covered in detail in Episode 5: Five Questions to Ask A Financial Advisor. All those ways will reduce your return (some more than others).

I charge a fee. Don’t hide it. I’m proud of it. I believe and can show that the value of my planning and advice is worth a multiple of the fees I charge. But mostly, that’s because my fees don’t just cover investing. They cover all aspects of Financial Planning and the behavioral component. If you’re paying an advisor a big fee just to try and beat the S&P 500, good luck with that.

Fees Affect Performance

Fees, in a vacuum (more on that later), drag down performance. In some sense, we should always expect to do slightly worse than the index.

Diversification

“Diversification means always having to say you’re sorry.” (Brian Portnoy said it first.) Unless you’re only invested in an S&P 500 fund, you will never have returns that equal the S&P 500. Diversifying into multiple equity sectors means that some will do better, and some will do worse than the S&P 500 each year.

Year-Over-Year Differences

One of the dangers of investing is making decisions based on very short-term results (quarterly or annual performance). That can cause big mistakes. Here’s a great example in recent history.

In 2023, the Large Cap Growth Index was up 46%! The Large Cap Value Index was only up 9%. What a difference! Why are we invested in Value at all when growth is clearly king?

Hold your horses.

In 2022, the Large Cap Growth Index was down 33%! The Large Cap Value Index was only down 2%. What a difference!

$1,000 invested in each¹ at the beginning of those two years and held throughout would be worth $978 in Growth and $1,068 in Value. Despite Growth outperforming Value: 46% to 9% in 2023, you’d have been better off in Value over the last two years. You would have won by not losing.

So, which is better? Value or growth?

Neither! They both have great value over stretches of time. You cannot compare individual returns in individual years and use that as a basis for long-term investment policy.

So the S&P 500 did 26% last year, and you only did 18%. Did you look at the year before? Perhaps you were only down 10% while the S&P 500 was down 20%.

Asset Allocation

In general, equities outperform fixed income. The long-term average of equities is 10%. The long-term average of fixed income is 5%. The more fixed income you have in your portfolio, the less your return is likely to be.

Anytime the S&P 500 has a good year, if you have any fixed income in your portfolio, you will do worse. If you have a Target Date Fund, you will do worse. If you have a “Balanced Fund,” you will do worse. Finally, if you have a “Guaranteed Fund” or a “Total Bond Fund,” you will do worse.

Thus, you can never expect to equal, let alone beat, the S&P 500 if you have even the smallest amount of fixed income.

I’m not going to belabor this point. I talked about it all the time. The difference between fixed income and equities is the foundation of my book, 3D Retirement Income. If this is the first time you’re hearing about this, go get my book on Amazon or Audible or get it for free by becoming a RetireMember.

Behavior

Your investing behavior can be a big contributor to your “underperformance.” Bad investing behavior shows up more over long periods, halving the lifetime return of the average investor. Go listen to Episode 1: Investor vs. Investment Returns if you haven’t (or listen again if the last time you listened was 2021).

Bad investing behavior shows up in four primary ways:

1. Chasing Returns
2. Poor Diversification
3. Market Timing
4. Panic

Yes. These are the Four Horsemen, which we covered in Episode 4, The Four Horsemen, and in chapter nine of 3D Retirement Income. Personally, I think everyone should read or listen to this annually.

The Reason My Investment Performance Worse Than the S&P 500

Your investing performance may be “worse” than the S&P 500 in any given year for any combination of these reasons. I say “worse” and not worse (without quotes) because your multi-year performance may be better. Performance should be compared over decades, not years. Be careful about looking at your short-term returns.

Principles are Principles

If you’re a consistent listener, you may be noticing a theme here. It may feel as if you have already heard this episode before. The more you think about it, the more you realize it’s just Episode 111: Investing Order of Operations in reverse. You may notice elements from Episode 112: The Immense Stupidity of Investment Selection. And it may sound similar to Episode 133: When Should I Change My Investments?.

(If your performance was worse than the S&P 500 in 2023, or any year for that matter, all three of these episodes should be reviewed before making any changes.)

The reason these are all so similar is that everything we teach is based on principles and not predictions. Principles don’t change. They are immutable, apply to everyone, and are always true for all people at all times.

While they may not be sexy. They may be easily forgotten. But that’s why we come back to them over and over. The same principles can be used to answer all kinds of questions.

Get a plan. Stick to the plan. Stay the course.

1. Again, you cannot invest directly in an index. This example is a relative performance example. ↩

This article is educational only and is not intended to be investment, legal, or tax advice or recommendations, whether direct or incidental. Again, this is not investment advice. Consult your financial, tax, and legal professionals for specific advice related to your specific situation. Never take investment advice from someone who doesn’t know you and your specific situation. All opinions expressed in this article are those of the people expressing them. Any performance referenced is historical and is no guarantee of future results. All indices are unmanaged and may not be directly invested in.