In the past, investors had to pay an actively-managed mutual fund to manage their money. The average actively-managed mutual fund charges about 1% of assets under management. Hedge funds typically charge 2% of assets and 20% of profits. And the worst of the worst mutual funds charge up to 3% or 4% of assets. Now, index funds can give you a diversified portfolio for less than 0.1%.

One of the easiest and high-yield ways to boost your returns is to minimize your fees. But how much of your return is actually lost in fees? The answer may surprise even those who are already aware of the importance of minimizing fees. These fees really do add up and can be absolutely devastating to your investment returns.

Example: The typical actively-managed mutual fund

Assume that you start with a $10,000 investment with a 8% return. If there is 1% taken away in fees, as is the case for the average actively managed mutual fund, then you have approximately a 7% after-fees return. This happens, year after year, not just to your gains, but to your entire amount invested.

For example, $10,000 invested with an 8% return over 35 years becomes $147,853. On the other hand, $10,000 invested with a 7% return over 35 years (i.e., after 1% is taken away) grows to only $106,765. You lose four times your original investment in fees.

Example: The typical hedge fund

The typical hedge fund has even higher fees (approximately 2% of assets and 20% of profits, subject to high water mark rules). Assume that you start with a $10,000 investment with an 8% return. If 2% of assets are taken away, you’ve lost 25% of your return. If the hedge fund does not beat the benchmark, than you generally don’t have to pay a fee on profits.

For example, $10,000 invested with an 8% return over 35 years becomes $147,853. On the other hand, $10,000 invested with a 6% return over 35 years becomes $45,135. Because of fees, the investor ends up with less than a third of his money because he chose to invest with them.

Hedge fund fee structures are a little more complex than this, but it highlights the point that if the hedge fund is not able to beat the market (and most have not recently), then you lose an incredible amount of potential investment return to fees.

It gets worse: fees can compound as well

What’s even worse, if you think about it, is where do the fees go? They go into the pockets of the investment managers. They can then choose to invest it in the stock market and earn 8% (the expected stock market return) if they choose to. Your fees, as well as your investments, can earn compound interest.

Therefore, investment management fees are essentially a transfer of wealth from the investor to the fund manager. In our first example of the actively-managed mutual fund, where an initial $10,000 investment could be $148,000 without fees and $103,000 with fees, the fund manager ends up with the $45,000 difference, and you keep the $103,000.

Investment management fees are essentially a transfer of wealth from the investor to the fund manager.

In the hedge fund example where an initial $10,000 investment might become $148,000 with no fees, but $45,000 with fees, then after 35 years, the fund manager has $103,000, and the investor is left with only $45,000. The fund manager can theoretically end up with more of the investor’s money than the actual investor!

One way to think about it is to treat your investment portfolio like a pie. The pie will grow with time, but your slice of the pie gets reduced as you have higher fees and invest over a longer time period. In the hypothetical extreme hedge fund example, the amount of pie you get in retirement is 4x what you had 35 years ago, but you are only getting a third of the pie, while the fund manager gets over 2/3 of the pie.

John Ameriks at Vanguard illustrated this concept nicely in a table within a blog post back in 2011. I recreated this table, which measures how much of your returns are transferred to the investment managers based on a range or annual fee and time horizon assumptions, assuming an 8% market return. You see that in many scenarios (i.e. when the value is greater than 50%), the fund manager can end up with more money than the actual investor.

Where are the customer’s yachts?

There was a book originally published way back in 1940 called Where Are The Customer’s Yachts? Fred Schwed was a former Wall Street trader who became disillusioned by the industry after the 1929 stock market crash and published this classic story of a visitor to New York who noticed that the marina was filled with the yachts of bankers and financial advisors, but rarely their clients.

In some cases, financial advisors can end up wealthier than their clients. In an age where paying 1% for asset management was routine, this is not surprising. A financial manager can end up with more money than any single individual client, and if a manager can have tens or hundreds of clients, they can become very, very wealthy.

In the age of index funds and do-it-yourself investing, more physicians are cutting costs, enabling them to minimize the share of their money that goes to Wall Street and keeping more of the pie for themselves (which they can spend on yachts if they so desire).

Conclusion

It’s critically important to minimize fees in investing. You have little control over the daily movements of the stock market, but you can control your fees. And you should aim to minimize these fees, because fees are lost opportunities for additional returns. With particularly onerous fee structures, your investment manager may end up with more of your portfolio than you when you retire.

“Wall Street Physician,” a former Wall Street derivatives trader , is a physician who blogs at his self-titled site, the Wall Street Physician.

Image credit: Shutterstock.com