The missing data on Mutual Fund star rating

Sometimes missing data tells us a lot more about a question than the data that is presented.

To understand this, let’s start with an anecdote.

In World War 2, USA assembled a bunch of statisticians to evaluate which parts of the fighter planes needed more armour. As planes would come back from soirees they would count which part of the plane (engine, fuselage, etc) had been hit by how many bullet holes.

They found that, on the planes that returned, the engine area had 1.11 bullet holes per square foot while the fuselage had 1.73 bullet holes per square foot. The common wisdom was to add more armour where the planes were getting hit more.

That’s when a brilliant statistician, Abraham Wald, came with the insight, the extra armour does not go where there are more holes (fuselage), but where there are fewer holes (engine). The insight was simple, planes were able to survive hits on the fuselage and were returning back. But those planes which were getting hit more on the engines were not returning at all. The missing planes and the missing holes were the keys to solving the problem.

If your ratings are so good, where is the data that 5-star funds outperform 4-star funds?

So, what is the missing data in the Mutual Fund star rating show?

Say you are the CEO of one of the rating agencies and your Chief Statistician or Chief Information Officer comes to you and says – “Hey boss, we have conclusive data that our 5 star rated funds outperform our 4 star rated funds by 3% annually over a 5-year holding period”.

What would you do? You would, of course, publicize this as much as you could, make the ratings very expensive to buy and earn a lot of money from them. Or maybe open your own hedge fund and invest based on your rating methodology.

All the rating agencies employ a lot of smart statisticians and investment professionals and if they had the above data they would be publishing it, updating it and talking about it every quarter. It’s not that they are not looking for this data, it is just that what they have does not show any outperformance of higher rated funds. This is the missing data in the Mutual Fund star rating show. If your ratings are so good, where is the data that 5-star funds outperform 4-star funds?

So what do we have instead?

Well, Morningstar says:

“We have always been very clear that it’s not intended to predict future performance.”

Similarly, Valueresearch Online says:

“The assessment does not reflect Value Research’s opinion of the future potential of any fund. It only gives a quick summary of how a fund has performed historically relative to its peers.”

Both Morningstar and Value Research Online also disclose that star ratings are completely mathematical with no subjective inputs, and represent a fund’s risk-adjusted return over the past 3y and 5y with the best performers receiving 5 Stars.

Hmm… so another fancy way to talk about past performance, as if 3y and 5y return and information and Sharpe ratio were not enough. Just like returns then, maybe the rating’s should also come with their own disclosure – “Ratings are subject to market risk and past ratings do not predict future returns.”

The data that exists, confirms that ratings are not predictive of future outperformance. The Wall Street Journal in a review called “The Morningstar Mirage” concluded –

“Investors everywhere think a 5-star rating from Morningstar means a mutual fund will be a top performer—it doesn’t”

A similar study by Advisor Perspectives finds that the probability that a randomly selected five-star fund will outperform a randomly selected four-star fund is only 50.6%. Essentially a coin flip.

If ratings do not predict future returns, and rating agencies universally disclaim that ratings are not predictive, then how did they come to be so important in the Mutual Fund ecosystem?

The answer lies in human psychology. No one ever fired an advisor or a broker for recommending a 5-star fund. The ratings then are both a safety net and a shiny badge for the advisor to be sold to the investor. And if a 5-star fund underperforms, well no one else saw that coming. But if you picked a lower rated fund which underperformed, then you are not a good advisor.

From the same Wall Street Journal piece –

“Advisers get in trouble when they go against the grain. You isolate yourself more if you sell something else rather than just go with what research recommends.”

Since most advisors go with the grain and recommend 5-star funds, these funds see huge inflows creating reasons for AMC with such funds to unofficially bless the ratings in their own investor materials. And thus the myth of the Mutual Fund star rating continues.

In part 2, the missing data on advisor outperformance 🙂

This blog was initially published by CNBCTV18 in the Personal Finance section.

Start investing through a platform that brings goal planning and investing to your fingertips. Visit kuvera.in to discover Direct Plans and start investing today.

#MutualFundSahiHai, #KuveraSabseSahiHai!

You may also like...

5 Responses

  1. nitin says:

    Instead of just mentioning what others do wrong, why not talk about what you are doing right? What is Kuvera doing to highlight the best risk adjusted MF for instance? After all, pure returns alone can’t be the answer. The volatility matters.

    • Gaurav Rastogi says:

      This is not a critique of what others are doing. This is just an observation that investors put a lot of faith in rating agencies where as they themselves say they are just packaging past risk / return information.

      We do show information ratio, which is risk adjusted return that includes volatility. We also show “wisdom of the crowd” rating (Top Rated, or funds with a star), which is based on ratings, fund flows and risk adjusted return. At every stage the idea is to simplify information presentation.

      The point made here is different – all the MF data (volatility, return, sharpe ratio or information ratio) is essentially backward looking, so using that to make a forward looking decision is not a good idea.

  2. Aniket says:

    Brilliant insight and head on.

  3. Karthik Kanniyappan says:

    Historically, top 5 funds have changed every year in most of the segments which clearly means the ratings really doesn’t help much but don’t you think for an investor to begin with gives s good confidence to start investing from here ?? Overall… How does Kuvera come up with the suggested good funds based on a risk appetite ?? If not the formula which should obviously be confidential, what the ground theory you suggest ???

  4. Anup Deshpande says:

    Excellent article. Really appreciate it! Is Kuvera, in near future, introduce a metric which could predict MF’s future returns and also the comparison of future returns within the same category? As an investor, it would be of great help !

Leave a Reply

Your email address will not be published. Required fields are marked *