Rating the ESG rating agencies

As reliance increases on the rating agencies that evaluate companies’ performance on environmental, social and governance metrics, there is a growing need for tougher scrutiny, regulation and reform of the raters’ own practices.

Investors around the world rely increasingly on ESG rating agencies, which have become influential in guiding the decisions of retail investors, asset managers and corporate managers alike. So, as the business of rating has grown, and technology brings disruption, regulators on both sides of the Atlantic are preparing new rules.

The agencies cater to demand from investors who want to reduce asymmetries in information about the companies in which they invest. They inform investors about which businesses treat employees well, consider the environment and take sensible measures to maximise the positive value they create for society.

ESG raters harness economies of scale, because evaluating a company on such diverse and intangible issues requires time and expertise. The ratings provide an efficient solution to a very common difficulty. Yet there are several serious problems in the industry today that need to be addressed.

The first problem is obvious and easy to fix: conflicts of interest. ESG ratings are commonly paid for by investors, which generally avoid such conflicts. However, some agencies also offer consulting services to the companies they rate on how to improve their scores. Regulators must bar this practice to ensure impartiality and trust.

FT Business School Insights: Sustainability

Research by leading professors, features and academic and business opinion. Read the report here.

The second problem is that ESG raters disagree. This is not so easy to fix. Rival ratings often provide divergent assessments of the same company, which causes confusion. However, the fundamental problem is not disagreement per se but rather that it is hard to make sense of the disagreement. There are legitimate reasons to disagree about ESG performance that need to be reinforced, but others should be removed.

One legitimate reason for differing ratings is their varying objectives. Some emphasise impact materiality, which assesses pollution based on its ecological harm. Others concentrate on financial materiality, whereby pollution is a concern only if it incurs costs for the company. This fundamental difference will obviously lead to different rating outcomes.

A fund managed using a rating focused on financial materiality should not be marketed as sustainable

The problem is that both users of ESG ratings and the raters themselves do not clearly discriminate between these objectives. It is convenient to have it both ways: doing good for the environment while doing well financially. But mixing these two aims in one rating obscures potential trade-offs and may compromise the achievement of either.

The solution is greater transparency about what a rating seeks to measure, and the methodology behind it. Users should be able to scrutinise the way the data is collected and aggregated, and whether that is consistent with the rating’s objective.

Both raters and users need to better differentiate between impact-oriented and financially oriented ratings. For example, a fund managed using a rating focused on financial materiality should not be marketed as a sustainable investment fund. It is simply an investment fund that uses an expanded information set.

The problematic reason why ESG ratings disagree is that they rely on different underlying data. Even if two raters pursue the same objective and follow the same aggregation methodology, they will still generate different results if they use different input data. Our research paper, Aggregate Confusion: the Divergence of ESG Ratings, suggests that more than half of the disagreement between ESG ratings is due to such differences.

Until reforms happen, investors need to recognise this issue and work round it. In our most recent project, ESG and Stock Returns, we show that it can help to use several ESG ratings together to remove noise in the data. Meanwhile, some investors have developed their own in-house ESG assessment models to compensate.

In the future, regulators should not attempt to synchronise ESG rating methodologies. It is important to maintain a competitive market. Instead, the focus should be on standardising corporate ESG disclosure for a few core metrics that underpin all ratings. This would shift the business of rating agencies from collecting non-standardised and voluntary disclosure and towards interpreting data available to all.

Even with standardised disclosure, there is a need to contextualise data points and to evaluate not only the latest disclosure but also to predict progress if a company seeks to improve its performance. Like financial analysts who all work with the same earnings data, ESG ratings would disagree less about the past and more about the future. The best result is that competition over time will reveal which ESG analysts are getting it right.

Regulators should establish an accountability mechanism requiring ESG agencies to respond to the criticism by rated companies. In conversations with company representatives, we hear frequently how businesses struggle to communicate sometimes serious concerns about how they are rated. ESG raters need to be able to explain to businesses how their rating was determined and react when companies can point to errors in the underlying data.

The future of ESG rating agencies hinges on regulatory action to establish standardised reporting for companies and maintain a competitive market that promotes quality and efficiency. Emphasising transparency, addressing conflicts of interest and refining methodologies will help ESG ratings better serve investors and society at large in a rapidly changing landscape. 

Julian F Kölbel, of the University of St Gallen, Florian Berg, of MIT Sloan School of Management, and Professor Roberto Rigobon, of MIT: Sloan, are co-authors of Aggregate Confusion: the Divergence of ESG Ratings (Review of Finance, 2022)

Letter in response to this article:

Untangling ESG mess will need more than regulation / From Joe Zammit-Lucia, London TW1, UK