Hold a referendum to let the 50,000 undergraduates and post-graduate students of UM and UKM decide whether to join or boycott the THE university rankings

The Universiti Malaya Students Association (PMUM) has come out in support of Universiti Malaya and Universiti Kebangsaan Malaysia opting out of the annual Times Higher Education Top 400 World University Rankings.

PMUM president Fahmi Zainol said the two universities deserve praise for their move, pointing out that the institutions have been plagued with problems for being too focussed on rankings.

I can understand the view that quality education is not dependent only on university rankings and that universities should not be too obsessed with them. In fact, it has been asked: “University rankings: reliable or rubbish?”

The Norwegian government has even commissioned a study of the placement of Norwegian universities in global rankings and concluded that university rankings are “useless” as a basis for information if the goal is to improve higher education as they are so based on subjective weightings of factors and on dubious data. This Norwegian report was described as “A Kiss of Death for university rankings” by a Norwegian academician.

There are undoubtedly pros and cons on university rankings.

Have Fahmi thought through the whole subject of university rankings and are his views his personal ones, or the official view of the PMUM Executive Committee, as well as representing that of the UKM Students Council?

Are they the views of the 50,000 undergraduates and post-graduate students in the two universities?

If Fahmi is opposed to UM and UKM taking part in the annual THE World University Rankings, why does he support the two universities taking part in the annual QS World University Rankings and UM’s decision to take part in the THE World University Ranking in 2018 when UM is more ready to compete with other established universities around the world?

It would be a good for hold a referendum to let the 50,000 under-graduates and post-graduate students of UM and UKM decide whether the two university should join or continue to boycott the THE university rankings, and let the proponents of both the pros and cons of the argument to state their case, not only to the university student-and-lecturer population but to the nation at large.

This referendum proposition is a subject which PMUM and the UKM Students Council should seriously consider to take up with the university authorities.

Let us look at the history of UM and UKM with university rankings.

The Times Higher Education Supplement (THES) commissioned QS Quacquarelli Symonds, an education consultancy, to collect and compile the data which formed the basis for the THES-QS World University Rankings from 2004 to 2009.

Recall that in the first year of this ranking, UM came in ranked 89 in the world and USM was ranked at 111.

The reason for this high ranking was that QS mistakenly thought that Chinese and Indian Malaysian students in these universities were foreign students leading USM to score 78 out of 100 (ranked 4th overall) and UM to score 68 out of 100 (ranked 6th overall) for the International Student Score.

A similar mistake was made for the Chinese and Indian faculty by QS. QS later came out to confirm this mistake.

Once this data was corrected, UM fell from 89th in 2004 to 169th in 2005. Its international student score dropped from 68 in 2004 to 7 in 2005 and its international faculty score dropped from 29 in 2004 to 12 in 2005. USM dropped out of the top 200 completely.

In the 2006 rankings, UM fell further from 169 to 192 although UKM made an appearance in the top 200 rising from 289 in 2005 to 185 in 2006, overtaking UM.

In 2007, both UM and UKM dropped out of the Top 200 to 246 and 309 respectively. USM and UPM remained outside the top 200 at 307 and 364 respective.

In 2008, none of the Malaysian universities cracked the top 200. In 2009, UM managed to crack the top 200 by coming in at 180.

After the THE-QS World University Rankings 2009, QS and Times Higher Education went their separate ways with QS continuing the old methodology while THE formed a partnership with Thomson Reuters, the data and media giant.

In the QS World University Rankings 2010, UM fell out of the top 200 to 207. Since 2011, UM has been gravitating between 151 (2014-2015) and 167 (2011 and 2013-2014). For now, UM’s position in the top QS World University Rankiings 200 seems to be secure even though the QS rankings may be volatile for universities above 100.

Table 1: THES-QS and QS World University Rankings (2004 to 2014/2015)

Source: QS-THES World University Rankings (2004-2009), QS World University Rankings (2010 to 2014-2015)

The Difference between the QS and THES Rankings

THES broke away from QS in 2010 because it decided to ask Thomson Reuters to provide data for the basis of the THES rankings. One of the reasons was because of the criticism against the heavy weightage of QS academic reputation survey (40% of total rankings). Thomson Reuters went on to compile new data from a Global Institutional Profiles Project (GIPP).

Instead of relying on just 6 measures which is used by QS, THES uses 13 indicators to achieve a greater balance and consistency in their ranking system.

Both QS and the new THES ranking systems have had their share of criticisms.

What is Malaysia’s approach to QS and THES?

After the break-up between the QS and THES rankings, most Malaysian universities seems to have participated in the QS rankings but not the THES rankings.

The number of Malaysian public universities being featured in the QS survey has increased from 5 in 2009 to 7 in 2010.

According to the list of participating institutions provided by Thomson Reuters, only UKM, UPM and UUM participated in this survey of global institutions. However, in the latest 2014-2015 report, THES says that both UKM and UM failed to participate in this survey.

Are the reasons for non-participating in the THES survey valid?

The Malaysian Insider have the following reasons for UM’s decision not to participate in the THES survey:

“UM vice-chancellor Prof Datuk Dr Mohd Amin Jalaludin says the institution did not take part in the THE rankings. ‘There are two reasons why UM chose not to participate in the THE World University Rankings. The first is because UM is still building up its publications and citations in the Web of Science database. UM’s policy on this began only in 2008. It is now only beginning to bear fruit but is not yet sufficiently mature to make an impact. ‘The second reason is related to the amount of funding that UM receives. It is not in a strong enough financial position to compete with other richer, older and better-ranked universities,’ he explains.”

When speaking on UM’s rank of 151 in QS World University Rankings 2014-2015, an improvement of 16 places, the UM VC had this to say:

“The results of the ranking is a useful measure for the university to work on. The university will strive to improve its performance further by critically analysing its performance,” he said.

The citation reason given by the UM VC looks much weaker when we consider that is also part of the QS World University Rankings.

In fact, when we examine UM’s citation score in the QS World Rankings, its citation ranking at more than 400+ is unchanged from its 2013-2014 ranking (although its citation score did improve from 7.5 in 2013-2014 to 11.1 in 2014-2015). For comparison, top ranked MIT scored 100 for the citation score. If the citation score is in the QS World Rankings and UM decided to participate in this ranking, why not also participate in the THES ranking?

The second reason given by the VC is that UM receives insufficient funding compared to other institutions. But this is also not a very strong reason since many other universities from developing countries also participated in the THES rankings including the Middle East Technical University in Turkey (rank = 85), the University of the Andes in Colombia (Rank = 251-275) and the Indian Institute of Science in India (Rank = 276-300).

Furthermore, among the public universities in Malaysia, UM and UKM receives the 2nd and 4th largest allocation (operating and development budgets), according to the 2014 budget estimates. UM and UKM’s operating budgets is over half a billion RM!

If both of these institutions feel that they are underfunded, what about the other public universities including UUM which is listed as one of the participating institutions in Malaysia with only the budget of RM260m?

Clearly, the point of participating is to see how much our public universities can improve over the years with a given amount of budgetary allocation. Without participating in the THES rankings, we won’t know if the performance of our pubic unis have improved over time.

Table 2: Operating and Development Budgets for IPTAs according to the 2014 Budget Estimates

What is even more interesting is UM’s decision to participate in the QS Stars Rating System for which it must pay a few to be audited every year.

Under the QS Stars Rating system, UM received 5 Stars for which it had to pay an initial audit and license fee of US$16,750 and a yearly license fee of US$6850.

If UM is willing to pay for the privilege of being audited, why not participate in the THES rankings which does not require a participation fee?

Even Universiti Malaysia Perlis (UniMAP) participated in the QS Stars rating system for which it was praised by Prime Minister Najib: "Congrats to @PROUniMAP on the 3-star rating by QS Intelligence. Hope so to see more public unis follow suit," said Najib.

The UKM has also given The Malaysian Insider its reason for not participating in the THE World University Ranking 2014, viz:

“In an immediate response to Baty's comments, UKM's Strategic Centre deputy executive director (performance assessment) Associate Professor Dr Masturah Markom told The Malaysian Insider that the university preferred to focus on rankings that were fair to its direction.

"’This ranking (THE) has different indicators and it's more suitable for universities that have been established far longer. Indicators, such as 'industry income', are unfair to us as Malaysia's (industry income) is not as much as in the United States, for example. We cannot compete with them so it is best if we spend on rankings that are better suited to our direction and focus," she said, referring to the QS (Quacquarelli Symonds) rankings.’”

Again, this is not a good reason for not participating in the THES rankings since (i) industry income comprises less than 5% of the total score for the THES rankings (ii) Malaysian universities are being pushed to innovate more and to have more public private collaborations which would bring in research income for the researcher and the their university. This is an important tracking mechanism for Malaysian public universities moving forward.

Summary

One cannot help but think that Malaysian universities such as UM prefer to participate in the QS Survey over the THES survey because:

(i) QS is a client of UM and moving forward, other public universities as well in the provision of consultancy services, marketing and subscription of the QS Stars rating system and hence would be more favourable towards these institutions in terms of ranking;

(ii) Under the QS Ranking, a university may find it easier to ‘game the system’ by increasing scores on certain indicators e.g. faculty / student ratio:

(iii) Citation scores under THES make up 30% of the overall score compared to 20% for QS. Since this is an area which is the hardest to ‘game’ the system and where Malaysian academics have performed badly, it is not surprising that some public universities want to avoid ranking systems which give a higher % weighting to citation scores.

Clearly, it would be preferable for public universities in Malaysia to participate in as many rankings as possible so that it has numerous data points to judge its performance.

But at the same time, the universities must have its own internal targets to improve on its own performance which are not subject to the vagaries and ‘gaming’ of the various ranking systems.