What is U-Multirank?
U-Multirank stands for “multi-dimensional ranking of higher education institutions”. U-Multirank is based on a proposal in the Commission Communication on modernisation of Europe’s higher education systems (COM (2011) 567 final) [1] (accompanied by Staff Working Document (SEC (2011) 1063 final), p. 5-6) and is implemented by a consortium of research organisations – CHERPA Network (Consortium for Higher Education and Research Performance Assessment) under a two-year project funded by the European Commission.
A preparatory study “Design and Testing the Feasibility of a Multidimensional Global University Ranking” concluded in June 2011 demonstrated the feasibility of this project.
U-multirank was officially launched at EU presidency conference in Ireland on 30 January 2013 (RAPID press release IP/13/66) and is now in its implementation phase (see also speeches of Ruairi Quinn, Irish minister for Education and Skills and Androulla Vassiliou, Commissioner for Education and Culture).
The first ranking is expected in early 2014 covering at least 500 higher education institutions from Europe and beyond in four specific subject areas: business studies, mechanical engineering, electrical engineering and physics (RAPID press release IP/12/1373). The list of specific disciplines will be gradually expanded in future years.
Overviews
The Commission SCADPlus factsheet on Modernising Universities provides useful background information about the EU agenda and initiatives in the area of higher education and measures to be taken in order to modernise European universities.
The path to U-multirank:
International university rankings form choices and priorities of governments, businesses and students. Rectors and university councils see the achievement of high ranking as a strategic objective.
Some of the shortcomings of existing rankings such as Times Higher Education World University Rankings and Shanghai Jiaotong University’s Academic Ranking of World Universities, or Ranking Web of Universities:
- they focus on research aspects rather than teaching, prompting universities to put more efforts in research, while wasting potential in other areas;
- they rank entire institutions rather than programmes and departments;
- due to criteria that favour American and British universities, only a small proportion of European universities is included in these rankings.
In this context Germany launched the Excellence Initiative [2] in 2005 and the French Presidency of the EU [3] advocated the need for a European system of university ranking (identified in a 2008 report by the French Senate [in FR]). Following these events, the idea of setting a European system has been developed since 2008, when the then European Commission Director-General for Education and Culture Odile Quintin announced a call for proposals for projects for a European University classification (see University World press release and French Presidency Conclusions on Typology and Ranking of the Higher Education Institutions: the European Approach setting the main principles that a European ranking and a parallel mapping should follow).
This has led the European Commission to fund two major feasibility programmes – U-Map and U-Multirank.
The U-map.eu webpage and 2011 Update report provide further information. Currently the U-map.org website is still “protected”, which means that results are visible only to the institutions that have submitted data. It is expected to enter the public domain and be fully functional by the end of 2013 (see results of the U-map project in the Nordic countries presented in October 2012).
U-Multirank is not to be confused with U-Map. U-map is a classification tool to describe different types of universities and what they do, whereas U-Multirank aims at ranking, i.e. showing how well universities do certain things by providing broader information in five areas – reputation for research; quality of teaching and learning (e.g. levels and orientations of degrees, subject range), internationalisation, knowledge transfer (e.g. partnerships with business and start-ups) and contribution to regional growth.
The U-multirank project webpage provides detailed information about the concept of multi-dimentional ranking and current development of the project, as well as background information about the consortium implementing it. 5 main stakeholder organisations are identified in the implementation of the project: Business Europe; Conference of European Schools for Advanced Engineering Education and Research (CESAER); European Students’ Union; International Research Universities Network (IRUN) and Universities of Applied Sciences Network.
Analysis
Multidimensional ranking: a new transparency tool for higher education and research, Frans A. van Vught and Don F. Westerheijden, Higher Education Management and Policy Volume 22/3, OECD 2010 – This paper gives an overview of the concept of transparency tools and those that are currently available. It then criticizes current transparency tools’ methodologies, looking in detail at data sources, risks involved in constructing league tables and challenges in using composite indicators. Lastly, it argues in favour of a new principle for transparency tools: multidimensional ranking.
Understanding Rankings and the Alternatives: Implications for Higher Education, Ellen Hazelkorn January 2012 in Bergan S. et. al. (Eds.) “Handbook Internationalisation of European Higher Education” Stuttgart: Raabe Verlag (Forthcoming) – This book chapter gives an overview of global university rankings. It defines U-multirank, among other multidimensional rankings, as an alternative model, followed by an assessment of the benefits that such alternative models can bring.
How Rankings are Reshaping Higher Education, Ellen Hazelkorn in Climent, V., Michavila, F. and Ripolles, M. (eds): Los Rankings Univeritarios: Mitos y Realidades, Ed. Tecnos, 2013.in Climent, V., Michavila, F. and Ripolles, M. (eds): Los Rankings Univeritarios: Mitos y Realidades, Ed. Tecnos, 2013.
University Ranking Systems – From League table to Homogeneous Groups of Universities, Jarocka, M., World Academy of Science, Engineering and Technology Issue 66, 2012 – This paper starts with a review of the critical analyses of university ranking system followed by an overview of the initiatives supported by the European Commission (U-Map and U-Multirank) and the CHE Ranking. It also contains a list of the most popular existing university ranking systems. The author argues that the approach used in multi-dimensional ranking is an appropriate tool because it gives the user the possibility to find and compare similar institutions for specific purposes.
Rankings and accountability in higher education: uses and misuses/ UNESCO, 2013. The growing impact of university rankings on public policy – and on students’ choices – has stirred controversy worldwide. This volume brings together the architects of university rankings and their critics to debate the uses and misuses of existing rankings.
U-Multirank from a researcher’s perspective/ Endika Bengoetxea ; Gualberto Buela-Casal in International Journal of Clinical and Health Psychology 13 (2013). This paper describes the motivation for designing U-Multirank, the principles of the methodology proposed, as well as the steps foreseen to have it ready for end users by 2014.
Reflections on a Decade of Global Rankings: what we’ve learned and outstanding issues/ Ellen Hazelkorn. European Journal of Education (2013). This paper reflects on three inter-related issues; i) considers the way rankings have heightened policy and investment interest in higher education, ii) discusses whether the modifications to rankings have resolved some of the questions about what they measure, and iii) looks at how rankings have influenced stakeholder behaviour.
U-Map and U-Multirank: some first results and reflections on two new profiling and ranking tools for higher education institutions [available via Library website], Ben Jongbloed and Frans Kaiser (University of Twente), paper submitted for the ENID 2011 conference September 2011 – This paper explains the differences between the two projects (U-map and U-multirank) and their complementarities. It then shows examples of results from the U-Map and U-Multirank projects and lists concerns expressed by stakeholders with regards to the indicators, statistics comparability, choice of dimensions, as well as possible replies.
Community Engagement and University Rankings, blogpost by Hans Schuetze, Emeritus Professor University of British Columbia, September 2012 on PASCAL International Observatory – This blogpost provides a handy overview of U-map and U-multirank and is a useful source of informal discussion and opinions.
“Ranking in higher education: its place and impact”, Jan Sadlak, World of Learning, 2010 – This essay provides an overview of the various types of university rankings, from “league tables” to multi-dimensional or subject-focused rankings. It assesses the sources of information, methodologies and influence (both positive and negative) of rankings.
Simpson’s Paradox and confounding factors in university rankings: a demonstration using QS 2011–12 data, Kay Cheng Soh, European Journal of Higher Education December 2012, – This academic article presents different criticisms of university ranking systems, such as the choice of indicators, the lack of methodological transparency and over-simplicity of obtaining overall scores. It argues that rank users (students, parents and other stakeholders) need publicly available consumer information on how they should read the rankings.
“Identifying the Best: The CHE Excellence Ranking”, Sonja Berghoff et. al. CHE Working paper № 137, 2010 – This working paper presents the results of the CHE Excellence Ranking. The CHE rankings do not compare universities as a whole. They are based on the belief that each university has individual profiles with strengths and weaknesses in different subjects. CHE is a lead partner in the U-multirank project. The Centre for higher education development (CHE) is a German think tank for higher education managing various ranking initiatives with different focus (CHE University ranking – assessing higher education institutions for the first-year students; CHE Research ranking – assessing higher education institutions on the basis of their research performers; CHE Excellence Ranking – assessing universities according to programme offerings and academic qualities; CHE/DAMP Employability Ranking – assessing bachelor programmes in Germany on the basis of how well they promote qualifications and competencies that contribute to the professional capability of their graduates).
Stakeholder views
European Parliament
Besides the discussion in the EP CULT Committee on the Modernising Europe’s higher education systems (Oeil procedure file 2011/2294(INI)), MEPs follow the development of the U-multirank project. The following parliamentary question regarding U-multirank has been asked:
U-Multirank (multi-dimensional global university ranking) by Chrysoula Paliadeli (S&D), regarding measures to ensure that EU higher education institutions rise to top positions in the ranking system and that private universities are not in a more favourable position that public ones. The Commission answered on 31 May 2012 that responsibility for achieving excellence rests education institutions themselves and the ranking system will give possibility to compare ‘like with like’. Therefore, public and private institutions must pursue excellence in terms of their own priorities.
UNESCO
In 2004 UNESCO European Centre for Higher Education (UNESCO-CEPES) and the Institute for Higher Education Policy established an International Ranking Expert Group (IREG). In 2006 they produced the Berlin Principles on Ranking of Higher Education Institutions. In October 2009, on the basis of IREG was created the IREG Observatory on Academic Ranking and Excellence.
OECD
The OECD is carrying out another new initiative that could be interpreted as a hybrid of quality assessment and rankings – Assessment of Higher Education Learning Outcomes (AHELO). Its aim is to assess what students in higher education know and can do upon graduation. 17 countries from and outside Europe participated in the feasibility study (concluded in 2012) which confirmed that an international assessment of higher education learning outcomes is scientifically and practically possible. The feasibility study report will be officially presented in March 2013 at a conference in Paris.
EUA
Global university ranking and their impact, Rauhvargers, A. 2011, and Global university ranking and their impact II, Rauhvargers, A. 2013, commissioned by European University Association – By looking at 13 global ranking systems, these studies provide a good overview and critical analysis on universities’ rankings, their usefulness and uses, as well as their shortcomings and resulting distortions. They refers to a number of ongoing projects measuring university performance. With the increasing popularity of rankings increases the importance of users’ understanding of what is in fact being measured, how the scores are calculated and what they mean. The second report sets out various new developments since 2011 that the author believes will be important for European universities. This includes information on methodological changes in more established rankings, on new rankings that have emerged, and on a range of related services developed as well as an analysis of the impact of rankings on both public policies and universities.
ESU
European Students’ Union position on U-multirank is expressed in a press release from 31 January 2013. – ESU welcomes the Commission’s initiative to develop a multi-dimensional ranking system as an alternative to traditional ranking systems, where the rise and fall of the reputation leads to adjusting funds and tuition fees, and considers students’ involvement in deciding on indicators necessary for the relevance of the project.
UK House of Lords position is expressed in the Report on “The Modernisation of Higher Education in Europe” published on 22 March 2012, Chapter 3 “The EU’s contribution of the modernisation of higher education”, points 50-57. (pdf version is available here). The UK Parliament is sceptical about the added value of the project and advices the European Commission to focus on other priorities.
Interview with Frans van Vught, project leader of U-multirank and CHEPS member, September 2011 – In this interview Frans van Vught discusses the development of the multi-dimensional ranking system and defies some of the arguments and concerns about the validity and usefulness of U-multirank. He explains that the indicators used in U-multirank were chosen after a wide consultation with stakeholders so that they can overcome deficiencies in existing ranking systems.
An Euractiv article on “New EU university rankings to challenge global league” (June 2011, updated November 2012) in its “Positions” section contains a handy overview of main points expressed by stakeholders, including that of the European Students’ Union; DG Education and Culture at the European Commission; University of Liverpool and the French government.
League of European Research Universities (LERU) – LERU, representing 21 research-intensive universities including Cambridge, Oxford and Edinburgh, is skeptical about the added value and reliability of U-Multirank (see Times Higher Education article, 7 February 2013) [4]. See also University rankings: diversity, excellence and the European initiative, LERU’s advice paper by Geoffrey Boulton / June 2010 – the paper considers the impact of rankings on students, universities and businesses, and presents LERU’s views on the strengths and shortcomings of U-map and U-multirank projects, namely data incomparability from country to country, potential game playing when reputations are at stake, additional burden to collect information, etc. LERU generally supports launching U-Multirank as a pilot project and exploring its potential to overcome the problems of other systems underlining, however, the difficulties and dangers which might outweigh its value. Research initiatives and research assessment / LERU May 2012 is a recent LERU’s paper which questions the necessity to assess university research.
Universities Denmark, the organization of the eight Danish universities, published Position paper “The Future of European Research, In-novation and Higher Education” in 2010, where their position on Multidimensional transparency tools is expressed (p. 13).
Press
Experts warn of university rankings bias as EU prepares new table / Euractiv, July 2013. To boost the global standing of EU universities, the European Commission has pegged in its higher education strategy a new form of university listing, which analysts say may overturn the US “elite” bias of current commercial rankings.
Next steps
U-Multirank will receive a total of €2 million in EU funding from the Lifelong Learning Programme in 2013-14, with the possibility of a further two years of seed-funding in 2015-2016. The goal is for an independent organisation to run the ranking thereafter.
On 30 April 2013 the 5th Annual International Symposium on University Rankings and Quality Assurance 2013: “Towards Comprehensive and Robust Global Ranking Systems”, organised by Public Policy Exchange, UK will take place in Brussels with the main aim to assess the latest developments and future outlook of the U-Multirank initiative (see the flyer of the event here).
On 22 August 2013 The new website of U-Map went live. It shows at a single view the profiles of higher education institutions from across the world.
[1] European Parliament adopted its Resolution on the Communication on 20 April 2012
[2] The Excellence Initiative aims at strengthening research at Germany’s universities and raising the visibility of German science and research vis-à-vis international competitors
[3] Second half of 2008
[4] Jordi Curell, director of Higher Education and International Affairs at DG EAC justifies U-multirank, suggesting that Times Higher Education worries about competition to its world university ranking (see University World News article, 16 February 2013).
