...

gero_federkeil - Universidade Nova de Lisboa

by user

on
Category:

education

38

views

Report

Comments

Transcript

gero_federkeil - Universidade Nova de Lisboa
Mapping Diversity –
The U-Multirank Approach to Rankings
Gero Federkeil
Workshop Universidade Nova de Lisboa,
29th June 2012
Presentation
Portuguese Universities in Global Rankings
A (Short) Critique of Existing Global Rankings
An Alternative Approach: U-Multirank
Universidade Nova de Lisboa, 2012/06/29
2
Portuguese Universities in Global
Rankings
Shanghai Ranking
QS Ranking
THE Ranking
401- 500 U Lisboa
394
U Coimbra
301-350
U Aveiro
301-400
401-450
U Nova de Lisboa
351-400
U Porto
401-450
U Porto
501-550
U Catolica
Portuguesa
U Porto
Portuguese universities are not well represent edin international
rankings
Hence those rankings are not a good instrument to look at the
Portuguese HE system
Universidade Nova de Lisboa, 2012/06/29
3
Presentation
Portuguese Universities in Global Rankings
A (Short) Critique of Existing Global Rankings
An Alternative Approach: U-Multirank
Universidade Nova de Lisboa, 2012/06/29
4
The coverage of Global Rankings
Existing global rankings cover only a small minority of
all universities
EUA: Global University Rankings and their Impact. Bruxelles, 2011.
Universidade Nova de Lisboa, 2012/06/29
5
The Indicators
Existing global rankings are rankings of internationally
oriented research universities only
Shanghai Jiaotong Ranking
Indicator
Weight
QS
Indicator
Weight
SCI publications
20 %
Reputation among scholars
40 %
Publications Science & Nature
20 %
Reputation among employers
10 %
Highly cited authors
20 %
Citations
20 %
Nobel Prizes & Field Medals
20 %
Student-staff-ratio
20 %
Alumni with NobelPrizes
10 %
International students
10 %
Size
10 %
International staff
10 %
Universidade Nova de Lisboa, 2012/06/29
6
The methodology: Ranking orthodoxy
Ranking of whole
institutions
 Most users are interested in information about
“their” field”
 Institutional rankings give misleading averages
across fields/units
Composite overall
score
• Composite indicators blur profiles and strengths
& weaknesses
 There are neither theoretical nor empirical
arguments for assigning specific pre-defined
weights to single indicators
League table
approach
• Small differences in the scores of indicators lead
to big differences in league table positions
 Give false impression of exactness (“Number
123 is better than number 127”)
Universidade Nova de Lisboa, 2012/06/29
7
Conclusions
1. Existing global rankings cover only a small minority of
universities
2. Due to their indicators they only measure research and
cover internationally oriented research universities only
3. They devaluate other profiles and missions (teaching, LLL,
regional engagement)
4. They are a threat to diversity in higher education
Universidade Nova de Lisboa, 2012/06/29
8
Presentation
Portuguese Universities in Global Rankings
A Critique of Existing Global Rankings
An Alternative Approach: U-Multirank
Universidade Nova de Lisboa, 2012/06/29
9
The Project
• Commissioned by the European Commission
• 2-year feasibility project, 2009 – June 2011
• Report now available:
http://ec.europa.eu/education/higher-education/doc/multirank_en.pdf
• Ján Figel, the former European Commissioner for
Education, Training, Culture and Youth:
“- to allow stakeholders to make informed choices;
- to help institutions to position themselves and
improve their performance”
• Two phases:
o Design of new instrument
o Testing the feasibility of new instrument
Universidade Nova de Lisboa, 2012/06/29
10
Specification of the Project
• Five dimensions:
o Teaching & learning
o Research
o Knowledge transfer
o International orientation
o Regional engagement
• Development of a list of indicators to be tested in pilot project
• Development of data collection tools and processes (questionnaires, definitions, FAQs, communication + feedback processes)
• Done by CHERPA consortium: CHE, CHEPS, CWTS, INCENTIM; OST
Universidade Nova de Lisboa, 2012/06/29
11
Specification of the project
• Two levels:
• Institution (FIR)
• Fields (FBR)
• Global sample of higher education and research institutions:
• 159 (target: 150), 2/3 Europe,
• 109 completed institutional questionnaires
• Two pilot fields:
• Business studies
• Engineering (electrical and mechanical)
Universidade Nova de Lisboa, 2012/06/29
12
The Basic Methodology
+  There is no single objective ranking
 Each ranking reflects the ideas and preferences
of those doing them
 The decision about the relevance of indicators
should be left to the user
Multi-dimensional
ranking
+
• Different levels of analysis are relevant for
different users
 Field-based and institutional rankings
Multi-level
ranking
+
Grouping
approach
Universidade Nova de Lisboa, 2012/06/29
• Rank groups instead of league tables provide
more meaningful and valid information
13
Mapping Diversity
• A basic aim of U-Multirank is to make visible the diversity of
Higher Education Institutions
• and to show excellence beyond research excellence
HOW ?
Universidade Nova de Lisboa, 2012/06/29
14
Starting point: Diversity of higher education institutions in
Europe & the world
Identifying comparable
institutions that can be
compared in one ranking
Description of horizontal
diversity

Types/profiles
+
Assessment of vertical
diversity

Performance
Complementary instruments of transparency
15
Universidade Nova de Lisboa, 2012/06/29
Mapping and Ranking
1. Step: Mapping:
Selection of a comparable set of universities based on
institutional profiles
Teaching and
learning
Research
involvement
Knowledge
exchange
Example:
• Comprehensive, teaching oriented institution
• Mainly undergraduate education
• Low research orientation
• Low international orientation
• Regionally embedded (e.g. recruiting)
International
orientation
Regional
engagement
Student profile
2. Step: Ranking
of subset of comparable institutions
with similar profiles
Universidade Nova de Lisboa, 2012/06/29
16
Multi-Dimensional Ranking for Subset of
Comparable Institutions
Universidade Nova de Lisboa, 2012/06/29
17
User-driven, Personalised Ranking
Universidade Nova de Lisboa, 2012/06/29
18
A Personalised Ranking …
… Helping to make an
informed choice
based on invidual
preferences
Universidade Nova de Lisboa, 2012/06/29
19
Results of the feasibility study
• Generally the concept, indicators and instruments of UMultirank are feasible both on the institutional and the field
level
• There are some problems concerning indicators, mainly on
• Issues of employability
• Knowledge transfer, and,
• Regional engagment
• There has to be some refinement of concepts and indicators
Universidade Nova de Lisboa, 2012/06/29
20
Outlook : U-Multirank II
•
•
•
•
New call for tender launched by the European Commission
CHE applied with partners (CHEPS, CWTS, …)
Decision at the end of July (?)
Start of the project on 1 October (?)
• 2 * 2 years
• Implementation of the concept: First publication of ranking at
the end of 2013
• Min. 500 institutions, institutional and 4 fields
• Annual extension of number of institutions and fields
• Development of a business model for a sustainable system
Universidade Nova de Lisboa, 2012/06/29
21
But after all, there still might be some
limits to ranking…
„You‘re kidding! You count publications?“
22
Muito obrigado!
For more information:
www.u-multirank.eu
[email protected]
Mapping Diversity –
The U-Multirank Approach to Rankings
Gero Federkeil, CHE Centre for Higher Education
Workshop Universidade Nova de Lisboa,
29th June 2012
Fly UP