The Academic Reputation Index is the centerpiece of the QS World University Rankings® carrying a weighting of 40%. It is an approach to international university evaluation that QS pioneered in 2004 and is the component that attracts the greatest interest and scrutiny. In concert with the Employer Reputation Index it is the aspect which sets this ranking most clearly apart from any other. It seeks to answer the powerful question of which universities are performing world-class research. The answer to this question not only illuminates the quality of the research, but the strength of the university in communicating that research, and the strength of the impact the research makes across the world.
Source of Respondents
The results are based on the responses to a survey distributed worldwide academics from a number of different sources:
- Previous Respondents
- Submitted contact lists from institutions (see Survey Nominations Procedure)
- Sign-ups on our sign-up facility (see Survey Nominations Procedure)
- IBIS database - see IBIS
The survey is sent to many thousands of global academics each year. It has largely followed the same principles since inception, with some variation depending on academic themes of interest over time. At the beginning of the survey, academics state their field specialism and their regional familiarity. The answers to this then guide the range of answers they can give in the remainder of the survey. We ask the following questions of each respondent:
- Their name
- Their institution
- Their job
- The number of years they have been in academia
- Which country/territory they are most familiar with, from an academic perspective
- Which region(s) they are most familiar with, from an academic perspective
- The faculty area in which they are most active and knowledgeable
- The specific field (up to a maximum of two) that they specialize in
Top Domestic Institutions
- Academics are asked to nominate up to 10 institutions from their country/territory of knowledge that they think are producing the top research in their faculty area. Their own institution is excluded.
Top International Institutions
- Academics are asked to nominate up to thirty international institutions from their country/territory of knowledge that they think are producing the top research in their faculty area. Their own institution is excluded. The list consists solely of institutions from the region(s) with which they express familiarity with.
To answer certain higher education insight needs, or to receive feedback on our products, other additional questions may be asked. These questions necessarily vary.
Data cleaning and validity checks
Once the survey has been collated, a variety of checks and balances are performed to ensure the responses are valid, useable and complete.
We first remove the following:
- Submitted surveys with no nominations
- Survey was completed too quickly as to be trustworthy
- Responses submitted by any QS office
- Responses submitted by students (ineligible for the Academic Survey)
- Responses submitted by students or academics (ineligible in the the Employer Survey)
We then seek to ensure that the nominations provided are being done in good faith. This anomaly testing includes, but is not limited to, removing responses where:
- The same email address was used multiple times
- Multiple email addresses are associated to the same full name & nationality
- The same IP addresses have been used in both surveys
Where we see an above average number of repetitive nominations from a single IP address, we will seek to identify the respondent and investigate.
Finally, we sort and match string entries and map them to the correct variables in our analyses.
Step by Step Analysis
Once the responses have all been processed, we apply the following procedures for all of the nominations for our five broad faculty areas (insert link).
- Devise weightings based on the regional and faculty familiarity of respondents. Respondents are able to relate to more than one region.
- Derive a weighted count of international nominations for each institution (excluding self-nominations). Here, we use a 5 year aggregation of nominations, where the earlier two years count for 25% and 50%, and the most recent three years at full 100% weight.
- Derive a weighted count of domestic nominations for each institution (excluding self-nominations). This is adjusted against the number of institutions from that country with a certain level of international nominations and the total response from that country. Larger countries with more recognized institutions naturally face more competition in terms of gaining nominations, and this is designed to reflect and reward this.
- Apply a straight scaling to each of these to achieve a score out of 100.
- Combine the two scores with a weighting 85% for international nominations, and 15% for domestic. Note that we use 50:50 for Employer Reputation.
- Square root the result – we do this to draw in the outliers but to a lesser degree than other methods might achieve – our intention is that excellence in one of our five areas should have an influence, but not too much of influence.
- Scale the rooted score to present a score out of 100 for the given faculty area.
The scores across the five faculty areas are then combined with equal weighting to produce the final score per institution for Academic Reputation.