Research Impact: Sustainable Development Goals

The Context

United Nations Sustainable Development Goals (SDGs), established in 2015, are the culmination of decades of work on pushing forward a global commitment to tackling the biggest issues facing our world. All 193 member states adopted these ambitious plans.

The UN states that they are “a blueprint to achieve a better and more sustainable future for all. They address the global challenges we face, including poverty, inequality, climate change, environmental degradation, peace and justice”. Researchers have been increasingly highlighting the role that universities can take in forwarding this agenda. Jardali et al. (2018) write: “Universities are uniquely placed to lead the cross-sectoral implementation of the SDGs and advance the 2030 agenda”.

A 2019 report by QS found that 94 percent of prospective international students thought that universities should be doing more to be environmentally sustainable. In 2021, Forbes argued Gen Z are emerging as the ”sustainability generation”, and it is this generation who predominantly occupy the ‘prospective international student’ audience that QS speaks to now.

There are many ways to assess a university’s commitment to advancing Sustainable Development Goals, from their on-campus policies to their strategic planning. Research is pivotal in advancing any of the SDGs – everything that has happened and will happen to alleviate climate change, reduce inequality, clean our waters or stop hunger will come as a result of research. From economics to biochemistry, public policy to queer theory – researchers from across the academic spectrum must bring their fields together to meet the UN’s call.

The Data

The data for our SDG research metrics is provided by our research partners, Elsevier SciVal. Please see here for the approach they took to creating their SDG mappings. Each SDG is mapped to certain publication keywords which are searchable within SciVal. We requested the following data points for each institution in the QS database, for each SDG:

• The total number of papers published over the last 5 years (same timeframe as the corresponding rankings cycle). Our usual exclusions by paper type and affiliation cap are applied. 
• From the above, the total number of papers cited by policy documents (as defined by PlumX, Elsevier's subsidiary, and Overton.io).
• The total number of citations received by this subset of papers from such policy documents (Policy Citations).
• The total number of citations received by all papers over the last 6 years, with author-level self-citations excluded.
• The number of papers published in the Top 10 percent of academic sources as defined by Elsevier's CiteScore Percentile.
• The average per paper using Elsevier's Field-Weighted Citation Impact (FWCI), with author-level self-citations excluded. 
• The median of per paper citations, with author-level self-citations excluded.
• The median of per paper FWCI, with author-level self-citations excluded.

With the above data points ready per institution and per SDG, the following metrics are then used:

The Metrics

1. Research Impact per SDG

The following sub metrics are aggregated as a weighted sum into 16 research impact indices. Each of these is then z-scored and scaled to 100. 

2. Policy Citations per SDG

This metric looks at the total number of citations received by all papers published by an institution in the corresponding SDG field from policy documents. This is a measure of impact of papers published by an institution in the given SDG field on non-profits, governmental organizations, think-tanks, and other policy making institutions and legislators. Please find below further details on any adjustments¹.

After all 16 Policy Citations indices are created, each of them is then z-scored and scaled up to 100. 

¹ All of the above mentioned metrics are adjusted on a sliding scale up to four times the global median paper output per SDG (either overall in case of Research Impact metrics, or only for documents cited by policy documents in case of Policy Citations metric). This method is an adaptation of a similar approach used in the QS World University Rankings by Subject. The intent of this is to establish a dependency between the metrics we use in the methodology, and a university's expertise (in the form of its research footprint) in the given SDG. As such, the more expertise a university has, the more credibility its indicators have and vice versa. Universities with a low research footprint but high visibility and impact numbers will be penalised in their scoring. Moreover, institutions in the lowest quartile by the total number of papers in the corresponding SDG receive a zero score in all metrics specifically for that SDG.

It is quite common that a university's impact metrics may be influenced by several anomalous papers, which do not reflect their regular research output. Although we acknowledge that any improvements and new initiatives may result in positive results that differ significantly from the historical performance, we choose to reward those universities which demonstrate stability and robustness in their SDG commitment. One sign of this is how close the average per paper citation and average per paper FWCI are to the median values for a given university: the closer to those medians, the more credibility the first two indicators have. This is applied to Total Citations and Average Per-Paper FWCI metrics.

Application in QS evaluation projects 

Research Impact per SDG scores are used in the inclusion criteria for QS World University Rankings: Sustainability and its five metrics:

• EQ1. Research Impact on SDGs for Equality
• IE1. Research Impact on SDGs for Education
• EO2. Research Impact on SDG's for Employment and Opportunities
• HW1. Research Impact on SDG's for Health and Wellbeing
• ER1. Research Impact on SDGs for Sustainable Research

The latter metric is also used in QS Stars University Ratings.

Policy Citations per SDG scores are used in the following metrics of QS World University Rankings: Sustainability:

• KE4. Policy Citations (Social)
• ER4. Policy Citations (Environmental)
• GG10. Policy Citations (Governance)