REF 2014: Measuring Research Quality in UK Higher Education

Posted by: Kate Forrester

REF 2014

This week the results of the latest UK university audit were released. The research excellence framework, also know as the REF, assesses the quality and impact of academic research. 154 institutions took part in REF 2014, with research submissions from 52,000 faculty across the UK. The results determine research funding and league table rankings, matters of vital importance for both universities and academics. Put into perspective, it’s estimated that institutions have spent a collective £47m on fine-tuning their submissions for the 2014 survey.

How exactly does the REF audit work?

Every 6 years, institutions submit examples of their best research for assessment. Individual academics entered to the 2014 REF were required to present a minimum of 4 examples of work published between 2008-2013.

A combination of practising academics and industry experts (research users) review the submissions.

The results are filtered by subject area. Each subject area can be awarded up to four stars for research excellence.

The idea behind the framework is to safeguard public money spent on research funding, making sure it’s distributed effectively.

Controversies

For REF 2014 the process used to assess the quality of research changed, sparking anxiety among many academics.

The main difference is that 20% of the overall score is now based on the “impact” of the research outside of academia. The introduction of the external “impact” weighting was designed as a way to recognize institutions that actively engage with society and business in a broader way. But many academics have argued that this simply adds extra complications and more paperwork. Documents proving “impact”, such as case studies and research strategies, are now required as part of the REF submission process.

In addition to the “impact”, assessment is also based on the quality of research (65%) and the vibrancy of the department’s research environment (15%).  The scoring breakdown has lead many academics to question whether the quality of university research can be accurately measured through this process?

It’s also been suggested that the judging sub-panels (of which there are only 36) don’t have sufficient time or expertise to make evaluations on what constitutes research quality.

The Results

Despite widespread concerns over the inclusion of “impact” scoring, this actually appears to have had very little impact on the 2014 results. Instead, REF 2014 has given a huge boost to the number of institutions now classed as world leading in the UK.

Imperial College London and the London School of Economics both climbed up the table. The University of Oxford retained its top spot, but the University of Cambridge fell three places.

Among post-1992 institutions, Cardiff Metropolitan University and the University of Brighton stole the limelight, demonstrating strong results in research impact.

In the aftermath, debate surrounding the accuracy and necessity of the research excellence framework continues, particularly amid claims that the results have been skewed by substantial grade-inflation. Overall, however, the response to REF 2014 suggests that it is fascinating and flawed, but nonetheless, essential.