Hefce's impact woes continue

Universities, learned societies and educational establishments have raised further concerns about the UK’s proposed Research Excellence Framework (REF), with particular controversy surrounding the impact assessment element of the evaluation process. 

Currently the REF, a new system for assessing research quality due to be implemented from 2013, proposes that 25 per cent of the assessment is based on economic and social impact of the research (see Chemistry World, November 2009, p6).

In December 2009, the Higher Education Funding Council for England (Hefce)’s consultation on the REF closed after receiving hundreds of responses. Common concerns included the high weighting of impact, the effect of the REF on academic behaviour and the timescales Hefce has set for introduction of the programme.  

Impact is a new feature of the research assessment which Hefce admits is ’comparatively untested’, and the council therefore launched a pilot exercise to test and develop the proposals, due to conclude in summer 2010. 

Most organisations responding to the consultation called for the weighting of impact to be reduced, citing a variety of reasons. The Russell Group of large research-intensive universities in the UK said that ’the introduction of impact at a level of 25 per cent carries substantial risks which are unlikely to be fully addressed by the impact pilot projects,’ and suggested that 15 per cent should be the absolute maximum considered.  

However Million+, a think tank representing new universities, said many of the concerns regarding impact are ’largely exaggerated’ or will be addressed through the pilot exercise. The organisation believes impact should be given at least a 15 per cent weighting, and research output boosted to 70 per cent.  

Stealing thunder?  

Learned societies have also voiced concerns about how impact will be attributed. Fundamental breakthroughs in science are often exploited by researchers in other disciplines where the eventual impact arises. The current REF proposal explicitly disallows the originating department to claim an impact of this nature.  

The Institute of Physics disagrees ’in the strongest possible way’ with REF proposals that it is not sufficient for research to have impact, but that researchers must have also been involved in the exploitation of that research themselves. The Russell Group agrees that ’it is not acceptable for a third-party institution to be able to take someone else’s research, carry out non-research activity to exploit this and then be able to claim the impact as part of the REF submission.’  

The Royal Society was also anxious as to how the new programme could affect academic behaviour. ’Metrics can drive behaviour. If they are narrow, they can weaken those current strengths of the system which happen not to be measured,’ the society said. It called on Hefce to ’ensure that the impact indicators used in the pilot study are carefully considered and do not provide perverse incentives’.  

Impact lessons 

In late 2009, a report commissioned by Hefce from think tank Rand Europe investigated the progress made by other countries in assessing research impact.  

A theme throughout the report is the introduction of flexibility for higher education institutions, for example adapting the weighting given to the three areas of assessment, perhaps according to an institution’s own mission plan. 

The report also highlights the need for more investigation into how to deal with attributing research impact, the burden involved in completing a reliable assessment and the commitment of end users to participate in the REF.  

’Assessing impact is perfectly doable, the question is whether the burden is worth it,’ says Jonathan Grant, president of Rand Europe. The report found that while there were good models for impact assessment, they still produced an excessive workload and undesirable perceptions in the academic community.  

The authors recommended that Hefce continues to consult widely on various aspects of the REF.  

Jumping ship 

While Hefce continues to digest the responses to the REF consultation and the report on international practice, a survey by the University and College Union showed that a third of professors would consider leaving the UK if the plans to assess impact are implemented. Of the 589 professors polled, over two thirds said they do not support the impact proposals and one in five said they had colleagues who were planning to leave if the impact agenda continues. 

Leila Sattary