A group of research funders, sector bodies, and infrastructure experts are working in partnership to promote the responsible use of research metrics.
The Forum for Responsible Research Metrics, chaired by Professor Max Lu, Vice-Chancellor at the University of Surrey, supports the responsible use of research metrics in higher education institutions and across the research community in the UK.
The Forum have a programme of activities, including:
- advice to the higher education funding bodies on quantitative indicators in the Research Excellence Framework (REF) 2021
- advice on, and work to improve, the data infrastructure that underpins metric use
- advocacy and leadership on the use of research metrics responsibly
- international engagement on the use of metrics in research and researcher assessment
The group was established in 2016, on the recommendation of the independent review on the role of metrics in research assessment and management. The review panel, chaired by Professor James Wilsdon, published their final report 'The Metric Tide', which identified 20 specific recommendations for further work and action by stakeholders across the UK research system.
Advice, reports, and meeting papers will be made available on this webpage in due course. Full membership can be found below.
Revised terms of reference (November 2021) [PDF]
Research Excellence Framework 2021 (July 2018 and February 2017)
In February 2017, the FFRRM produced advice for the UK HE Funding Bodies on the use of quantitative indicators in the assessment of outputs in REF2021. This advice was given in response to questions circulated by the former HEFCE as a basis for discussion at the Forum's first meeting.
FFRRM response to REF consultation on outputs [PDF]
The responsible use of metrics
Metrics form part of an evolving and increasingly digital research environment, where data and analysis are important. However, the current description, production, and use of these metrics are experimental and open to misunderstanding. They can lead to negative effects and behaviours as well as positive ones.
Responsible metrics can be defined by the following key principles (outlined in The Metric Tide):
- Robustness – basing metrics on the best possible data in terms of accuracy and scope
- Humility – recognising that quantitative evaluation should support, but not supplant, qualitative, expert assessment
- Transparency – that those being evaluated can test and verify the results
- Diversity – accounting for variation by research field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system
- Reflexivity – recognising and anticipating the systemic and potential effects of indicators, and updating them in response
Membership
The forum is chaired by Professor Max Lu, Vice-Chancellor at the University of Surrey. Its members are representatives and experts from university management, academic leadership, research administration and research metrics.
The partners recognise the wide interest in this work and expect to contact and involve other stakeholders in their discussions. The membership is provided below.
- Professor Max Lu, Vice-Chancellor, University of Surrey (Chair)
- Steven Hill, Director of Research, Research England (a Council in UK Research and Innovation)
- Ian Viney, Director of Strategic Evaluation and Impact, Medical Research Council (a Council in UK Research and Innovation)
- Emily Richards, Data Insights Manager, Digital and Data Team, The Wellcome Trust
- Charlie Coyte, Policy Analyst, Universities UK
- Victoria Moody, Research Strategy Lead, Jisc
- Jennifer Stergiou, Director of Research and Innovation at Northumbria University (representing the Association of Research Managers and Administrators)
- Professor Roger Kain, Professor of Humanities, School of Advanced Study, University of London and Vice-President (Research and Higher Education Policy) British Academy (representing the British Academy)
- Professor Linda King, Pro Vice-Chancellor, Research and Global Partnerships, Oxford Brookes University
- Professor Richard Catlow, Professor of Chemistry at University College London (representing the Royal Society)
- Keith McDonald, Assistant Director, Research and Innovation, Scottish Funding Council (representing the UK higher education funding bodies outside England – the Scottish Funding Council, the Higher Education Funding Council for Wales, and the Department for the Economy Northern Ireland)
- Professor Mike Thelwall, Professor of Information Science, University of Wolverhampton
- Professor James Wilsdon, Professor of Research Policy and Director of Impact and Engagement, University of Sheffield
- Professor Derek Woollins, Vice Principal (Research and Innovation), University of St Andrews
- Joseph Marriott, Policy Adviser, Research England (Secretary)
For further information please contact Joseph.Marriott@re.ukri.org.