The Forum for Responsible Research Metrics, chaired by Professor Max Lu (Vice-Chancellor at the University of Surrey, supports the responsible use of research metrics in higher education institutions and across the research community in the UK. The Forum have a programme of activities, including:
· Advice to the higher education funding bodies on quantitative indicators in the Research Excellence Framework (REF) 2021
· Advice on, and work to improve, the data infrastructure that underpins metric use
· Advocacy and leadership on the use of research metrics responsibly
· International engagement on the use of metrics in research and researcher assessment
The group was established in 2016, on the recommendation of the independent review on the role of metrics in research assessment and management. The review panel, chaired by Professor James Wilsdon, published their final report 'The Metric Tide' which identified 20 specific recommendations for further work and action by stakeholders across the UK research system.
Advice, reports, and meeting papers will be made available on this webpage in due course. Full membership can be found below.
UK forum for responsible research metrics - revised terms of reference (November 2018)
UK Progress towards the use of metrics responsibly (10 July 2018)
This report outlines the FFRRM's work to date, reflects on the progress made by the UK since the Metric Tide report in 2015, and the chair sets priorities for future work.
UK progress towards the use of metrics responsibly 10072018.pdf
Research Excellence Framework 2021 (July 2018 and February 2017)
The advice from the FFRRM on the quantitative indicators for REF2021 environment and the standardised measures of impact can be downloaded below.
FFRRM advice to REF2021 panels on impact and environment indicators 26July2018.pdf
In February 2017, the FFRRM produced advice for the UK HE Funding Bodies on the use of quantitative indicators in the assessment of outputs in REF2021. This advice was given in response to questions circulated by the former HEFCE as a basis for discussion at the Forum's first meeting.
FFRRM response to REF consultation on outputs 2017 01Feb2017.pdf
The responsible use of metrics
Metrics form part of an evolving and increasingly digital research environment, where data and analysis are important. However, the current description, production, and use of these metrics are experimental and open to misunderstanding. They can lead to negative effects and behaviours as well as positive ones.
Responsible metrics can be defined by the following key principles (outlined in The Metric Tide):
Robustness – basing metrics on the best possible data in terms of accuracy and scope
Humility – recognising that quantitative evaluation should support, but not supplant, qualitative, expert assessment
Transparency – that those being evaluated can test and verify the results
Diversity – accounting for variation by research field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system
Reflexivity – recognising and anticipating the systemic and potential effects of indicators, and updating them in response
The forum is chaired by Professor Max Lu, Vice-Chancellor at the University of Surrey. Its members are representatives and experts from university management, academic leadership, research administration and research metrics.
The partners recognise the wide interest in this work and expect to contact and involve other stakeholders in their discussions. The membership is provided below.
· Professor Max Lu, Vice-Chancellor, University of Surrey (Chair)
· Steven Hill, Director of Research, Research England (a Council in UK Research and Innovation)
· Ian Viney, Director of Strategic Evaluation and Impact, Medical Research Council (a Council in UK Research and Innovation)
· Chonnettia Jones, Head of Insight and Analysis, Strategy, The Wellcome Trust
· Leonie Shanks, Policy Researcher, Universities UK
· Rachel Bruce, Director, Open Science and Research Lifecycle, Jisc
· Stephanie Bales, Director of Research and Innovation Services at Teesside University (representing the Association of Research Managers and Administrators)
· Professor Roger Kain, Professor of Humanities, School of Advanced Study, University of London and Vice-President (Research and Higher Education Policy) British Academy. (representing the British Academy)
· Professor Linda King, Pro Vice-Chancellor, Research and Global Partnerships, Oxford Brookes University
· Professor Ottoline Leyser, Director, Sainsbury Laboratory, University of Cambridge (representing the Royal Society)
· Keith McDonald, Assistant Director, Research and Innovation, Scottish Funding Council (representing the UK higher education funding bodies outside England – the Scottish Funding Council, the Higher Education Funding Council for Wales, and the Department for the Economy Northern Ireland)
· Professor Mike Thelwall, Professor of Information Science, University of Wolverhampton
· Professor James Wilsdon, Professor of Research Policy and Director of Impact and Engagement, University of Sheffield
· Professor Derek Woollins, Vice Principal (Research and Innovation), University of St Andrews
· Claire Fraser, Senior Policy Adviser, Research England (secretary) Claire.firstname.lastname@example.org
For further information please contact Claire.email@example.com