On 7 December 2015 the Australian Government launched its National Innovation and Science Agenda (NISA). One of the measures under the agenda is for Australia to introduce an Engagement and Impact (EI) assessment framework, which will assess the engagement of researchers with end-users, and show how universities are translating their research into economic, social, environmental and other impacts.
The methodology for the first engagement and impact assessment, EI 2018, was developed following consultation with university and industry stakeholders through a Steering committee and working groups and a pilot study in 2017.
As part of the development of the first national assessment of research engagement and impact, in 2017 the Australian Research Council (ARC) conducted a pilot exercise. The pilot is essential for informing the development and implementation of the first full engagement and impact assessment in 2018, which will run alongside ERA 2018 as a companion exercise.
The aim of the pilot was to test the proposed approach (that is the methodology and processes) for the full engagement and impact assessment. The pilot was not designed to be an end-to-end assessment exercise but instead was used to test the robustness of a wide range of indicators and methods of assessment of engagement and impact.
A selection of 10 broad discipline groups (Fields of Research) were tested in the pilot enabling a spread of disciplines across STEM and HASS to be involved with a minimum of burden. Discipline groups that were tested by the pilot included: Chemical Sciences, Medical and Health Sciences, History and Archaeology, and Philosophy and Religious Studies, Environmental Sciences, Agricultural and Veterinary Sciences, Engineering, Education, Studies in Creative Arts and Writing, Language Communication and Culture, and Indigenous research.
Engagement and Impact were assessed separately in the pilot. Engagement assessment involved metric indicators and a narrative statement. Impact assessment involved qualitative information (impact studies) supplemented with quantitative information if any was available. Engagement and Impact data and information was assessed by panels comprising a mix of academics with discipline expertise and industry representatives and other end-users of research. A review panel then considered the overall methodology.
All Australian universities were eligible to participate and participation was voluntary.
The report summarises the findings from the Engagement and Impact Pilot (EI Pilot) and its review, and proposes modifications to the methodology for Engagement and Impact Assessment in 2018 (EI 2018).