Impact: say what?

Added by Davidpapapostolou on 29 July 2009 11:58

As part of my internship at RIN, on July 16th I attended a conference organised by the Academy of Social Sciences, simply entitled Research Impact Conference.

It had been organised in the context of the RAE 2008 outcome and the announcement of the forthcoming REF. A follow-up conference would take place in the autumn soon after the publication of HEFCE?s proposal for the REF.

The aim of the conference seemed to be to stir thoughts around the notion of ?impact? as imposed on social sciences by HEFCE and research councils. To illustrate this, the first couple of talks that are summarised below, as well as some of the key points that came up in most talks. Furthermore, all presentations are available from the AcSS website.

Phil Cowley, a social scientist who once worked for treasury, kicked-off with a talk entitled ?Resarch Impact, a view from government?. Not so much a view from treasury after all (a journalist being in the audience, PC decided to change the content of his talk), but rather a mood-setting talk. The overall message, which came back many times during the day, was that social scientists should make a point to embrace the notion of impact and ?make it work for themselves?, corrupt it somehow. To do so, Phil argued that a few points needed to be addressed urgently. Firstly,  the need for an incentive to disseminate work to media, charities, government, NGOs, who are the end-users of the research; this theme also came back during most talks as well as many questions from the audience. Secondly, social scientists have to make a point to question the relevance of their work outside academia, which is not in opposition with conducting basic and theoretical research.

Questions from the audience also raised a couple of interesting point. One that came up a few times during the day (echoing PC?s first of two points above) was about the necessity to involve end-users in the creation of impact by increasing their use of research. It was noted that this is particularly an issue when the end-user is government, but not so much with NGOs and charities. Generally speaking, the discussion gravitated around the definition of impact as degree of relevance to the end-users. A few delegates mentioned their success in working closely with practitioners and engaging in an exchange of information that seemed very beneficial to both parties. Many others seem to build evidence from interviews but it wasn?t clear whether the interviewees were later offered to give feedback on the study. The ESRC funded project ?Future of Work? was highlighted by Steven Wooding (of RAND Europe) as a very successful one in many respects and particularly in liaising with the new labour government, freshly in office at the time of the study.

The second talk was given by HEFECE?s REF program manager Graeme Rosenberg, Needless to say this one was greatly anticipated and might partially explain the large attendance. According to Gaeme, the REF aims at reducing the administrative burden of the RAE; looking more explicitly at the economic and social impact of research; monitor the system so as to favour the creation of impact, which was defined as ?world leading work with visible benefit: economic, social, cultural, public policy ??
Graeme then presented the three stage structure of the REF framework:

  1. Output from a unit of assessment reviewed by experts to assess evidence in terms of reach and significance; in some disciplines the use of bibliometrics will be implemented (most likely restricted to STM and medical research);
  2. Impact, to contribute by 20-30% to the overall REF outcome, will be described in bids through narrative statements and case studies, supported by sets of relevant indicators chosen by assessors from a menu shared by all disciplines. Example of indicator: sustained income from users (e.g. charities, industry);
  3. Environment: seems to combine aspects from both points above: narratives and cases studies illustrating the quality of the research environment.

Of course many hurdles lie ahead of the REF and some challenges in measuring impact were acknowledged by all speakers, including: the issue of time lag between the research process and the impact, many years later in most cases, well beyond the 3-4 years of the RAE/REF exercise; the attribution of impact to a given research output and the need for case studies to show how research feeds into impact.

From Graeme?s talk I gathered that HEFCE will keep the notion of impact relatively open; partly to please everyone and partly because it has no choice: how can such a loose term describe and bear meaning in so many disciplines? Impact has to be understood in its broadest sense, in order to do justice to all disciplines and hints were given that even prospective impact would be considered.

Sadly it was hard to see any dialogue taking shape between the audience and Graeme Rosenberg. My overall impression was that Graeme was not willing and /or in a position to answer them, even though answers were often embedded in the questions.

The notion of impact as it is used these days seems to work, to use another batch of buzzwords, from the top-down (treasury wants to see impact as economic reward for its efforts) and more importantly for researcher, not quite from the bottom-up but rather in a dialogue with society. It informs us about the way scholarship is perceived at large in society as well as how this feeds into how researchers perceive their activity. This was interesting on this point of view to hear many speakers emphasize that at the end of the day, impact is what you want to make of it (the ?make impact work for social sciences? moto). Very appropriately, Elliot Stern suggested that the social context of the ?the creation of impact? and the RAE/REF exercise ought to be studied by social scientists.

© Research Information Network 2005–2009