HEFCE proposals for new Research Excellence Framework (continued)
Post-scriptum to my earlier blog on the proposed Research Excellence Framework (REF), regarding the use of Thomson Scientific impact factor data as its principal quantitative measure of research quality. In its consultation document, HEFCE suggests that, according to advice it has received, Thomson Scientific Web of Science is the most reliable, all-round source of such data.
An editorial published earlier this week in the Journal of Cell Biology takes a somewhat different view. The editorial, signed by the Executive Directors of the Journal of Cell Biology, the Journal of Experimental Medicine and the Rockefeller University Press, argues that over-reliance on that single indicator would be dangerous and raises questions about the underlying validity of the data used by Thomson Scientific to calculate impact factors and therefore the accuracy of the metrics that are published.
This is well worth a read to help reflect on a fundamental assumption of the REF.
Comments
You need to be logged in to submit comments. Sign in
Chris Rusbridge (not verified) said on 21 December 2007 at 2:54pm:
It seems bizarre to me to even contemplate leaving something of such huge national importance to a single commercial supplier, who consequently acquires a monopoly hold over which literature sources are acceptable for the RAE... sorry, REF. Surely there must be a range of approaches.
Stéphane Goldstein (not verified) said on 03 January 2008 at 5:35pm:
Hello Chris - first of all, it's great to hear from you, and I hope you're now well on your way to recovery... very best wishes for 2008 from all of us here at the RIN.
Reliance on a single indicator does indeed seem strange, although from what I understand, the criticism levelled at the Thomson Scientific data relates to their usefulness as a measure of impact factors (rather than citation), which is not what REF is about. But at a more fundamental level, there can be legitimate concerns about largely basing a renewed assessment exercise on indicators of journal published output. The HEFCE consultation document leaves a bit of a door open to the possible use of other indicators of research usefulness, but only in specific areas such as applied research - and, as I stated earlier, data isn't even mentioned. RIN is consulting at the moment with a view to responding to the consultation - all views welcome!