A ‘teaching REF’ won’t please everyone

Photo coutersy of: DFID

As a nation we prefer to read party manifestos only after we’ve cast our votes, and the media has now picked up on an entire bucket list of weird and wonderful pledges in the 2015 Conservative manifesto. One of them pertains to teaching at universities. The Tories promised, in not so many words, to introduce a “research excellence framework for teaching” if elected. For those who prefer to avoid conversations about whatever new project has possessed their tutor, the research excellence framework (REF) is a centralised procedure for assessing the quality of publicly funded research. After all, even your tutor’s work has to be marked.

The idea is to take this framework and create a new version with which to evaluate teaching. To many academics I’m sure that the idea of a second REF doesn’t sound like fun. The framework is already criticised for costing millions of pounds to do the work of one computer but much more slowly. Universities send their papers to tiny, subject-specific panels that rarely know anything about the particular topic in each piece of research, and the papers are returned with an ambiguous score. Because there are so few academics on each panel, some are tasked with upwards of 1,000 journal articles a year, giving them barely enough time to read the abstracts.

The catch is that a university’s total score determines its yearly slice of the UK’s £2 billion of block grant funding, not to mention its reputation. Consequently, some universities have set up internal steering groups to screen their academics for high-scoring research. Only papers with enough “impact” to secure generous helpings of public money are sent for assessment, further damning higher education’s record on inclusivity. Our very own Prof Derek Sayer, former head of the History department, made headlines last year complaining that he was entered into the REF when many of his co-workers weren’t.

In theory, a REF for teaching could restore the balance. Lectures and tutorials are precisely the kinds of “low-impact” responsibilities that academics have to sacrifice for their research, such that many universities are reluctant to advertise teaching-only positions. A national teaching REF could draw some of the sector’s focus back towards tutelage, improving standards for students and maybe even averting the closure of unprofitable departments like Music. In practice, however, the devil is in the detail, and detail is not the strong point of the Conservative manifesto.

Think about the methodological problems of a teaching REF. For starters, who would do the marking? A key principle of academic review is that qualified researchers are trusted to evaluate each other. In the REF’s defence, therefore, it’s very much a framework of the academics, by the academics, for the academics. With regard to teaching, though, I’m not convinced that one lecturer can reliably judge the ability of another. If anything, it’s the end users we should be consulting. It’s much easier to gauge whether or not a student is learning if you are that student.

A student-led teaching REF could be measured in several ways. We could, for instance, round up a bunch of undergraduates and ask them what they think of their course convenors. On a large enough scale, this would simply amount to a National Student Survey. Alternatively, we could cut to the chase and test how much the students have learned, or simply borrow their modular aggregation scores. Knowing the Conservatives, graduate employment rates would feature heavily in the REF, but that’s already a key indicator in league tables. None of these answers brings anything new to higher education, and each is open to as much statistical manipulation as the real REF.

Worse, a teaching REF could do to study what it’s done to research, further congesting lecturers’ timetables with course design, self-evaluation, learning objectives, mission statements, and hours of general, formless anger. The policy could also jeopardise the reputations of research universities like Lancaster. Reputation may, however, mark the limit of the damage, as the manifesto only promises a framework to “recognise” instructive universities. It doesn’t have to fund them. This would give universities no real reason to fear the teaching REF, but not much incentive to improve teaching, either.

The government has made its promise to higher education. If it wants to avoid further injury to academia, it now has to figure out exactly what it meant.