Surveying satisfaction: the politics of dread and desire

Last week, a number of members of my PGCE course struck up discussion about the history and politics of UK the National Student Survey. I thought about preparing a handout with some references on the theme. And then I decided to stop pretending like I think that is sufficient for opening up possibilities for actually talking about things, or pretending that I want to perform sort of neutral arbitration of information. My references are still below. If you’d wisely like to make up your own mind through a more balanced review of the debate, the HEA’s ‘Annotated bibliography of key resources’ on the NSS is a good place to start.

If only I had the time to write another piece on the violence of ranking in education, I would love to think through the NSS in a serious way. We all should have been doing this for some time, of course. For while the NSS feels like a permanent part of the inevitable dailyness of university of life, it has, in a very short time, become an extraordinarily powerful technology which enables the marketisation, metricisation and governing of knowledge, and disciplining of academic labour and pedagogical possibility (see Kelly and Burrows 2011). It is, of course, one of many. Roger Burrows has depressingly suggested that ‘it would be quite easy to generate a list of over 100 different nested measures to which each individual academic in the UK is now (potentially) subject’ (2011). So why bother thinking about the NSS, when we are pressed to worry about the ‘Research Excellence Framework’ (REF), citation indexes, TRAC-ing and FEC-ing, quantified workload models, cereal-box-style ‘Key Information Sets’ and all the rest?

I see no reason not to think about anything, particularly something as unstable and contested as the NSS. There are many arguments, and many of these worth considering, that the exercise is a ‘force for good’, in so far as one of its possible purposes is to give students an opportunity to intervene in some minute and not personally beneficial way in decision-making processes in which they otherwise have no say at all. The National Union of Students supports the survey wholeheartedly, particularly as a strategic resource of material for course representatives serving on university committees — which, given its social and economic power, it certainly is. Methodological flaws of the NSS notwithstanding, the desire for student empowerment in universities is obvious.

I have three concerns about the NSS, presented here in order of degree.

One is that it is believed by sociologists to be methodologically flawed in construct validity and for the purposes of comparing institutions — which is now the main purpose for which it is used to inform student ‘choice’. John Holmwood, current president of the British Sociological Association, has argued very bluntly that universities, as institutions of higher learning and scholarship, should ‘know better’ than to allow or propogate its use in this way. Indeed, according to the Higher Education Funding Council for England,

‘The design of the NSS means that there are limitations on its use for comparative purposes … In particular, its validity in comparing results from different subject areas is very restricted, as is its use in drawing conclusions about different aspects of the student experience. One issue to be borne in mind is that, in most cases, the differences between whole institutions are so small as to be statistically and practically insignificant.’

No self-respecting social researcher would allow her data to be applied so far beyond its limitations. I’ll bet no university that is paying attention would even allow it, for fear of legal reprisal and reputational damage. The question is, why do we allow it, and why do the institutions that appear to pride themselves so much on ethical practices and excellence, embrace it?

The second concern is that the NSS is a complex mechanism, whose ambiguous politics are obscured in its representation as a form of student empowerment and an economic necessity. As with many metricised instruments of measurement used in the management of institutions today, the NSS is simultaneously a mechanism for individual student ‘choice’ (although I argue an illusory one) and a mechanism for marketisation, institutional control and individual discipline. I would venture to say that most academics who teach in the present circumstances understand that a poor NSS score can easily provoke knee-jerk demands to ‘do something’ to fix student ‘experiences’, with the expectation that a ‘reform’ will follow whether it is pedagogically sound and humanly sustainable or not. In the most competitive institutions, lecturers are threatened, coerced and disciplined on the basis of such scores. The NSS, in the context of its usage, is at least in one sense a quintessential technology of neoliberal power.

My third and deepest concern is that the NSS is a tragic rip-off for university students, and for meaningful and transformative higher learning. So much time, money, brain-power and energy pumped into a basic, quantitative, generalised survey of one’s entire university ‘experience’ — and one that students complete only when they are about to finish their studies. So much enthusiasm put into encouraging students to be ‘satisfied’ and to participate in the reproduction of a system whose benefits are decidedly unclear and whose damages in ranking institutions are becoming clearer. Students deserve to be more genuinely involved in shaping their educational experiences, in their learning, in determining the nature and politics of the universities they are members of. We, society, need them to be involved in this way. Some of us, as teachers, long for such a scholarly community. If people don’t join the university believing that they should consume it, we certainly try hard to teach them this lesson by the time they leave. Students deserve to make real choices in and for their lives — the kind that C. Wright Mills and Herbert Marcuse once wrote of — the kind that is not choosing between whatever options you are presented with, but creating your own range of possibilities with others and then choosing together between them. We are encouraged to demand that students complete the National Student Survey, and discouraged from suggesting that they might have some serious control within our classrooms and the university itself. We are assured that it is for our own ‘survival’ in the new ‘HE landscape’. What a depressing philosophy of education, not to mention of competitive survivalist politics.

I like to think about other ways of organising student ‘engagement’. When the concept of student research and knowledge production is taken seriously, exciting things do happen (journals, public scholarship, exhibitions, films). If the NSS can’t communicate or expand this sort of activity, what forms of communication might? What are effective and appropriate forms for assessing the state of, and encouraging the growth of, good higher education for all? I’d like to take the items from the NSS (for example) and reread them through the lens of Michael Fielding’s work. I’d like to work with students to imagine some alternative way of having a presence in the university. What great projects these could be. But I’d also like to discuss it with students…

Further reading

Buckley, A. (2012) ‘Unlocking KIS’, WonkHE, 27 September. Online here.

Cheng, J. H. S. (2010) Methodological Issues of Using the NSS to Rank UK Universities. Thesis submitted to the University of Oxford for the degree of D.Phil., online here.

Cheng, J. H. and Marsh (2010)  ‘National Student Survey: are differences between universities and courses reliable and meaningful?’ Oxford Review of Education, 36(6): 693-712. Abstract here.

Fielding, A., Dunleavy, P. and Langan, M. (2010), ‘Interpreting context to the UK’s National Student (Satisfaction) Survey data for science subjects’, Journal of Further and Higher Education 34 (3): 347-368. Abstract here.

Higher Education Academy (2012) ‘The NSS: Annotated bibliography of key resources’. Online here.

Holmwood, J. (2011) ‘Code of practice needed to prevent degree course mis-selling’, Research Blogs, 7 Februrary. Online here.