Rethinking analytical methods – ‘messy’ analysis, the crisis of legitimacy and creative research

It most probably reflects my own approach to analysis but I enjoy reading texts that reflect upon the iterative stages of analysis and that spend time assessing the many overlapping influences that affect the analytical process as it is played out. I have noticed that students do not always leave much space to explain how they undertook to analyse the data they have. They enjoy the collection of it and go into detail around that aspect, but then, once they have their beautiful data in, there can be a tendency not to follow through. Yet it is, of course, critical to understand how to treat data in order to lead to rich findings. Instinct and immersion in a topic are both compelling reasons why good conclusions can still emerge from properly collected data, but the process of analysis, building codes on themes, reviewing and assessing the data systematically, reflects a lovely process of refinement – a term resonant with a sense of heat and ore, of smelting that can synthesis to form new things.

 
So, I have enjoyed Tim Rapley’s discussion of the analysis and coding in his Doing Conversation, Discourse and Document Analysis for the SAGE qualitative research kit (2008). He discussed briefly and straight forwardly the planning and execution of the analysis stage of research, showing by following procedures how, it is expected to be done, broadly in standard social science terms (following loosely a Grounded Theory approach). The book is not focusing on theory development nor knowledge creation but is more of a discussion of ways to think about a discourse or content analysis archive, so he does he focuses on this aspect. However, what he always likes to do is to root research processes in reality so he then outlines how each of these stages is in actuality more complex, more overlapping, more ‘endangered’ (my quotes) by contamination of other events, as well as affected by different interpretations at different moments; these can sometimes be entirely random: ‘Bizarre bolts from the blue’ gain a place in this context which is valuable, something we of course all recognize.

 
Nevertheless despite all this potential ‘messiness’ a process underpins it and his view is that this can all still remain solid enough for sensible and credible research. So this very self-reflective, sometimes circular, sometimes testing, spiral of review and re-review is all a stage of filtering the material. One might argue it is reflection that ties these different influences together but that is for elsewhere. To some extent this shows that considering texts, realities of other people, their constructions etc. should not be summarised and boxed up too much by formal process anyway. But importantly that is not to give way entirely to randomness – choices need to be made and explaining those choices is key.

 
I take some time to explain vocabulary of research with students – terms like reliability, validity – but as Rapley says, quoting Denzin (1988), the language is based on positivism; this reflects the issues that maybe the earlier social scientists faced – ie how to make your research as confidently scientific as you can in order to justify its worth, its provability – the research processes and research terms used helped to do that. The preoccupation with how to prove your work is valid still seems to dominate today though, despite this sort of discussion which is now much more prevalent. New terms are being coined for qualitative research that attempt to move away from positivism – eg some frameworks from deconstruction, social constructivism, some descriptive terms for archives and materials, ‘readerly texts’ or modes of discourse; but this does not quite solve it. Rapley sees a crisis of legitimacy and crisis of representation where rethinking is required, but even in this it reflects a nervousness around qualitative research as embodied by using the word ‘crisis’.

I am with Rapley though overall in his lucid summary of the things to do to ensure that the research is thorough and believable – he is, as evidenced throughout his book, keen to ensure practical application and real-life examples to root his book into something that can actually be done. But I wonder if all this can be summed up with the term ‘quality’ ( as used by Flick (2009), Seale (2002) etc.) without the need to protest too much and to develop a new language. While researchers are concerned with the terminology of validity – are readers (even as researchers) as concerned? I think they accept many of these points without worry – the need to for it to be done is implicit of course in all research, so the need to do it (ie follow the processes to ensure research is done properly) is key but the need to justify it is less critical. Maybe there is something here about open access and the development of new research mechanisms beyond the traditional peer review: it is still critical for trust and authenticity, and to be done properly requires money, efficiency and knowledge, but can be done is new innovative ways.
Creative research needs to take elements of this – building on the qualitative social scientists’ understanding of the nature of reality and ways to research into it as well as their approach to interpretation and constructivism, while also recognizing the pragmatism of business-oriented research. And go further with it, embedding creativity into the process – whether through methods of reflection maybe, or the representation of the research (Rapley briefly mentions this but it is something tantalising – for another time) visual methods perhaps being the most compelling and the most emergent. So more thinking around ways to emphasise quality, accepts a level of messiness while encompassing innovation is something that could be of interest for research into creative industries and creativity.

Denzin, N.K., 1988. Review of Qualitative Analysis for Social Scientists. Contemporary Sociology 17, 430–432. doi:10.2307/2069712
Flick, U., 2009. An Introduction to Qualitative Research, Fourth Edition edition. ed. SAGE Publications Ltd, Los Angeles.
Rapley, T., 2008. Doing Conversation, Discourse and Document Analysis. SAGE.
Seale, C., 2002. Quality Issues in Qualitative Inquiry. Qualitative Social Work 1, 97–110. doi:10.1177/147332500200100107