Karen O'Reilly

Please resist the drive to turn your qualitative work into meaningless numbers. 

Are you being asked or expected to turn your qualitative findings into quantitative or positivist results? There are several reasons why this is not a good idea (and we don’t talk about this enough to be honest). 

First, when you give a result as a number – say for example 7 out of ten men spend several hours a day gaming. You are implying that if you were to ask all men, 70% of them do the same. You are implying statistical inference. 

But you can only make such inference if you have sampled a large number of men (not just 10) and you’ve included all sorts of variability within that sample, and you expect that sample to numerically reflect differences in the population of men that are relevant to the thing you’re interested in. So, you probably should have included in your sample, some younger men, some older men, some working-class and middle-class men, men of different ethnic backgrounds, and so on. Then, your 7 out of ten is implying that, of all those men, regardless of differences, 7 out of ten spend several hours a day gaming.

In other words, you can only make statistical inference when you have numerically sampled for relevant variability. When you have collected qualitative data, it is very unlikely that you have done that kind of statistically representative sampling. So, when you present results, in that way, you are not being truthful or faithful to your sampling, you are doing bad quantitative research. (to read more about qualitative sampling click here) 

Qualitative researchers are usually not trying to be statistically representative but we still hope to represent types of people, together with types of condition, types of circumstances and so on. If some men enjoy gaming, we are interested in what it might be about those men that makes them make that choice, how they feel about it, and so on. 

Thinking about the craftmaking project that I’m working on at the moment, lots of people I have spoken to are finding it difficult to sell enough stuff to make it a profitable pastime. I’m not going to say 20 out of 30 because then you’re going to assume that I mean 2/3 of all crafters are struggling and you will also hear – without me saying it – that one third are not struggling. That’s another reason not to quantify. 

When we numericize, we tend to dismiss the minority response.  

Yet that minority might have something very insightful that we can learn from.

In telling you that several people are struggling, I would not stop there. My job as a qualitative researcher is to give you details around why they’re struggling, in what areas are struggling. What types of people are struggling with what types of craft? 

And then, if there are those who are not struggling, those who are doing really well, then my job as a qualitative researcher is to talk to you what it is about their experience that is working better in terms of them selling. In other words, what can we learn about those who are struggling and those who are not?

Qualitative researchers are interested in the why, wherefore and how, the processes, in understanding the experiences that drive outcomes.

This is going to be much more useful. If for example I wanted to tell the organisers of craft fairs how to improve the chances of craft makers, knowing that one in three are doing okay is not going to help at all. But if I tell them that those who are doing better, are doing better for this reason and that reason, then this is something we can explore. 

 if I tell you that 10 out of 30 people are doing alright, you might just hear that and ignore the others. 

So to reiterate, the job of qualitative research is to explore, examine, think about pathways and interconnections about meanings behind decision-making, about assumptions and so on. Turning qualitative insights into numbers is not insightful and it is poor qualitative and poor quantitative work. 

Using qualitative insights to design a survey can be an interesting thing to do. If I did want to know how many people are struggling out of a certain number of people, then it might be good if I turned what I’ve learned into a survey, but I will be asking much more than ‘are you struggling’. I would ask how often they do craft fairs, what sort of things they’re selling, how much the materials are, what their stall looks like, how the rest of their life looks, and so much more. It would be a massive survey with a very long questionnaire. I know what to ask, because I’ve done very good qualitative research to inform that survey, but I’m not really sure that I then need to survey because I’ve got the meaningful understandings already.When you analyse qualitative research, please don’t focus too much on how many people said this and how many people said that. Instead ask what can we learn from these people that is going to be useful, or meaningful, or insightful – so that we can learn from them about how to do things differently, or how to think about things in a different way, or we can hear voices and topics that have not been heard before – new perspectives,  new lessons, new interconnections and pathways. For more on interpretive analysis, click here

Please see my book Qualitative Research Methods for Everyone for more on this. You might also enjoy The Qualitative Research Methods for Everyone Podcast https://shows.acast.com/qualitative-research-methods-for-everyone-podcast/episodes