Focus groups aren't worthless
I first read about Erika Hall's hate for focus groups in her A Book Apart book 'Just Enough Research'. It rubbed me up the wrong way then but I was prepared to overlook it as Erika's book is one of the main user research books I recommend to people who want to do more research.
Yesterday Erika wrote in more detail about why she has a problem with focus groups as a methodology on Medium, 'Focus Groups are Worthless.' I found myself again disagreeing with her stance and rather than send several 140 character tweets about it, I thought it might be useful to write some of my thoughts down in long form.
The Medium piece started with this:
"If I achieve one thing with my time here on earth, I might be content if that one thing could be burning to the ground the practice of running focus groups in place of actual user research."
The piece mainly talks about the use of focus groups as a user research tool but there are several places where the whole practice of focus groups as a useful methodology is questioned:
"A focus group is an artificial construct that is so much about the group dynamic."
"If you are doing market research, and want to keep doing focus groups because that’s your jam as well as your bread and butter, don’t let me stop you (although I invite you to stop yourself). But if you are doing research to inform the design of a product or service, run far away from that two-way mirror."
Now, whilst I don't tend to recommend focus groups to anyone doing user research and haven't actually commissioned or run any myself for the last 5 years whilst running research projects for our clients at Mark Boulton Design, I don't think this is a helpful way to talk about a whole industry (Market Research). It implies that all projects relying on focus groups are worthless. It seems to imply that user researchers are better than market researchers and have better tools. This is either extremely arrogant or extremely ignorant.
Of course, choosing the right methodology for research always depends on the scale of the brief, the goals of the work, the audience, the context, who the stakeholders are and the product or service. I recently wrote about the Research Funnel, a category system I have been thinking about for commissioning and planning user research at Monotype. I talked about how research should be designed to address a problem - either to solve it or to get closer to a solution and there are a range of ways to get closer to a defining a problem or a solution.
One of the ways of getting closer to a problem is to run focus groups. An example of this is a large strategic research project I am currently working on. The first stage of my research is internal, individual stakeholder interviews and perhaps some focus groups with various internal teams to establish 'what we already know' and to further define what we think the gaps in our knowledge are and the problems we want to investigate further.
I once worked on a client project where we suggested internal and external stakeholder interviews during our sprint zero. The client came back and said there were 24 people she wanted me to interview. These 24 people were both a range of people with useful experiences and knowledge but also crucial to get 'on board' with the project at an early stage. When we costed this, it soon became clear that this wasn't going to be possible as most of the budget would be used before we'd even got started on the project. We scaled back the number of interviews and did further research later in the project instead. I would also have had no problem suggesting internal focus groups if the team dynamics and company culture had been right for it.
'Buy in' is such an important thing for a lot of projects, especially when a big change occurs to a service or product. Involving people in research - both directly and indirectly can be crucial to its effectiveness as I've written about previously in the Danger of the Big Research Reveal.
"...the times I've seen the most 'lightbulb' moments have been when clients, producers or designers have been fully immersed in the research process and not just a receiver of research findings. Just like in design/client relationships, there is a real danger with a 'big reveal' for research projects too. The Research Debrief, PowerPoint strategy or the 32 page report is not the research. It's an artefact of a conversation, of a collaboration. Being involved in the process is the research and will give you, the decision maker, the insights you need."
There are plenty of ways to involve people in research. I've used plenty of methods including getting my clients at CERN to do their own internal interviews with people we were designing a new app for; sending BBC Wales Senior Managers on 'home visits' to meet their audiences in the context of their own homes and observe life during the early evening when the BBC news programmes are scheduled; I've used members of the audience in ideas workshops; and I've got designers designing their own surveys and running user interviews.
I have also commissioned focus groups and invited along managers and production staff to observe them. Yes, in fact, I spent a large portion of BBC Wales's sizeable research budget on focus groups. And it was money well spent for the most part. I have worked with some really fantastic researchers at Market Research agencies in the UK who are smart and progressive and aren't afraid to suggest bold methodologies. I have watched many really well facilitated focus groups by these very researchers. It's quite a skill to do well and I have to say that after facilitating groups myself, it's easier to sit on the sidelines and find fault.
Group discussions can be extremely useful and can evoke responses that interviews might not (if well recruited). Some of the most interesting and useful focus groups I watched were where the moderator used collaborative techniques such as respondent mood boards to produce visual representations of a brand. This kind of exercise really doesn't work well with individuals and needs pairs or more to help tease out interesting discussions. I've seen this work well with card sorting for IA too.
Another example of groups being more useful than depth interviews is topics that people have very strong opinions about and can be recruited accordingly such as TV 'soap' fans. The context of TV viewing is often social so friendship groups are a perfect example of a good way to get around people feeling nervous to share their opinions in front of strangers. Watching people argue about whether Eastenders or Coronation Street is best was very enlightening!
An interview situation is also an artificial construct and especially when 'user testing' in a 'lab'. I'm not a big fan of labs but understand the reasons for when they are necessary, some of which I have talked about - people being able to observe research, cost effectiveness, ease of managing and so on. You could throw the same criticisms at labs that Erika throws at focus groups:
"Design for the real world, not the fictional one you’ll hear discussed on the other side of the looking glass."
"User research should be ethnography."
True ethnography is the study of groups and can happen collectively or with individuals. The researcher is supposed to observe and record what happens without interfering too much with events and remaining distant. This isn't really what happens with most user research in my experience. I can't say I've observed anyone officially in this way. I prefer to use the word 'contextual'. User research should be contextual. The context of the customer/respondent/audience should be considered but also the context of the research remit - who is a stakeholder, what the goals are and what the outcomes will be.
Research often has a hard time being taken seriously, clients won't always pay for it and designers don't always have the time or remit to do their own. Debunking a methodology and a whole research industry is damaging and unhelpful. I'm all for discussion of appropriate methods and new ideas but in a positive, supportive way.
As researchers we should be promoting best practice and facilitating the conversation about research but not by slinging mud. I want the designers I am encouraging to do their own research to be confident in knowing when to pick one type of method over another but I don't want someone's whole research project to be thought as rubbish by someone else because they once read an article that said focus groups are worthless.
There is no 'one size fits all' in my research toolkit. There is no black and white 'this thing is good, this thing is bad'. There is a lot of grey and a lot of it depends. I'd encourage you to take the same approach.
