How to Run a Focus Group for a Delphi Study or Survey

Does the thought of moderating your first focus group make you anxious? Or, have you moderated focus groups in the past and are looking for a way to upgrade your technique? Maybe you are considering a focus group for your research project and don't know where to start?

Although not always easy to perform, focus groups are a great way to rapidly develop new ideas. Compared to other forms of information collection, focus groups are very fast and very cost-effective. Often, a focus group of 60 to 90 minutes can obtain information that may have taken weeks to obtain otherwise.

Focus group are typically  used for one of two objectives: consensus or diversity.

Firstly, focus groups can be used as a tool to gain consensus. Here, focus groups are asked to debate an issue and come up with a group decision on how to act. For this to work well, the issue to be discussed must be relatively well encapsulated and defined. In addition, the focus groups' participants must be carefully chosen to ensure that they have the knowledge to adequately debate both the desirability and practicality of their choices. When the primary goal of the focus group is consensus, the result is a set of consensus statements that the group believes should be acted upon.

Alternatively, and perhaps more effectively, focus groups can be used as a vehicle to develop diversity of opinion. In this situation,  the emphasis is to identify a large quantity of information rather than looking for consensus. The focus group is tasked with developing this information to potential ideas, without addressing whether the group members personally agree with the idea. Usually, this  information is then used to feed a second part of the project: often a survey or Delphi study. Why bother to do a focus group as a prelude to a survey or Delphi study? Surveys and Delphi studies are expensive. And, errors made in the planning of these studies are often not-correctable. For instance, if you fail to ask a question on a survey, there is no easy way to go back and get that information. When the primary goal of the focus groups is diversity, the result is a set of statements that the group believes should be explored further.

Consider this example that we will use for our discussion. A group of city planners is working to improve bicycle safety for its citizens. The city quickly realized that there were too many stakeholders to simply set up an advisory panel. The city initially planned a survey, but then found they were unable to even decide which questions should go on the survey. They decide to host a focus group before developing the survey. The goal of the focus group will be to develop a set of important ideas that will be used as questions on the survey. The emphasis for the focus group will be diversity of opinion, not consensus.

Unfortunately, hosting a focus group is not always easy. And, if not properly performed, focus groups can lead to harmful bias in the study which can destroy the utility of the results. In this post, we will discuss how to host a focus group for a Delphi study or survey. Fortunately, five simple rules will help you develop an efficient and effective focus group.

1. Do Be Agile

Small and agile focus groups are ideal. Remember that the goal of the focus group is to rapidly discuss numerous ideas. Usually, a group size of 8 to 12 people is ideal. When there are too few people, it may be hard to get everyone to engage in the conversational approach that is needed. When there are too many people, the group may fractionate into small subgroups, again hindering the conversation. I like to follow the Amazon Two Pizza Rule. Every team should be small enough to have lunch with two pizzas.

Getting the composition of the focus group correct is also critical. The team composition must favor diversity and inclusion. Ideas that are not brought up in the focus may group will probably not appear in the study, making it critical that a diversity of opinions are heard at the outset. Who should be present? Do we want experts on the topic? Opinions of the lay-person? This will depend on the objectives of the study : objectives which must be clearly set before the focus group is held. Who shouldn't  be present? To encourage free flow of ideas, the focus group should probably not include anyone's boss or supervisor. Having a supervisor present will tend to make the focus groups more intent on pleasing the supervisor than creating new ideas.

In our example, how will the city planners set up their focus group? The research team agrees to meet to discuss the planning of the focus group participants. They decide that the focus group will include 8 to 12 people with an interest in bicycle safety. They make a list of the stakeholders in bicycle safety: bicycle riders, city planners, health care professionals, and others. In conclusion, they plan  a focus group to include: cyclists from the community, city engineers, health care providers that treat cycling injuries, representatives from a bicycle manufacturer, sports store managers and children who ride their bikes to school.

2. Do Set a Flexible Agenda

Remember that one of the great advantages of focus groups is the flexible arrangement to allow free flow of ideas. The agenda should encourage this flow of ideas, not suppress it. It is important to give guidance to the group through some type of agenda, but it is a mistake to give a rigid time-based program.

Good agendas for focus groups provide ideas for discussion, not specific yes/no questions to answer. For instance, a good agenda may provide topics for discussion such as "what are the barriers for employees returning to work after COVID-19," or "what topics should be taught in high-school math." Overly specific agenda items such as "should employers be forced to allow employees to continue to work from home," or "should integral calculus be taught in high-school math," are too constraining and should be avoided. Furthermore, the focus group should be allowed to address the issues in any order they prefer. The best focus groups will move fluidly from topic-to-topic in a conversational manner and need only encouragement to keep going.

How much information should the participants be given about the agenda before the meeting? This will depend on the study design and the knowledge-base of the participants.

When the topic is very innovative and few established ideas are present, minimizing the amount of information given to the participants can be a great way to encourage new and innovative ideas and avoid example containment.

On the contrary, if there are already established guidelines on the topic, it may be useful to forward them at the outset to the focus group participants. I spoke to Eric Weinstein from the CRIMEDIM Center for Research and Training in Disaster Medicine, Humanitarian Aid, and Global Health, who has been involved in several focus groups that served as the initial step in Delphi studies. Eric prefers that his research team performs a scoping literature review before the focus group to extract themes and subthemes.. Existing guidelines are extracted from the published literature and distributed to the focus group participants before the meeting. The meeting agenda includes discussion of these guidelines, and, participants are given the opportunity to suggest other topics that they would like to have included in the first round of the Delphi study.

Returning to the city planners in our example, a very loose agenda is set down. The discussion will be initiated with a broad statement such as "what do you think are important issues in bicycle safety." From there, the focus group participants will be encouraged to engage in a spontaneous and natural discussion. Topics that must be addressed, such as helmet use, will also be added to the agenda to ensure that they are discussed. However, these specific topics -- if not spontaneously discussed by the participants -- will be brought up toward the end of the meeting.

3.Do Be a Moderate Moderator

Focus groups are guided by a focus group moderator - not a focus group leader.

What makes a  good focus group moderator? In general, most people are more comfortable sharing information with someone who shares many of their characteristics: such as language, experience, and demographics. The moderator should "fit in" with the rest of the group. Good moderators are curious rather than authoritarian. And, although some knowledge of the topic is necessary for a moderator, they should not present themselves as experts on the topic but rather curious facilitators of conversation.

How does a good moderator act? By moving fluidly through the agenda from the general to the specific. Moderators should be flexible and direct the conversation only enough to keep the participants on topic. It is important to allow the conversation to wander naturally.

Recently, many focus groups have been forced into online-only venues, where the participants are not in the same room together but connect remotely via some type of web-based conference platform. While far from ideal  (focus groups held in-person tend to be far more naturally conversational) this is simply a reality for many researchers. Mediating online focus groups is even more challenging, and moderators will often need to be more actively involved . For instance, it is far easier for participants who are shy to remain uninvolved in online groups, and the moderator will need to be more vigilant in encouraging the more reserved participants to voice their ideas. It is also much more common during online venues for one or two participants to dominate the conversation. The moderator may need to intervene.

Choosing a moderator for the city planner bicycle safety project should not be difficult. The moderator should be someone who is familiar with riding a bicycle and ideally familiar with the present facilities in the city. For instance, the group might find a leader of a local cycling club to facilitate the discussion.

4. Don't Aim for Consensus

The aim of most focus groups is not consensus: it is just the opposite. In most cases, the objective of a focus group is diversity over consensus.

In a previous blog post on Delphi Studies , we discussed the importance of developing high-quality statements for the initial rounds. Focus groups can be a great way to develop these statements, provided that the focus group participants remember that their objective is to provide a set of statements for the Delphi study, not to debate their personal views. Moderators should be very attentive to watch for groups that get stuck in an argument or discussion trying to convince one another of their views. In this case, the moderator may need to step in to ask, "Do we all agree that this is a topic that should be explored in the next step of the study?"

Should focus groups ever aim for consensus? Although some focus groups are assembled with the specific intention to obtain consensus on a topic, this consensus should always be treated with healthy skepticism. Focus group consensus does not always generalize well to the population at large. Remember that the composition of a  focus is usually the result of the very specific choice of participants made deliberately by the research team: they are not usually a random selection of the whole population. It is usually not appropriate to extrapolate the consensus of the focus group to the general population. Statistically, focus groups should never be treated as a random sample unless they are truly chosen at random.

For the bicycle safety focus group, diversity of opinion is the primary goal for the bicycle safety project, not consensus. The focus group need only come up with ideas that can be taken to the next step of the project. For instance, helmet use will undoubtedly be a topic of discussion for the group. What if the focus group gets bogged down in discussing the merits of mandatory use of helmets by children and adults? It is very likely that members of the group will not agree, and may get fixated on trying to convince one another. It would be up to the moderator to step in and say, "do we all agree that mandatory helmet use for adults is an important topic and needs to be addressed?" If yes, then the statement is added to the final results of the focus group and the group is encouraged to move on to another topic.

5. Don't Remember Everything

The moderator of the focus group should not be expected to remember all the details of the discussion. Some form of recording the focus group is mandatory.

Low tech methods are often successful. For small groups with very specific objectives, just having the moderator write the concepts on a whiteboard as the group discusses them may be adequate. When done, the moderator should attempt to retain the language and tone of the focus group. Having a dedicated note-taker can also be useful.

More technical options are also available. Audio recordings are effective, low-cost, and unobtrusive. Video recording of the focus group can also be obtained by  allowing the research team to review it  carefully at a later date with better ability to assess the non-verbal communications of the group. Streaming video is also an option, and can allow for a larger number of observers to see the focus group live while not being overly obtrusive. This can also be an effective solution when the supervisors, projects sponsors, or company administrators want to see the venue live - allowing them to do so without presenting an uncomfortable space for the participants. Another great advantage of recording a focus group is the ability to listen specifically to the language used by the participants. Using the participants own language for development of the subsequent survey or Delphi study helps to make the study more culturally sensitive and relevant.

Regardless of the method of recording chosen, two principles are mandatory. First., if the session is recorded, the participants must be informed of how the recording is done, how it will be distributed, and who will be able to access it. Secondly, the recording  process  should remain as unobtrusive as possible. Close-up recordings of emotional participants responding to questions are cinematically beautiful. However, the goal of video recording a focus group is not beautiful cinema, but archiving the responses in a way that still allows participants to feel comfortable expressing their feelings. Usually, setting the video recorder of a tripod at one side of the room and leaving it there is preferable to having a person roving around the room recording.

The bicycle safety project team has decided that they will record the focus group on video. Participants are informed when they enroll in the study that a recording will be taken and that only the research team will have access to the recording. They will gather all the participants in one room, and the small video recorder will be set on a tripod at one corner. Just before starting the discussion, the moderator will again explain that a video recording will be taken for use only by the research team. Participants who object to the recording will be allowed to excuse themselves from the group. During the focus group, the moderator will encourage conversation but will not take notes or write on the whiteboard. Later, the entire research team will review the video together and extract the key concepts.

Using the Dos and Don'ts

Focus groups are a great way to gain information quickly. While sometimes used to assess consensus directly, a more common use of focus groups for research is for idea generation. They can be very time and cost-effective. However, like any research technique, performing them correctly is critical to obtaining valid data.

Using a focus group before a Delphi study can help focus the initial round of statements. Using a focus group before developing a survey can help ensure that your survey is culturally, socially, and academically relevant. Knowing how to run a focus group for a Delphi study or survey is vital. Whether the purpose of your focus group is to gain consensus or diversity,  following these five simple rules can dramatically increase the quality of your research results:

1. Stay agile
2. Set a loose agenda
3. Be a moderate moderator
4. Don't aim for consensus
5. Don't remember everything

Download our infographic for a quick summary of how to run a focus group for a Delphi study or survey.