Article Number: 0643
©Copyright 2000 Quirk's Marketing Research Review (www.quirks.com). All rights reserved.

December, 2000

Online qualitative research task force: report of findings

By: Casey Sweet, Jeff Walkowski

Editor’s note: Casey Sweet is president of Quesst Qualitative Research, Brooklyn, N.Y. She can be reached at 718-783-3237 or at [email protected]. Jeff Walkowski is principal of Qualcore.com Inc., a Minneapolis research firm. He can be reached at 612-377-3439 or at [email protected]. The following findings are from a collaborative effort by Qualitative Research Consultants Association members including Naomi Brody, Kim Funcik, Abby Leafe, Gina Thorne, David Van Nuys, and Foster Winter.

While the use of the Internet as a research tool is firmly established in the quantitative research industry, use of the Internet as a viable option for conducting qualitative research appears to be less well-established. The board of directors of the Qualitative Research Consultants Association established an online qualitative research task force in the summer of 1999 to explore this research medium. A summary of the task force’s work was presented at the QRCA conference in San Diego in October. Here, we provide highlights of that presentation.

The methodology

Task force members designed a three-step program to take an objective snapshot of what is going on in the online qualitative field today. This program encompassed secondary research (to summarize what the business and academic press had to say about online qualitative), a QRCA membership survey (to understand what independent moderators think of online qualitative), and in-depth interviews with clients (to understand what research buyers think about the technique).

For the secondary research, a variety of databases were used to locate articles addressing online qualitative research over the past 10 years. We looked for documentation about how much qualitative research is being done online, who is having it done, who the online qualitative suppliers are, when the technique tends to be used, and assessments of the pros and cons of the technique.

Another arm of our project was a member survey. A survey was e-mailed to a random sample of 300 QRCA members around the world. We hoped for 30 returns but ended up receiving 64 useable returns (a 21 percent response rate). Despite the larger than expected number of returns, the data was analyzed qualitatively - as originally planned. Key issues addressed in this survey included experience with online qualitative, perceived advantages and disadvantages of online qualitative, sources used for learning about online qualitative.

The third part of our project was to get the perspective of research buyers. Twenty-one clients were interviewed from a wide array of industries including ad agencies, publishing, fashion, financial services, retail, high-tech, etc. We purposely recruited a mix of those with and without experience with online qualitative. In general we covered the same issues with qualitative research buyers as we did with independent moderators.

We were uncertain about whether our findings would be consistent from one study to the other. Fortunately, we discovered a high degree of congruence across the three studies. Thus the results highlighted below are consistent across all three arms of the study (unless otherwise noted).

Size of the online qualitative research industry
We could find no quantification of the number of groups conducted online. Nor were we able to find any estimates about whether the use of online groups is stable, declining, or growing. However, task force member experience, along with anecdotal evidence suggests that it is growing.

A very small percentage of the QRCA members surveyed had had any online moderating experience. Among those who had any experience, most had conducted less than a handful of groups online.

Many Fortune 1000 companies have at least tried online qualitative already.
Advantages of online qualitative

The list of perceived advantages of the technique are consistent in the press, as well as among independent moderators and research buyers.

- No geographic barriers. While in-person qualitative typically is limited to two or three markets, and while online never make claims about the projectability of results, the online venue is better able to tap into a more widespread market for each group.

-Easier to recruit difficult-to-reach populations. These can be geographically difficult recruits (e.g., rural consumers) as well as top-level executives (with otherwise too little time to participate in an in-person market research).

-More candidness due to anonymity. The perception is that there is less social risk in truly speaking one’s mind in an online focus group.

- Less opportunity for any respondent to dominate a discussion. Because each respondent in an online session is equally as “loud” as the next person, and because there is no physical presence to deal with, respondents in an online session are more on an equal footing with each other.

- More cost-effective. Online groups eliminate the need for travel, which can represent a very high percentage of total project costs. In addition, the development of online panels and automated recruitment mechanisms can theoretically lower recruitment costs.

- Faster turnaround of results. Transcripts are an automatic byproduct of all virtual facilities. This permits the analytical process to proceed faster than it typically does.
Disadvantages of online qualitative

We uncovered several perceived disadvantages, from those with and without online qualitative experience.

- No visual or auditory cues. To some, the lack of this information negates any advantages of the technique.

- Responses online are more shallow. Some feel that we cannot get in-depth information from respondents.

- Discussion is difficult to follow. In an online chat session, respondents are, in a sense, talking over each other. And questions are often asked before all answers to a previous question have been received, making it very difficult for anyone to adequately follow the discussion.

- The online population is not representative. Even though the penetration of Internet access is high, it is nowhere near universal. Thus, we are always left with the question about how those without Internet access might respond.

- Anonymity breeds dishonesty. Some feel that the anonymous nature of an online session allows participants to take on other roles and give responses that do not really represent their own beliefs. Related to this is the opportunity for imposters to sit in for those who have been recruited to participate.

The bottom line

The task force acknowledges that the perceived disadvantages of online qualitative, coupled with lack of experience with online qualitative, serve as barriers to use of the online medium. Experiences of several task force members suggest that many of these barriers are surmountable. While the work of the task force is not complete (see next section), we want to emphasize these two points:

1) Online qualitative should not be used as a blanket substitute for all in-person and phone-based qualitative methodologies. Online qualitative is more appropriate in some circumstances than in others.

2) Online qualitative is a viable option for the professional moderator to add to his/her toolchest. Online qualitative will be embraced by some and not by others.
Next steps

The term of the online task force has been extended for an additional year. In that time, the task force’s primary goal will be to develop a set of guidelines for the membership on how to conduct online qualitative research. We will also develop more membership support activities. For example, we hope to be involved with the QRCA Web Committee in hosting an area on the QRCA Web site pertaining to online qualitative.

*