 |
||||||||||
 |
||||||||||
 |
||||||||||
 |
||||||||||
 |
||||||||||
|
|
||||||||||
|
Published in Qualitative Market Research: An International Journal, Research in Cyber Space 2001, June 2001. Online qualitative studies have been conducted to evaluate online and offline advertising, evaluate mock websites, critique existing websites, test and evaluate new products (product mailed in advance of the groups), uncover competitive website information, evaluate training programs, explore decision-making, uncover imagery, evaluate concepts, evaluate package visuals, generate ideas, and to ascertain customer and employee satisfaction. At present, the two most utilized online methodologies are real-time, virtual focus group rooms where six to eight respondents participate simultaneously and asynchronous online bulletin boards with (ideally) twelve to twenty respondents lasting over a period of days. Both venues allow respondents to participate from their desired location and, with the bulletin board, at a time that is most convenient for them. There are a variety of real-time and asynchronous virtual facilities offering different formats. The specific features, capabilities, and sophistication will vary according to which provider is selected. The use of either methodology requires a re-thinking and re-application of qualitative design and techniques. Some activities and techniques used with in-person groups work as well (e.g. sentence completion, brain dumps) or better (e.g. pantry recall/checks, brand verification) with online groups. Other activities and techniques need to be adjusted (e.g. brainstorming, imagery exercises), do not work as well (e.g. mind mapping, picture sorts), or do not work at all at this technological point (e.g. controlled sensory tests, paired assignments). The design and development activities necessary to conduct successful online groups are similar to offline groups, specifically: establishment of objectives, screeners, discussion guides, moderation, analysis and report writing. However, there are crucial differences, reflecting the influence of the technology, that require expertise beyond the skills required for in-person groups. Below is a summary of some of the similarities and differences between online and in-person qualitative research. Screeners, Recruitment, and Virtual Facilities. Recruiting online groups requires specially crafted screeners that are similar in content and depth as those used for in-person groups: as always, the quality of the screener can determine the quality of the respondents. Since many of the screeners are administered electronically, some questions are worded differently to disguise qualifying and disqualifying answers. A well-written screener, in combination with a professional online facility, will thank and release all disqualified respondents without them knowing why. This, as well as putting a block on their electronic address, discourages them from re-trying to qualify by logging back in or from sharing information about the specific screener questions with friends. Online groups (real-time or asynchronous) can be initiated by contacting a moderator or company that specializes in online focus groups. Most will provide turnkey solutions and have a virtual facility that they work with regularly that provides virtual rooms as well as recruitment services. Respondents can be recruited electronically from established panels, compiled online lists, targeted websites, or client-provided lists. Sometimes, telephone recruiting is utilized to recruit audiences less likely to respond online, highly valued audiences where “high touch” interactions are necessary, or to confirm and obtain email addresses. View for the Client Observers. If all is going well, most of the moderating elements discussed below will be transparent to the research observers. In fact, it may even seem slow for them as they passively sit in front of their computer watching the interaction. It is important to point out that the optimal way for the client to interact with the moderator is through one designated client liaison. Similar to in-person groups where notes are passed into the moderator, the designated liaison decides what is important to pursue and approves questions given to the moderator. These “notes” may be submitted to the moderator in private message form or entered in the backroom dialogue stream for the moderator to see. The method of communication between the client and moderator may vary depending on the virtual facility being used and the software capabilities. Invitations and Preparation. Respondents and observers that are invited to the group receive invitations with passwords and passnames, instructions, dates, and times. The invitation requests that they sign on to the site in advance of the group, using the computer they will use during the group, to guarantee that all technology is compatible. If there are any complications or questions, the respondents can contact tech support in advance to resolve them. They can also contact tech support during the group for online support, as can the moderator and client observers. Discussion Guide Development and Design. The content and structure of the inquiry resembles in-person groups. The major difference is in the actual presentation of questions that are mostly written in full sentence form, in advance. The main topic questions must be written clearly and completely or else respondents will be provoked to ask for clarification and this will delay the response process. Real-time online groups tend to last 90 minutes though shorter or longer ones are possible. Ideally, 40-45 questions are written in advance for input during the discussion. With a ninety-minute group this usually leaves enough time to insert additional spontaneous probes into the dialogue stream. Bulletin board groups can last 3 to 5 days or more depending on the amount of material to be covered. At the beginning of each day there are a set number of questions posted that all respondents answer. The number of daily questions depends on the complexity of the issues, relatedness of the topics, attention span of the respondents, and level of respondent involvement in the subject. During the response process throughout the day, additional probes can be interjected to request elaboration on topics or new subjects can be added. (Note: There is also an email version of asynchronous groups that tends to be less sophisticated and less dynamic in its interaction and this type is not covered in this discussion.) Show Rates and Selection. Show rates can vary dramatically based on a number of factors, including: the target respondent (i.e., as with in-person groups, some types of respondents are more reliable); origination of the respondent (e.g. online database, established panel, website intercept, client customer or employee lists, etc.); confirmation procedures (e.g. online only, phone only, or phone and online combined); respondent comfort and familiarity with online venue in general (i.e., respondents should indicate at least adequate keyboard skills); and the typical kinds of other personal/business commitments that can inhibit attendance. One positive influence is that the weather, traffic, and transportation have less of a negative impact on show rates since the respondents are participating from a variety of locations and not encountering the same delays. For both kinds of online groups the number of respondents recruited is roughly 50% to 100% more than the desired attendance. The final selection of respondents is based on the rescreener information and final screener spreadsheet. The moderator and client select the respondents together, similar to in-person groups. Moderating. The excitement and pace of moderating an online group, for the moderator, can be likened more to a roller coaster ride than an in-person group. Ideally, the discussion guide is downloaded directly onto the site so the moderator can, with one click, enter a question into the dialogue stream. Another method that is more frequently available and workable, (though requiring more time, concentration and actions by the moderator), is having the discussion guide document loaded in a separate window behind the virtual room to use for cutting and pasting each question. To begin a group, the moderator introduces the purpose of the group and lays the ground rules. This includes a personal introduction, purpose, timeline, instructions for entering responses, encouragement to be candid and honest, and instructions for signing back on if they accidentally drop off. Respondents are also encouraged to “feel free to agree, disagree, or ask questions of each other that relate to the subjects being discussed” and told that this interaction will help bring the discussion to life. Online groups demand that a moderator possess strong and fast keyboard skills or be willing to hire an assistant who does. There are no unused moments during a group to accommodate slow typers on the moderator side. While a group is under way, there may be technical problems with respondents and clients requiring telephone calls back and forth to resolve. Simultaneously, the moderator is reading and interpreting the dialogue stream, responding to client notes, composing probes and entering questions and all the while (potentially) dealing with various technical issues. Also, moderating online groups requires someone who relates to the online venue and recognizes that respondents are adept at developing relationships in this medium. Many respondents have and/or do participate in chat rooms and feel comfortable relating online. At the same time, it is the responsibility of the moderator to help make the less experienced respondents feel comfortable and valuable. The strategy of online moderating resembles in-person moderating, that is, the moderator follows the discussion guide to the extent that it is obtaining the desired information. If a subject that was supposed to be covered later in the group is brought up earlier by the respondents, those questions can be inserted as determined by the moderator. In addition, if topics not covered in the guide are introduced, the moderator can choose to interject a new line of questioning. View for the Respondents. Respondents see all of the moderator’s questions and comments of other respondents as they are input. Each respondent has a name, often a pseudonym, that identifies them and sometimes the responses are color coded. They do not see any of the comments from the observers or observer notes to the moderator. Analysis and Reporting. Analysis and reporting is similar to in-person groups, with the exception that transcripts are quickly available for every group. The analysis will be very inclusive and reflect the input of most respondents since most of them answer every question. In the absence of visual and verbal cues, analysis of some areas, such as appeal, will be based on an interpretation of respondent statements and any (qualitative) ratings they use to indicate levels of appeal. The final reports cover areas similar to other qualitative reports such as Objectives, Methodology, Conclusions, and Key Findings. They can be in topline, executive summary, or full report form. Typically, reports can be turned around quicker due to the immediate availability of the transcripts. In considering the use of these new online research tools, there are a number of unanswered questions for most researchers. Some of the most frequently asked questions are: 10 Tips for Conducting Virtual Groups the First Time -Similar to in-person groups, begin with an introduction that discusses the purpose of the group, guidelines to follow, and establish the moderator as the discussion leader. (Assume respondents are not fluent in this methodology.) 10 Ways To Avoid Technical Obstacles - Moderator should have excellent keyboard skills and expect multiple and simultaneous inputs from clients and respondents. Frequently Asked Questions Who and what are being researched in online focus groups? Online focus groups are ideal for locating and researching markets that are hard to recruit, low incidence, sensitive topics, online based, and geographically dispersed. For instance, high level executives may be willing to participate in an online group, but never consider expending the amount of time required to attend an in-person group. Or, respondents connected to sensitive topics (e.g. HIV) may feel more comfortable and willing to share in the anonymous online environment. Others who have certain low incidence characteristics or unusual purchase behavior may be impossible to recruit within a geographic region, but across the U.S. or in a wider regional area they can be found. Lastly, audiences whose relationships and interactions with companies have been strictly online are good candidates for online focus group research. Some of the topics that are particularly appropriate are: e-commerce inquiries regarding purchase behavior and interest; website evaluation, development, and offers; sensitive topics allowing for anonymity; and, high-tech products and services. In some cases, online groups are used as pre- or post-phases in conjunction with online quantitative research. Some companies are beginning to experiment and show excitement about the possibilities of dovetailing in-person and online group methodologies to collect wider geographic input, gain experience in conducting combined studies, and develop comparison measures. To date, it is preliminarily suggested that online and in-person results are more similar than dissimilar. Are online groups going to replace traditional focus groups? Not in many situations. The demand for online groups is expanding the research arena and adding a new tool to the qualitative toolbox, not shrinking the demand for traditional groups. An 8” research pie is now growing into a 10” pie and will continue to grow as demand builds and new applications are created. In some cases, online groups may be used in conjunction with in-person groups or replace some of the in-person groups. This might be done to obtain a wider geographic representation, uncover similarities/differences from the online point of view, or conserve on travel expenses. However, many in-person projects will not be appropriate for online groups because the visual and vocal cues of respondents or visual review and evaluation by respondents are critical to the evaluation of the package, product, promotions, or other communications. Additionally, real-time online groups may not always provide the depth of response necessary. Are respondents who they say they are? If a database is used to recruit, the answers respondents submit can be compared to the database information to confirm their identity. In cases where little is known about the potential respondents, it requires a well-written, thorough screener, similar to in-person groups. Just prior to the group, respondents can be rescreened online to verify their identity. Or, rescreening can take place in advance over the phone as an additional check. So far, it appears that respondents are who they say they are and the threat of cheaters and repeaters is minimal. Similar to in-person facilities, each virtual facility can choose to establish their own database to document past participation to be used for comparison purposes during the selection and confirmation process. Do respondents in online groups really interact with each other? Yes, but it is not exactly the same as with in-person groups, mainly because it is perceived but not seen. Many of the individuals who participate in online groups have been online for at least a year and have participated in chat rooms so they are comfortable and adept at creating online relationships quickly. The guided discussion draws participants out and personalities begin to emerge, thereby, creating a dynamic that develops during the group and varies just like in-person groups. Some suggest that the absence of sight and sound strengthens the use of perceptual senses by not relying on subjective visual and vocal judgments and interpretations. In bulletin board groups the interaction between respondents can be extensive and on-going throughout the days. Respondents choose the best time to participate and often spend much more time than required answering questions and interacting with others. It is not unusual for them to pose additional insightful questions for others to consider and answer. In fact, this involved level of participation is encouraged from the outset. In comparing real-time groups with in-person groups it would be fair to say that the interaction in real-time, online groups can be more limited. The amount of interaction between respondents in this fast-paced environment can vary and may be influenced by the topic and moderator. At the same time, unrelated small talk and long stories are far less likely. One interesting and overall difference online is that it is much more difficult for someone to dominate a group. An overbearing respondent does not have the same power or influence with words in the dialogue stream. All respondents are composing their responses simultaneously and not waiting for others to respond. The online environment can create a more even playing field and can mitigate other influencing factors such as gender, age, ethnicity, accents, physical appearance or condition, and shyness, not to mention the weather. Also, there tends to be less inhibition online and respondents are often more direct in stating their opinions and less likely to edit their thoughts because time is of the essence and input takes time. Qualitative Caveat: Results from online groups depend on the expertise and qualifications of the professional who is conducting them. All of the information provided in this article assumes that the most knowledgeable and qualified professionals to conduct online groups are qualitative researchers who have research and marketing expertise, as well as, experience with managing group interactions. “Techies” sometimes attempt to do groups because they are comfortable with the technology and mechanics and some even have experience with chat groups. However, they often lack research, analysis, moderating, and marketing expertise and the results can suffer from these deficiencies. The future promises many imaginable advancements in online focus groups including sophisticated visual aids, real-time video and sound, accurate voice recognition, and videoconferencing of individual respondents. If the next five years bring the magnitude of changes as the past five years, there will be many more advancements, unimaginable at this time. * |
||||||||||
 |
||||||||||
 |
||||||||||
 |
||||||||||
 |
||||||||||
 |
||||||||||
 |
||||||||||
 |
||||||||||
 |
||||||||||
|
||||||||||