Ch 10 Book Notes - Davis Advertising Research: Theory and Practice 2e PDF

Preview

Summary

Davis Advertising Research: Theory and Practice 2e...

Description

Ch 10 Book Notes Wednesday, February 24, 2021

11:30 AM

Chapter 10: Data Collection Through Surveys Methods of Collection Survey Information • Personal Interviews ○ Occur when an interviewer administers a survey to a respondent in a face-to-face settings ○ Two main types: § Intercept: recruit respondents "on the spot" □ Generally conducted the same way regardless of location □ Malls are most common setting for intercept interviews □ Inefficient in locating individuals with less common characteristics § Pre-recruited: individuals are contacted first by mail, online, or telephone at which time arrangements for a personal interview are made ○ Provide the highest level of data quality due to personal administration of the survey by a trained interviewer ○ Have several advantages over other data collection § Work well when questionnaire is long and/or complex § Permit the use of visual stimuli § Permit the interviewer to make very certain the respondent understands

•

individual survey questions or complicated instructions ○ Primary disadvantage § High cost and large amount of time required for completion § Hard to obtain a true random sample Telephone Interviews ○ Typically conducted by trained interviewers telephoning from a central location ○ Almost all use computer assisted telephone interviewing (CATI) where a telephone interviewer reads the survey questions off a computer and types the responses as they are provided ○ Provide a opportunity for an interviewer to explain complicated instructions and questions ○ Advantages § Able to administer interviews with very complex skip patters § Have lower marginal costs because individuals in the sample frame can be contacted several times if not available the first § More efficiently obtain a true random sample, increasing generalizability and representativeness of the results ○ Disadvantages § Questionnaire must be relatively short § Question complexity must be relatively simple § Difficult to collect sensitive data as individuals are often unwilling to give this info to strangers § Extremely difficult to use visual stimuli ○ Sample selection must be done carefully b li i h i d

•

must be sure list is comprehensive and without bias Mail Surveys ○ Mailing each potential respondent a package containing a cover letter, the survey questionnaire, instructions for completion and return, and a stamped envelope addressed to the research company ○ An incentive may or may not be included in the package ○ Differ because there is no personal interaction ○ Can be conducted one of two ways § Mailings to a selected sample of the population can be conducted § A mail panel (continuing group of individuals who have agreed to participate in survey research studies) may be used ○ Advantages of mail panels § Response rates tend to be higher § Cost efficiencies tend to be greater § Selection of individuals without certain demographic or product usage characteristics can often be made without prescreening § Prescreening of low-incidence respondents can be efficiently conducted ○ Advantages of mail surveys in general § Respondent convenience § Cost efficiency ○ Disadvantages § Low response rate § Limited questionnaire length and complexity § Extended timing ○ Impersonal nature is both strength and

Impersonal nature is both strength and weakness § Allows to provide more accurate data on personal feelings or sensitive topics § Respondents do not have the ability to clarify areas of confusion or able to check that question is being answered properly Online Surveys ○ Administered over internet of mobile phone ○ Posted on central website that allows individuals to respond on their phone/computer whenever convenient ○ Has three components § Invitation □ Commonly made via email with a link § Questionnaire design and posting □ Websites allow host to design and post questionnaire without additional software § Data collection ○ Advantages § Allow for random assignment and random presentation of visual stimulus § Question bias can be significantly reduced or eliminated through randomization of question response options § Data collection is very quick § Costs are typically lower § Allows responses from all over the world giving larger sample size ○ Disadvantages § Sample composition - internet isn't widely used by people of all ages Hybrid (Mixed-Mode) Surveys ○

•

•

○ ○ ○

○

○

Use of more than one approach to data collection Process of accessing respondent susing more than one mode for the same study Could happen two ways § Different modes may be used only at the contact stage, while the sample itself remains on one mode § May also apply to questionnaire where some answer on phone and some answer online Have the potential to increase response rate and as a consequence increase data quality Have two limitations § They are complex to put into practice with different sample frames required for each mode of data collection § Hybrid designs assume that the same person would give the same response to the same question irrespective of mode, sampling frame, and sampling method

Criteria for Selecting a Data Collection Method • Cost ○ Cost reflects the questionnaire length, required response rate, geographic coverage, and sample characteristics ○ Personal interviews tend to be more expensive than other methods of data collection ○ Study cost: the initial starting point in evaluating the suitability of alternative data collection methods ○ Selecting the absolute least expensive method is always inappropriate when there is a significant sacrifice in data quality

•

•

Timing Requirements ○ Survey methods differ in their speed of data collection ○ Telephone and online require least amount of time ○ Telephone is fast for two reasons § All training and coordination can be performed at a central facility § Teams of interviewers can work simultaneously § As a result, study timing can be shortened by adding more interviewers ○ Online only has a very small lag between time of respondent contact and survey administration § Multiple questionnaires can be completed simultaneously ○ Mail surveys take a lot of time because of postage return rates ○ Personal interviews take the most time because of scheduling Sample, Interview, and Administrative Control ○ Sample control: the extent to which a researcher can control who responds to the survey questionnaire § All four methods differ in their ability to make certain that the target respondent is in fact the individual who participates in the survey § Personal and telephone provide the greatest degree of sample control § Mail and internet have least control ○ Interview control: the extent to which a researcher has control over the circumstances in which a respondent provides his or her answers to the survey § Personal and telephone are best since an interviewer can make certain the

•

•

an interviewer can make certain the respondent participates in all required parts and responds appropriately □ Immediate nature also allows interviewers to address errors in response when they occur § Online provide a relatively lower degree of interviewer control □ No assurance that respondent read all instructions and material before going to questions § Mail surveys provide least because there is no assurance that respondent will read instruction, answer questions in proper sequence, or correct errors in response ○ Administrative control: the degree to which a researcher is able to monitor interviewer quality, which can only be done in telephone or personal interviews § Supervisors at telephone facilities can monitor the quality of each telephone interviewer and correct any problems with tone, style, or questionnaire administration § Personal interviews provide least amount because a researcher can't monitor each interview Informational Needs ○ Some survey methods work better than other when the topic is sensitive or personal ○ Respondents tend to be more honest on phone and mail interviews ○ Mail surveys are better than phone at collection sensitive, personal info ○ Personal interviews are better for collecting complete and considered responses Complexity of the Topic and Questionnaire

○ ○

○

○

○

○

Some research topics are more complex than others Regardless of the complexity of the topic, some questionnaires are more complex than others Questionnaire complexity tends to increase as the number of skip patterns and verbally complex questions increases § Verbally complex questions require the respondent to expend a great deal of cognitive energy to process and remember elements or demands, for example □ Multiple choice questions with many possible choices □ Ranking questions that ask respondent to remember a large set of object and place them in order □ Constant sum questions that ask respondents to allocate points to a set of objects □ Questions that have long leadins or provide info that must be remembered prior to answering Personal interviews are most appropriate when both the topic and questionnaire are complex § Provides best setting for the discussion of complex issues Telephone interviews work well when the issue is complex and questionnaire complexity is entirely due to skip patterns § Not appropriate when complexity is due to verbal or question complexity Online are appropriate for relatively simple topics with or without questionnaire complexity

Makes it difficult to use open-ended questions to probe attitudes in complex areas □ Ability to see questions and be guided by computer permits high levels of questionnaire complexity Interview Length ○ The amount of time required for a typical respondent to complete the survey ○ Tends to increase as the number of questions, question complexity, and the number of open-ended questions rises ○ Personal contact is essential for longer surveys § Allows interviewer to keep respondent interested and attentive ○ Mail, phone, and online are only appropriate for shorter interviews § Long mail interviews are visually overwhelming and require a considerable commitment of time and energy on the part of the respondent § Telephone interviews complete with other home or business activities, making it difficult to maintain respondent participation for more than 5 or 10 mins § Long interactive interviews share many of the same problems as mail interviews which require a commitment of time and energy that most respondents are unwilling to give §

•

Response Rate, Nonresponse Rate, and Nonresponse Error • Response rate: the percentage of the valid sample who participate in the research by completing an

•

•

•

who participate in the research by completing an interview or survey ○ Low response rate can severely decrease validity and generalization of collected data which reduces the ability of the data to positively and accurately contribute to the decision-making process ○ Calculated by dividing the number of respondents completing the survey by the number of respondents in the valid sample ○ Nonresponse rate: the complement of response rate § Computed by subtracting the response rate from 100% Response and nonresponse rates are important indicators of data integrity ○ High response rate indicates that there are no meaningful differences between those who responded and those who didn’t ○ Low response rate indicates a significant problem with the source of the data § Probably bias in the data Cause of Nonresponse Errors ○ Two main sources § Refusals □ When a participant declines to participate in the study or fails to complete a survey once it has begun § Not-at-homes □ Happens when a respondent is unavailable at the time of potential contact Improving Response Rate to Reduce Nonresponse Error ○ Primary approaches to nonresponse bias due to refusals and not-at-homes are advance notification, incentives, callbacks, and recontacts

These are implemented differently depending on the data method Reducing nonresponse in personal interviews and telephone surveys § Explicitly mention the interview topic and describe why the topic is important and relevant to the correspondent § Effectively and accurately describe the purpose and goal of the research, noting the importance of the individual's responses to achieving the goal § Make certain that respondents do not feel threatened by participation in the research or by how their responses will be used § Une incentives at levels appropriate to the demands of the survey § Use advance notification to contact respondents in advance of the interview in order to gain cooperation § Reducing Nonresponse in Mall Surveys □ Keep the screener short, concise, and focused only on characteristics of primary importance □ Minimize the skip patterns on the screener and make it easy for the recruiter to distinguished qualified from nonqualified respondents It is important to make certain that interviewers are: § Successfully training to establish quick rapport with respondents § Carefully monitored with regard to th i i di id l f l t §

○

○

○

○

their individual refusal rates Nonresponse error due to respondent unavailability can be reduced by recontact § Attempt recontact between three and six times § Vary the time of day during which calls are made Reducing Nonresponse in Mail Surveys § Causes of nonresponse in mail surveys are different than phone and personal § Main problem is that recipients don't take time to complete and return survey § Advance notification is effective for inducing respondent cooperation and study participation § Techniques to increase response rate: □ Use a cover letter that clearly states the interview topic, describes why the topic is important and relevant to respondent, effectively and accurately describes the purpose and goal of the research, and explains the importance of the individuals responses □ Include a postage paid return envelope □ Prepare and print the questionnaire to maximize visual simplicity and professionalism □ Enclose monetary incentives with the questionnaire § Recontact after surveys have been mailed is an effective means of increasing response rate also

○

increasing response rate also Reducing Nonresponse in Online Surveys § A well-written, motivating invitation tends to encourage clicks and participation § Incentives tend to raise participation § The use of personal salutations in the e-mail invite also tend to increase participation § Also suggested □ Make sure the survey link is clickable □ Reassure people about their privacy and confidentiality □ Take care with the "from", "to", and "subject" □ Keep it short...