7 Measurement AND DATA Collection PDF

Preview

Summary

Download 7 Measurement AND DATA Collection PDF

Description

MCH 617: Research Methodology and Scientific Writing Course Instructor: Fomboh Richard, PhD in Cellular and Molecular Parasitology MEASUREMENT AND DATA COLLECTION Without high-quality data collection methods, the accuracy and robustness of the conclusions are subject to challenge. As in the case of research designs and sampling plans, researchers must often choose from an array of alternative data collection methods. This chapter provides an overview of various methods of data collection for qualitative and quantitative studies, and discusses the development of a data collection plan. Existing data versus original data One of the first decisions that investigators make with regard to research data concerns whether to use existing data or to collect data generated specifically for the study. Most researchers develop original data, but they often can take advantage of existing information. Existing records are an important data source for nurse researchers. A wealth of data gathered for non-research purposes can be fruitfully exploited to answer research questions. Hospital records, patient charts, physicians’ order sheets, care plan statements, and the like all constitute rich data sources to which nurse researchers may have access. Dimensions of data collection approaches If existing data are not available for the research question, researchers must collect original data. Many methods of collecting new data are used for nursing studies. For example, study participants can be interviewed, observed, or tested with measures of physiologic functioning. Regardless of what specific approach is used, data collection methods vary along four important dimensions: structure, quantifiability, researcher obtrusiveness, and objectivity. Structure Research data for quantitative studies are often collected according to a structured plan that indicates what information is to be gathered and how to gather it. For example, most self-administered questionnaires are highly structured: They include a fixed set of questions to be answered in a specified sequence and with predesigned response options (e.g., agree or disagree). Structured methods give participants limited opportunities to qualify their answers or to explain the underlying meaning of their responses. In some studies it is more appropriate to impose little or no structure and to provide participants with opportunities to reveal information in a naturalistic way. Most qualitative studies rely almost exclusively on unstructured or loosely structured methods of data collection. There are advantages and disadvantages to both approaches. Structured methods often take considerable effort to develop and refine, but they yield data that are relatively easy to analyse. Structured methods are seldom appropriate for an in-depth examination of a phenomenon, however. Consider the following two methods of asking people about their levels of stress: Structured: During the past week, would you say you felt stressed? 1. Rarely or none of the time, 2. Some or a little of the time, 3. Occasionally or a moderate amount of the time, or 1

4. Most or all of the time? Unstructured: How stressed or anxious have you been this past week? Tell me about the kinds of tensions and stresses you have been experiencing. The structured approach would allow the researcher to compute exactly what percentage of respondents felt stressed most of the time but would provide no information about the intensity, cause, or circumstances of the stress. The unstructured question allows for deeper and more thoughtful responses, but may pose difficulties for people who are not good at expressing themselves verbally. Moreover, the unstructured question yields data that are considerably more difficult to analyse. When data are collected in a highly structured fashion, the researcher must develop (or borrow) what is referred to as the data collection instrument, which is the formal written document used to collect and record information, such as a questionnaire. When unstructured methods are used, there is typically no formal instrument, although there may be a list of the types of information needed. Quantifiability Data that will be subjected to statistical analysis must be gathered in such a way that they can be quantified. For statistical analysis, all variables must be quantitatively measured—even though the variables are abstract and intangible phenomena that represent qualities of humans, such as hope, loneliness, pain, and body image. Data that are to be analysed qualitatively are typically collected in narrative form. Structured data collection approaches usually yield data that are easily quantified. It may be possible (and it is sometimes useful), however, to quantify unstructured information as well. For example, responses to the unstructured question concerning stress could be categorized after the fact according to the four levels of stress indicated in the structured question. Whether it is wise to do so depends on the research problem, the researcher’s philosophic orientation, and the nature of the responses. Researcher Obtrusiveness Data collection methods differ in the degree to which people are aware of their status as participants. If people are aware of their role in a study, their behaviour and responses may not be “normal,” and distortions can undermine the value of the research. Study participants are most likely to distort their behaviour and their responses to questions under certain circumstances. In particular, researcher obtrusiveness is likely to be most problematic when: (1) A program is being evaluated and participants have a vested interest in the evaluation outcome; (2) Participants are engaged in socially unacceptable or atypical behaviour; (3) Participants have not complied with medical and nursing instructions; and (4) Participants are the type of people who have a strong need to “look good.” When researcher obtrusiveness is unavoidable under these circumstances, researchers should make an effort to put participants at ease, to stress the importance of candour and naturalistic behaviour, and to train research personnel to convey a neutral and non-judgmental demeanour. Objectivity Objectivity refers to the degree to which two independent researchers can arrive at similar “scores” or make similar observations regarding the concepts of interest, that is, make judgments regarding participants’ attributes 2

or behaviour that are not biased by personal feelings or beliefs. Some data collection approaches require more subjective judgment than others, and some research problems require a higher degree of objectivity than others.

MAJOR TYPES OF DATA COLLECTION METHODS In addition to making decisions regarding these four dimensions, researchers must select the form of data collection to use. Three types of approach have been used most frequently by nurse researchers: self-reports observation, and biophysiologic measures. Researchers’ decisions about research design usually are independent of decisions about data collection methods. Researchers using an experimental crossover design can rely on self-report data—as can a researcher doing ethnography, for example. Moreover, the three main data collection methods—self-report, observation, and biophysiologic measures—can involve either existing data or original data created for research purposes. Self-Reports A good deal of information can be gathered by questioning people, a method known as self-report. If, for example, we were interested in learning about patients’ perceptions of hospital care, their preoperative fears, or their health-promoting habits, we would likely talk to them and ask them questions. The unique ability of humans to communicate verbally on a sophisticated level makes direct questioning a particularly important part of nurse researchers’ data collection repertoire. The vast majority of nursing studies involve data collected by self-report. The self-report method is strong in directness and versatility. If we want to know what people think, feel, or believe, the most efficient means of gathering information is to ask them about it. Perhaps the strongest argument that can be made for the self-report method is that it frequently yields information that would be difficult, if not impossible, to gather by any other means. Behaviours can be observed, but only if participants engage in them publicly. For example, it is usually impossible for researchers to observe such behaviours as child abuse, contraceptive practices, or “road rage.” Furthermore, observers can observe only those behaviours occurring at the time of the study. Through self-reports, researchers can gather retrospective data about activities and events occurring in the past or gather projections about behaviours in which people plan to engage in the future. Information about feelings, values, opinions, and motives can sometimes be inferred through observation, but behaviours and feelings do not always correspond exactly. People’s actions do not always indicate their state of mind. Here again, self-report methods can be used to capture psychological characteristics through direct communication with participants. Despite these advantages, verbal report methods have some weaknesses. The most serious issue concerns the validity and accuracy of self-reports: How can we really be sure that respondents feel or act the way they say they do? How can we trust the information that respondents provide, particularly if the true answers would reveal illegal or socially unacceptable behaviour? Investigators often have no alternative but to assume that their respondents have been frank. Yet we all have a tendency to want to present ourselves in the best light, and this 3

may conflict with the truth. Researchers who gather self-report data should recognize the limitations of this method, and should be prepared to take these shortcomings into consideration when interpreting the results. Observation For certain research problems, an alternative to self-reports is observation of people’s behaviour. Information required by nurse researchers as evidence of nursing effectiveness or as clues to improving nursing practices often can be obtained through direct observation. Suppose, for instance, that we were interested in studying mental patients’ methods of defending their personal territory, or children’s reactions to the removal of a leg cast, or a patient’s mode of emergence from anesthesia. These phenomena are all amenable to observation. Observational methods can be used to gather a variety of information, including information on characteristics and conditions of individuals (e.g., the sleep—wake state of patients); verbal communication (e.g., exchange of information during medication administration); nonverbal communication (e.g., facial expressions); activities (e.g., geriatric patients’ self-grooming activities); and environmental conditions (e.g., architectural barriers in the homes of disabled people). Like self-report techniques, observational methods can vary in degree of structure. That is, a researcher could observe nurses’ methods of touching patients in an unstructured manner, taking detailed narrative notes regarding their use of touch. Alternatively, the researcher could tabulate the frequency of the nurses’ use of specific types of touching, according to a pre-designated classification system. Observational research is particularly well suited to nursing. Nurses are in an advantageous position to observe, relatively unobtrusively, the behaviours and activities of patients, their families, and hospital staff. Moreover, nurses may, by training, be especially sensitive observers. Many nursing problems are better suited to an observational approach than to self-report techniques. Whenever people cannot be expected to describe adequately their own behaviours, observational methods may be needed. This may be the case when people are unaware of their own behaviour (e.g., manifesting preoperative symptoms of anxiety), when people are embarrassed to report their activities (e.g., displays of aggression or hostility), when behaviours are emotionally laden (e.g., grieving behaviour among widows), or when people are not capable of articulating their actions (e.g., young children or the mentally ill). Observation is intrinsically appealing in its ability to capture directly a record of behaviours and events. Several of the shortcomings of the observational approach include possible ethical difficulties, distorted behaviours on the part of the person being observed when the observer is conspicuous, and a high rate of refusal to cooperate. Another pervasive problem is the vulnerability of observational data to observer biases. A number of factors interfere with objective observations, including the following: • Emotions, prejudices, attitudes, and values of observers may result in faulty inference. • Personal interest and commitment may colour what is seen in the direction of what observers want to see. • Anticipation of what is to be observed may affect what is observed. • Hasty decisions before adequate information is collected may result in erroneous classifications or conclusions. Biophysiologic Measures The trend in nursing research has been toward increased clinical, patient-centred investigations. One result of this trend is an expanded use of measures to assess the physiologic status of subjects— typically through quantitative biophysiologic measures. Physiologic and physical variables typically require specialized technical instruments and equipment for their measurement and, usually, specialized training for the interpretation of 4

results. Settings in which nurses operate are usually filled with a wide variety of technical instruments for measuring physiologic functions. In comparison with other types of data collection tools, the equipment for obtaining physiologic measurements is costly. Because such equipment is generally available in health care settings, however, the costs to nurse researchers may be small or non-existent. A major strength of biophysiologic measures is their objectivity. Nurse A and nurse B, reading from the same spirometer output, are likely to record the same tidal volume measurements. Furthermore, barring the possibility of equipment malfunctioning, two different spirometers are likely to produce identical tidal volume readouts. Developing a data collection plan in a quantitative study Data collection plans for quantitative studies should ideally yield accurate, valid, and meaningful data that are maximally effective in answering research questions. These are rigorous requirements, typically requiring considerable time and effort to achieve. This section discusses steps that are often undertaken in the development of a data collection plan in quantitative studies. Identifying Data Needs Researchers must usually begin by identifying the types of data needed to complete the study successfully. Typically, researchers gather information about more than just the main study variables. Advance planning may help to avoid “if only” disappointments at the analysis stage. In a quantitative study, researchers may need to identify data requirements for accomplishing the following: 1. Testing the hypotheses or addressing the research questions. Researchers must include one or more measures of all the independent and dependent variables. Multiple measures of some variables may be required if a variable is complex and multifaceted or if there are concerns about the accuracy of a single measure. 2. Describing sample characteristics. Information should be gathered about major demographic and health characteristics of the sample. We advise gathering data about participants’ age, gender, race or ethnicity, educational background, marital status, and income level. This information is critical in interpreting results and understanding the population to whom the findings can be generalized. If the sample includes participants with a health problem, data on the nature of that problem also should be gathered (e.g., length of illness, severity of health problem, types of treatment obtained, length of stay in hospital). 3. Controlling extraneous variables. Various approaches can be used to control extraneous variables, and many of them require the measurement of those variables. For example, when analysis of covariance is used, each variable that is statistically controlled must be measured. 4. Analysing potential biases. Ideally, data that can help the researcher to identify potential biases should be collected. For example, researchers should gather information that would help to identify selection biases in a non-equivalent control group design. Researchers should give some thought to potential biases and then determine how they could be assessed. 5. Understanding subgroup effects. It is often desirable to answer the research questions not only for the entire sample but also for certain subgroups of participants. For example, we may wish to know if a special intervention for indigent pregnant women is equally effective for primiparas and multiparas. In such a situation, we would need to collect information about the participants’ childbearing history so that we could divide the sample and analyse for separate subgroup effects. 6. Interpreting results. Researchers should try to anticipate alternative results, and then determine what types of data would best help in interpreting them. For example, if we hypothesized that the presence of school-based clinics in high schools would lower the incidence of sexually transmitted diseases among students but found that the incidence remained constant after the clinic was established, what type of information would we want to 5

help us interpret this result (information about the students’ frequency of intercourse, number of partners, use of condoms, and so on)? 7. Checking the manipulation. When researchers manipulate the independent variable, it is sometimes useful to determine if the manipulation actually occurred. Such a manipulation check can help in interpreting negative results. 8. Obtaining administrative information. It is almost always necessary to gather administrative information to help in project management. For example, if there are multiple data collectors, identification numbers for each one should be recorded. Other administrative information might include participant identification numbers, dates of attempted contact with participants, dates of actual data collection, where data collection occurred, when the data collection session began and ended reasons that a potential subject did not participate in the study, and contact information if the study is longitudinal. Selecting Types of Measures After data needs have been identified, the next step is to select a data collection method for each variable. It is common, and often advantageous, to combine self-reports, observations, or physiologic measures in a single study. Researchers also combine measures that vary in terms of the four basic dimensions. In reviewing data needs, researchers should determine how best to capture each variable in terms of its conceptual or theoretical definition. Research considerations are not the only factors that drive decisions about methods to use in collecting data. The decisions must also be guided by ethical considerations, cost constraints, the availability of appropriate staff to help with data collection, time pressures, and the anticipated burden to participants and others, such as hospital staff or participants’ families. Data collection is typically the costliest and most timeconsuming portion of a study. Because of this, researchers often have to make a number of compromises about the type or amount of data collected. Selecting and Developing Instruments Once preliminary decisions have been made regarding the basic data collection methods to be used, researchers should determine if there are instruments available for constructs of interest. For most constructs, existing instruments are available and should be considered. After potential data collection instruments have been identified, they should be assessed to determine their appropriateness. The primary consideration is whether the instrument is conceptually relevant:...