CS3009 Exam Model Answer 2017 PDF

Preview

Summary

Download CS3009 Exam Model Answer 2017 PDF

Description

CS3009 Model Answers 2017/18

PART I 1. As a case study example you were given two academic papers to study: (1) Randrambelonoro et al’s (2015) paper on activity monitoring for diabetic and obese patients; and (2) Koskimaki et al’s (2017) paper on a game and wearable sensors to promote activity. Both papers address a similar problem but use different methodologies. Please answer the following questions in relation to these two papers: (a) For both cases, identify at least two types of beneficiary group and describe the ways in which they might benefit from the research findings. End users / patients – health benefits Society / healthcare providers – reduced costs, reduced tax burden System designers – improved system designs – more sales, more satisfied customers [pass: identify something along these lines for two of the above] (b) Identify two types of research contribution provided by Koskimaki et al’s (2017) research (try to use the HCI contribution types defined by Wobbrock and Kientz (2016) to guide you if possible). Artefact contribution – creation of the game design Empirical contribution / contribution to knowledge – new knowledge about features that attract users [pass: identify something along these lines] (c) Identify the similarities and differences in the research approach / methods used in the two papers. Give one example where you consider a good research design choice has been made in one or both papers. Given one example where you consider a poor research design choice has been made. Justify your answer. Similarities – both papers have an intervention where users are given the devices to use followed by an evaluation at the end. Both papers are field / in situ studies where users use the device within their normal day-to-day activities. Both studies are longitudinal but neither explores very long term use. Both rely on subjective measures for success. [pass: identify at least one of these] Differences – different user groups (patients vs young men), types of measure (qualitative vs quantitative) [pass: identify at least one of these / something along these lines] Example of a good choice might be the ecological validity of users using the devices in their day to day lives, the longitudinal design. Example of a poor choice might be the use of only male participants in the Koskimaki paper (limited generalizability / not justified in paper), the choice of only subjective outcome measures, the lack of control conditions. [pass: at least one valid example of good and bad] Marking process: If a candidate passes all elements as described above they will get at least a C and a provisional grade in the range C to C+ will be noted (depending on the quality of answers). Part answers (found anywhere in the paper) which are relevant to module learning outcomes will be recognised as follows: • No content relevant to module learning outcomes – module grade stays at D- based on coursework • Content relevant to one module learning outcome – D • Content relevant to two module learning outcomes – D+ • Content relevant to all three module learning outcomes – C- (and considered with other part I passes) All candidates with a C- to C+ will be further examined on part II to see if there is evidence to move the grade up to the B category.

PART II 2. (i) Give two examples of qualities that have been associated with methodological rigour in qualitative research. To what extent are these features evident in the paper by Randriambelonoro et al (2017)? (justify your answer with examples from the paper where possible). Could mention any two of the following from those referenced in the reading. Examples include: Lincoln and Guba (1985): • Credibility - confidence in the 'truth' of the findings • Transferability - showing that the findings have applicability in other contexts • Dependability - showing that the findings are consistent and could be repeated • Confirmability - a degree of neutraility or the extent to which the findings of a study are shaped by the respondents and not researcher bias, motivation, or interest. Or Miles and Huberman (1994): Objectivity/Confirmability - relative neutrality, freedom from unacknowledged researcher bias, explicitness about inevitable bias Reliability/Dependability/Auditability is the process of the study consistent and reasonably stable over time and across researchers and methods? (Miles and Huberman, 1994, p. 278): Internal Validity/Credibility/Authenticity - Truth value. Do the findings of the study make sense? Are they credible to the people studied, members of the research community, and others? External Validity/Transferability/Fittingness - Do the conclusions of a study have any larger import? Are they transferable to other contexts? Do they fit with what we already know? How far can findings be generalized? Utilization/Application/Action Orientation - What does the study do for participants? What is the pragmatic value of the research? Possible issues that could be picked up in relation to the paper (but will accept any valid points): External validity is strengthened in the paper by use of previous research to frame the questions that participants were asked. Paper is weak in describing how neutrality was ensured (e.g. no mention of double coding or checking of coding of responses). (ii) What outcomes are measured in the paper by Koskimaki et al (2017)? To what extent are the outcome measures used in the paper valid and reliable indicators of the constructs of interest in this research project? Justify your answer. Subjective ratings were taken of usability, attractiveness and overall rating of the fitness tracker; subjective ratings were taken of six game features; subjective ratings were taken of the extent to which four features acted as motivated (only for those participants who reported an increase in activity levels). It is implicit that subjective measures of increase in activity were taken, but this is not reported directly and it is unclear what measure was used for this. Overall the evidence for validity and reliability is low. Only subjective data is reported so there is potential for bias. Unclear how activity increase was measured and if subjective data only was used this is especially prone to possible bias given difficult in estimating / recalling activity over a three month period (objective evidence of this would have been available from the trackers so unclear why this wasn’t used). Lack of detail on question wording but appears that constructs may have been measured with a single question and therefore lack reliability. Unclear whether any validated scales were used for any constructs. Authors aren’t very clear on what they wanted to measure and why in this paper, but the key aim of the game was to increase activity and the method of measuring this was poor/unclear therefore the validity of the study is low. (iii) Explain why you might be cautious about concluding that fitness tracking is effective from the evidence provided in the two case study paper? There are multiple limitations that could be mentioned including:

• • • • •

sample size / sampling strategy - limited generalisability limited time frame – may be short term benefits that fail to be maintained over longer period potential for bias, particularly Hawthorn effects lack of objective measures of activity outcomes without a control condition causal effects can’t be inferred

Marking process Candidates passing some elements of part II will get credit as follows: • one question passed – B• two questions passed – B • three questions passed – B+ Additionally candidates with part answers which don’t fully pass any individual question, but that nevertheless demonstrate understanding of how to evaluate the quality of research work, will be awarded a BOnly candidates with B+ after part II (indicating that they have passed all three sub-parts) will be eligible to marked for an A grade. PART III 3. Imagine the scenario where you have been asked to design a controlled experiment to evaluate the possible benefits of a fitness tracking game designed for university students. (a) Define the independent variable(s) of your study and resulting experimental conditions. Explain and justify your choice. Simplest example would be presence of tracking game as the IV with two conditions (e.g. no tracking game, tracking game, though there may be some discussion about what an appropriate control would be) – answer should illustrate that respondent understands what an experimental condition is and some of the issues involved in the choice. (b) State whether your experimental design will be a between or within-subjects design (or mixed design) and briefly outline the procedure you will follow. Justify your answer. Longitudinal field study with between-subjects design may be chosen to maximise ecological validity while remaining relatively practicable. Alternatives will also be accepted if appropriately justified. (c) Briefly explain how you would decide on sampling strategy and sample size. Strong answers will likely discuss benefits of random sampling vs alternatives. For sample size may discuss effect sizes. Should use a student population given the aims of the study. Should aim for a spread of user characteristics to achieve generalizability. (d) Define the dependent variables of your study and how these would be operationalised in practice. Explain your choice. Answer should make it clear that the candidate understands what a DV is and some characteristics of a good measure. Example DV might be average level of daily physical activity. Example operationalization might be average daily step count. Example explanation is that the main aim of the study is to evaluate possible benefits and a major intended benefit is likely to be an increase in physical activity. An objective measure is chosen as more reliable than subjective recall. (e) What type of analysis would you expect to conduct on the data and what, if anything, might this be able to prove from this? Should mention the use of statistical tests to test for significant differences in outcome measures between experimental conditions. Strong answers may name appropriate test types (e.g. t test, ANOVA). Discussion may frame results in terms of null and alternative hypothesis, discuss relevance of significance values, discuss interaction effects, discuss causal inference.

(f) What psychological / behavioural theories might be relevant to the design of an effective fitness tracking game and why? Could mention: self-determination theory (Deci and Ryan), TAM, Flow, extrinsic and intrinsic motivators, social influence, individual differences; should include (for example) design features that draw on these theories Marking process An A- will be awarded for an answer that demonstrates applied understanding of key terms in experimental design with no major errors. An A must answer most parts well. An A+ must answer all parts well, or most parts well with additional demonstration of exceptional insight in design choices. An A* must answer all parts well and show exceptional insight in choice of design....