Steps in Quantitative Analysis PDF

Preview

Summary

Download Steps in Quantitative Analysis PDF

Description

Steps in Quantitative Analysis Stepping Your Way through Effective Quantitative Data Analysis

1. Data management – This involves familiarizing yourself with appropriate software; systematically logging in and screening your data: entering the data into a program; and finally, ‘cleaning’ your data. 2. Understanding variable types – Different data types demand discrete treatment, so it has important to be able to distinguish variables by both cause and effect (dependent or independent), and their measurement scales (nominal, ordinal, interval, and ratio). 3. Run descriptive statistics – These are used to summarize the basic features of a data set through measures of central tendency (mean, mode, and median), dispersion (range, quartiles, variance, and standard deviation), and distribution (skewness and kurtosis). 4. Run appropriate inferential statistics – This allows researchers to assess their ability to draw conclusions that extend beyond the immediate data. For example, if a sample represents the population; if there are differences between two or more groups; if there are changes over time; or if there is a relationship between two or more variables. 5. Make sure you selecting the right statistical test – This relies on knowing the nature of your variables; their scale of measurement; their distribution shape; and the types of question you want to ask. 6. Look for statistical significance – This is generally captured through a ‘p-value’, which assesses the probability that your findings are more than coincidence. The lower the p-value, the more confident researchers can be that findings are genuine.

Steps in Qualitative Analysis Stepping Your Way through Effective Qualitative Data Analysis

1. Identifying biases/ noting overall impressions – Doing this fully is extremely important. If you do not acknowledge preconceived notions and actively work to neutralize them, you are likely to find exactly what you expect to find! 2. Reducing and coding into themes your data – This involves building both categories and subcategories that are likely to expand as you work your way through your data. 3. Searching for patterns and interconnections – You are likely to have overlapping themes across your data sources – so this step asks you to search for commonalities and divergences. 4. Mapping and building themes – One small section of a preliminary map might be as follows (take music video analysis as an example). From here you would a) continue mapping all the main themes b) create even more sub categories as appropriate; and c) map various interconnections. Don’t forget to call on the literature in doing these tasks.

5. Building and verifying theories – this is your ‘hey you know what might be going on here’ moment that will hopeful dawn on you as you go over your data for the 103rd time and play around with your maps for the 72nd time. 6. Drawing conclusions – you are likely to find out much more through the processes than you could possible share, so you will need to decide what is most significant/ important and link this back to your project’s main questions, aims and objectives in the most compelling and credible way.

Steps in Systematic Data Analysis Stepping Your Way through Effective Systematic Data Analysis

1. Formulate the research question – Like any research process, a clear, unambiguous research question will help set the direction for your study, i.e. what type of health promotions campaigns have been most effective in reducing smoking rates of Australian teenagers or Does school leadership makes a difference to educational standards? 2. Develop and use an explicit, reproducible methodology – Key to systematic reviews are that bias is minimized and that methods are transparent and reproducible. 3. Develop and use clear inclusion/ exclusion criteria – The array of literature out there is vast. Determining clear selection criteria for inclusion is essential. 4. Develop and use an explicit search strategy – It is important to identify all studies that meet the eligibility criteria set in #3. The search for studies need to be extensive should be extensive and draw on multiple databases. 5. Critically assess the validity of the findings in included studies – This is likely to involve critical appraisal guides and quality checklists that cover participant recruitment, data collection methods, and modes of analysis. Assessment is often conducted by two or more reviewers who know both the topic area and commonly used methods. 6. Analysis of findings across the studies – This can involve analysis, comparison, and synthesis of results using methodological criteria. This is often the case for qualitative studies. Quantitative studies generally attempt to use statistical methods to explore differences between studies and combine their effects (see meta analysis below). If divergences are found, the source of the divergence is analysed. 7. Synthesis and interpretation of results – synthesized results need to be interpreted in light of both the limitations of the review and the studies it contains. An example here might be the inclusion of only studies reported in English. This level of transparency allows readers to assess the review credibility and applicability of findings.

Choosing Between a Nonparametric Test and a Parametric Test I t ’ ssaf et osayt hatmostpeopl ewhousest at i st i csar emor ef ami l i arwi t hpar amet r i canal ysest han nonpar amet r i canal yses.Nonpar amet r i ct est sar eal socal l eddi st r i but i onf r eet est sbecauset hey don’ tassumet hatyourdat af ol l owaspeci ficdi st r i but i on.

Youmayhavehear dt haty oushoul dusenonpar amet r i ct es t swheny ourdat adon’ tmeett he assumpt i onsoft hepar amet r i ct est ,es peci al l yt heassumpt i onaboutnor mal l ydi s t r i but eddat a.That soundsl i k eani c eands t r ai ght f or war dwayt oc hoose,butt her ear eaddi t i onal consi der at i ons . I nt hi spos t ,I ’ l lhel py oudet er mi newheny oushoul dusea:  

Par amet r i canal y s i st ot estgr oupmeans. Nonpar amet r i canal y si st ot es tgr oupmedi ans.

I npar t i cul ar ,I ' l l f ocusonani mpor t antr easont ousenonpar amet r i ct est st hatIdon’ tt hi nkget s ment i onedof t enenough!

Hypot hesi sTest soft heMeanandMedi an Nonpar amet r i ct est sar el i keapar al l eluni v er set opar amet r i ct est s .Thet abl eshowsr el at edpai r s hesi st es t st hatMi ni t abst at i s t i calsof t war eoffer s. ofhypot

Parametric tests (means)

Nonparametric tests (medians)

1-sample t test

1-sample Sign, 1-sample Wilcoxon

2-sample t test

Mann-Whitney test

One-Way ANOVA

Kruskal-Wallis, Mood’s median test

Factorial DOE with one factor and one blocking variable

Friedman test

Reasonst oUsePar amet r i cTest s Reason1:Par amet r i ct est scanper f or m wel lwi t hskewedandnonnor maldi st r i but i ons Thi smaybeasur pr i sebutpar amet r i ct est sc anper f or m wel l wi t hcont i nuousdat at hatar e nonnor mali fy ous at i s f yt hesampl esi z egui del i nesi nt het abl ebel ow.Thesegui del i nesar ebased onsi mul at i onst udi esconduct edbys t at i s t i c i ansher eatMi ni t ab.Tol ear nmor eaboutt hesest udi es , r eadourTechni cal Paper s.

Parametric analyses

Sample size guidelines for nonnormal data

1-sample t test

Greater than 20

2-sample t test

Each group should be greater than 15

 One-Way ANOVA



If you have 2-9 groups, each group should be greater than 15. If you have 10-12 groups, each group should be greater than 20.

Reason2:Par amet r i ct est scanper f or m wel lwhent hespr eadofeachgr oupi sdi ffer ent Whi l enonpar amet r i ct es t sdon’ tassumet haty ourdat af ol l owanor maldi s t r i but i on,t heydohav e ot herassumpt i onst hatcanbehar dt omeet .Fornonpar amet r i ct est st hatcompar egr oups ,a commonassumpt i oni st hatt hedat af oral l gr oupsmusthav et hesamespr ead( di s per si on) .I fy our gr oupshav eadi ffer entspr ead,t henonpar amet r i ct est smi ghtnotpr ovi dev al i dr esul t s . Ont heot herhand,i fy ouuset he2s ampl ett estorOneWayANOVA,y ouc ansi mpl ygot o t heOpt i onssubdi al oganduncheckAs sumeequalv ar i ances.Voi l à,y ou’ r egoodt ogoev enwhen t hegr oupshav edi ffer entspr eads ! Reason3:St at i st i calpower Par amet r i ct es t susual l yhavemor est at i s t i calpowert hannonpar amet r i ct est s.Thus ,y ouar emor e l i k el yt odet ectasi gni fi canteffectwhenonet r ul yexi s t s .

Reasonst oUseNonpar amet r i cTest s Reason1:Yourar eaofst udyi sbet t err epr esent edbyt hemedi an

Thi si smyf avor i t er easont ousea nonpar amet r i ct estandt heonet hati sn’ tment i onedof t enenough!Thef actt haty oucanper f or ma par amet r i ct estwi t hnonnor maldat adoesn’ ti mpl yt hatt hemeani st hebestmeasur eoft hecent r al t endenc yf ory ourdat a. Forex ampl e,t hecent erofask eweddi st r i but i on,l i k ei ncome,canbebet t ermeasur edbyt hemedi an wher e50% ar eabov et hemedi anand50% ar ebel ow.I fy ouaddaf ewbi l l i onai r est oasampl e,t he mat hemat i cal meani ncr easesgr eat l yev ent hought hei ncomef ort het y pi calper sondoesn’ tchange. Wheny ourdi s t r i but i oni ssk ewedenough,t hemeani ss t r ongl yaffect edbychangesf arouti nt he di s t r i but i on’ st ai l wher east hemedi ancont i nuest omor ecl osel yr eflectt hecent eroft hedi s t r i but i on. Fort heset wodi st r i but i ons ,ar andom sampl eof100f r om eachdi s t r i but i onpr oducesmeanst hatar e si gni ficant l ydi ffer ent ,butmedi anst hatar enotsi gni ficant l ydi ffer ent . Twoot herbl ogpost si l l us t r at et hi spoi ntwel l :  

Usi ngt heMeani nDat aAnal y si s :I t ’ sNotAl way saSl amDunk TheNonpar amet r i cEc onomy :WhatDoesAv er ageAc t ual l yMean?

Reason2:Youhaveaver ysmal lsampl esi z e I fy oudon’ tmeett hesampl esi z egui del i nesf ort hepar amet r i ct est sandy ouar enotconfidentt hat y ouhav enor mal l ydi s t r i but eddat a,y oushoul duseanonpar amet r i ct est .Wheny ouhav ear eal l y smal ls ampl e,y oumi ghtnotev enbeabl et oascer t ai nt hedi st r i but i onofy ourdat abecauset he di s t r i but i ont est swi l ll acksuffici entpowert opr ovi demeani ngf ulr esul t s . I nt hi sscenar i o,you’ r ei nat oughspotwi t hnoval i dal t er nat i v e.Nonpar amet r i ct est shav el esspower t obegi nwi t handi t ’ sadoubl ewhammywheny ouaddasmal ls ampl esi z eont opoft hat ! Reason3:Youhaveor di naldat a,r ankeddat a,orout l i er st hatyoucan’ tr emove

Typi c al par amet r i ct es t scanonl yassesscont i nuousdat aandt her esul t scanbes i gni fi cant l yaffect ed byout l i er s .Conv er sel y ,somenonpar amet r i ct es t scanhandl eor di naldat a,r ank eddat a,andnotbe ser i ousl yaffect edbyout l i er s .Besur et ocheckt heassumpt i onsf ort henonpar amet r i ct estbecause eachonehasi t sowndat ar equi r ement s . I fy ouhav eLi k er tdat aandwantt ocompar et wogr oups,r eadmypostBestWayt oAnal yz eLi k er t I t em Dat a:T woSampl eTT estv er susMannWhi t ney.

Cl osi ngThought s I t ’ scommonl yt houghtt hatt heneedt ochoosebet weenapar amet r i candnonpar amet r i ct estoccur s wheny ourdat af ai l t omeetanassumpt i onoft hepar amet r i ct es t .Thi scanbet hecasewheny ou hav ebot has mal ls ampl esi z eandnonnor maldat a.Howev er ,ot herconsi der at i onsof t enpl ayar ol e becausepar amet r i ct es t scanof t enhandl enonnor maldat a.Conv er sel y ,nonpar amet r i ct es t shav e s t r i ctassumpt i onst haty oucan’ tdi sr egar d. Thedeci si onof t endependsonwhet hert hemeanormedi anmor eaccur at el yr epr esent st hecent er ofy ourdat a’ sdi st r i but i on.  

I ft hemeanaccur at el yr epr esent st hecent erofy ourdi s t r i but i onandy oursampl esi z ei sl ar ge enough,consi derapar amet r i ct es tbecauset heyar emor epower f ul . I ft hemedi anbet t err epr esent st hecent erofy ourdi s t r i but i on,consi dert henonpar amet r i c t estev enwheny ouhav eal ar gesampl e.

Fi nal l y ,i fy ouhav eav er ysmal l sampl esi z e,y oumi ghtbest uckusi nganonpar amet r i ct est .Pl ease, col l ectmor edat anextt i mei fi ti satal lpossi bl e!Asy ouc ansee,t hesampl es i z egui del i nesar en’ t r eal l yt hatl ar ge.Yourchanceofdet ect i ngasi gni ficanteffectwhenoneexi s t scanbev er ysmal l wheny ouhavebot hasmal l sampl esi z eandy ouneedt ouseal esseffici entnonpar amet r i ct est ! The following table lists the nonparametric tests and their parametric alternatives.

N O N PA RA M E T R I C T E S T

P A R A M E T R I C A LT E R N AT I V E

1-sample sign test

One-sample Z-test, One sample t-test

1-sample Wilcoxon Signed Rank test

One sample Z-test, One sample t-test

Friedman test

Two-way ANOVA

Kruskal-Wallis test

One-way ANOVA

Mann-Whitney test

Independent samples t-test

Mood’s Median test

One-way ANOVA

Spearman Rank Correlation

Pearson’s Correlation Coefficient

Types of Nonparametric Tests

When the word “parametric” is used in stats, it usually means tests like ANOVA or a t test. Those tests both assume that the population data has a normal distribution. Non parametric do not assume that the data is normally distributed. The only non parametric test you are likely to come across in elementary stats is the chi-square test. However, there are several others. For example: the Kruskal Willis test is the non parametric alternative to the One way ANOVA and the Mann Whitney is the non parametric alternative to the two sample t test. The main nonparamteric tests are:  

      

1-sample sign test. Use this test to estimate the median of a population and compare it to a reference value or target value. 1-sample Wilcoxon signed rank test. With this test, you also estimate the population median and compare it to a reference/target value. However, the test assumes your data comes from a symmetric distribution (like the Cauchy distribution or uniform distribution). Friedman test. This test is used to test for differences between groups with ordinal dependent variables. It can also be used for continuous data if the one-way ANOVA with repeated measures is inappropriate (i.e. some assumption has been violated). Goodman Kruska’s Gamma: a test of association for ranked variables. Kruskal-Wallis test. Use this test instead of a one-way ANOVA to find out if two or more medians are different. Ranks of the data points are used for the calculations, rather than the data points themselves. The Mann-Kendall Trend Test looks for trends in time-series data. Mann-Whitney test. Use this test to compare differences between two independent groups when dependent variables are either ordinal or continuous. Mood’s Median test. Use this test instead of the sign test when you have two independent samples. Spearman Rank Correlation.Use when you want to find a correlation between two sets of data....