Lecture 2 PDF

Preview

Summary

lecture notes for lecture 2...

Description

Covariance and correlation - Correlation and covariance do not imply causality Ice cream sales and casualties on the beach on a hot day - Highly correlated - It is because of the hot weather! Not because of ice cream sold or casualties, there is no causation between the 2 Correlation Correlation is standardized covariance - The closer potential value to -1 or +1, the stronger the correlation between the variables If covariance or correlation is 0, there is no linear relation between the 2 variables X bar is the average X is the value Data mining - Discovering interesting and new patterns, correlations and anomalies using various data sets and methods - Ex. Various graphs, plots - Correlation matrix - Regression analysis Predict stock price (if correlation is positive) 1. Download monthly stock price information of all NYSE stocks 2. Compute correlation coefficients between the identified stock price for 31/07/201630/06/2018 and other stocks for 30/06/2016-31/05/2018 3. Identify stock with high absolute values of correlation coefficients 4. If correlation is positive and if stock price is higher on 30/06/2018 than on 31/05/2018, then stock price is predicted to increase in 31/07/2018 5. If correlation is positive and stock price is lower on 30/06/2018 than on 31/05/2018, then stock rice is predicted to decrease in 31/07/2018 Predict stock price (if correlation is negative) What is the point of ignoring the first month? Not getting why you need to use 2 different periods of time Visual analytics Visual presentation - Important in hypothesis development Focus attention where you want it Gnuplot - Can create figures with codes Text, colour, font sizes… Large numbers, use millions on dots Use of different colours Logarithmic scale - Each subsequent increment is 10x greater than the previous one...