**Significance Tests** (normal distribution assumed)

Groups | Requirements | Test | Tails | Statistic | Distribution | Degrees of freedom | Example |

1 | normal dist, n > 30 and known variance | z-test | 1 and 2 | z-score | Normal | NA | A sample of the population is tested for height. Do they match the known population? Population mean and standard deviation are known, sample > 30 |

1 | n < 30 and unknown variance | One-sample t-test | 1 and 2 | t-score | Students t | n-1 | A company wants to compare the average weekly sales of a new product to the historical average weekly sales of similar products, which is $5000. |

2 | Same 2 groups (before and after). 2 Dependent samples. | Paired t-test | 1 and 2 | t-score | Students t | n-1 | A company implements a new training program and wants to determine its effectiveness by comparing employee performance before and after the training. |

2* | 2 Independent groups, equal variance | Independent t-test (equal variance – pooled) | 1 or 2 | t-score | Students t | n1+n2−2 | A retailer wants to compare the average sales of two different stores located in different regions. |

2* | 2 Independent groups, unequal variance | Independent t-test (Unequal/Welch’s) | 1 and 2 | t-score | Students t | n1+n2−2 | |

3+ | 3+ groups – look at one variable | One-Way ANOVA | 1 Only | F-score | F | between groups k-1 (where k is num of groups). | Three groups are given different drugs in a test to see the improvement in blood sugar. |

3+ | 3+ groups – look at multiple variables | Two-way ANOVA | 1 Only | F-score | F | Effect of water and sun on height of sunflowers, where 3+ combinations of water and sun (3 + groups) | |

2 Categories | 2 Categories – comparison is to see if they are related | Pearsons Chi-Square | neither, just different | Chi-Squared | Chi-Squared | (no. of rows – 1) * (no. of columns – 1). | Individuals that received social media vs. those that received email – 2 Groups purchased and not purchased. |

Correlation | |||||||

2 | Correlation | Pearsons Correlation Coefficient (R) | 1 or 2 | t-statistic | t-distribution table | n−2 | Relationship between daily hours of sunlight and temperature. Best for continuous data with linear relationships. Provides both the direction (positive or negative) and the strength of a linear relationship between two variables. Note for simple linear regression. R squared is the Pearson r squared. It ranges between -1 and 1 |

2 | Correlation | Spearmans Rank | 1 or 2 | t-statistic? | Spearmans Rho? | n-2 | Relationship between the ranks of employees’ performance and their years of experience. Best for ordinal data or when data does not meet Pearson’s assumptions. |

r Squared (used in regression analysis) One (simple linear regression) or more independent variables (Multiple linear regression) + 1 dependent variable | Not used in significance testing directly | NA | Explaining the variance in academic performance based on study hours, attendance, and other factors. Used in the context of regression analysis to explain the proportion of variance in the dependent variable. Indicates the proportion of the variance in the dependent variable that can be explained by the independent variable(s). It does not provide information about the direction of the relationship. Ranges from 0 to 1 | ||||

Regression Analysis | F-Statistic | F-Distribution | Fit a regression model and computer squared. Perform an F-test to determine overall significance. | ||||

Non-Parametric Tests | Not Normal distribution | ||||||

2 | Compares distributions of the 2 groups | Mann Whitney U | U Statistic | Mann-Whitney U Distribution | NA | 2 Independent groups (like the t-test) | |

2 | Compares Medians of the 2 groups | Wilcoxen signed rank | T Statistic | Wilcoxen signed rank table | NA | When comparing two related samples, matched samples, or repeated measurements on a single sample. |

* Note: Uses Levenes test to determine equal/unequal variance

**Significant Testing Steps**

- Null hypothesis: There is no significant difference between variables between the two groups.

Examples:

1. In a drug efficacy study, the null hypothesis might state that the drug does not affect patients compared to a placebo.

2. The mean cholesterol levels of patients taking the new drug are equal to the mean cholesterol levels of patients taking a placebo.

3. The mean sales after the advertising campaign are equal to the mean sales before the campaign.

4. The mean sales of the product with the new packaging design are equal to the mean sales of the product with the old packaging design.

Alternative hypothesis: There is a significant difference between the means of the groups. - The significance level (𝛼
*α*) is the probability threshold for rejecting the null hypothesis. Commonly used significance levels include 0.05, 0.01, and 0.10. e.g. 5%: The result of the test only has a 5% chance of occurring. - Calculate test statistic (t-value, z-value, etc). The test statistic is used to look up the p-value from the relevant distribution tables.

4. Calculate the p-value using the test statistic. The p-value is used to determine if the test is significant. e.g. a p-value of less than 5% (0.05) is significant and we can reject the null hypothesis. If the p-value is higher than the significance level, then the results are reported as statistically significant.

1 or 2 Tailed Tests

One-tailed test if measuring greater or less than Left or Right Sided

Two-tailed if measuring the difference

Type 1 and Type 2 Errors

Type 1 Error: We reject the null hypothesis even though it was true (Alpha)

Type 2 Error: We accept the null hypothesis even though it was false (Beta)

Two Methods of accepting or rejecting Null Hypothesis

1. Critical Values (taken from normal or t-distribution)

2. P-values (if p < or > than 0.05)

## Leave a Reply