This here shall be a summary of the most common significance tests.

## Designing NHST

- specify and
- choose test statistic of which we know the null distribution and alternative distribution
- specify rejection region, significance level and decide if the rejection region is one or two-sided
- compute power using the alternative distribution(s)

## Running a NHST

- collect data and compute test statistic X
- Check if X is in rejection region (p<)

## Common significance tests

**z-test**

- Use: Compare data mean to the mean of our hypothesis
- Data: x_{ 1 },…,x_{ n }
- Assumptions: where the mean is unknown but the variance is known
- : for a specified value .
- :
- test statistic:
- null distribution: is pdf of Z~N(0,1)
- p-value: The p-value is calculated like always

**One-Sample t-Test for the mean**

- Use: Compare the data mean to the mean of our hypothesis
- Data:
- Assumptions: where both the mean and the variance is unknown.
- : for a specific value
- :
- test statistic: where
- null distribution: is the pdf of T~t(n-1)
- p-values: the p-value is calculated like always

**Two-Sample t-Test for comparing means (assuming equal variance)**

- Use: Compare the sample means of two groups
- Data: and
- Assumptions: and . The means and the variance is unknown but the variance of the two groups is the same.
- :
- :
- test statistic: where
- null distribution: is the pdf of T~t(n+m-2)
- p-values: the p-value is calculated like always

**One-way ANOVA (F-test for equal means)**

- Use: Compare the means of n groups each with m data points
- Data:
- …

- Assumptions: all groups follow a normal distribution with and where the means are possibly different but the variance is the same for all groups.
- : all means are the same.
- : Not all means are the same.
- test statistic: where:
- is the group mean;
- is the mean of all groups;
- is the sample variance of the group
- is the variance between the group means;
- is the sample mean of all the sample variances;

- null distribution: The null distribution is the pdf of W~F(n-1, n(m-1)) with n-1 and n(m-1) degrees of freedom.
- p-values: the p-value is calculated like always.

**Chi-Square test for goodness of fit**

- Use: Test if discrete data fits a finite probability mass function.
- Data: for each outcome an observed outcome
- Assumptions: none
- : The data was drawn from a specific discrete distribution
- : The data was drawn from a different distribution
- test statistic:
- Likelihood ratio statistic:
- Pearson’s Chi-Square statistic:

- null distribution: and have approximately the same pdf as where the degrees of freedom are the number of outcomes minus the number of parameters we have to calculate. That can be the mean, or any other thing.
- p-value: the p-value is calculated like always.

Advertisements