Statistical Tests

Describe the appropriate selection of non-parametric and parametric tests and tests that examine relationships (e.g. correlation, regression)

Parametric Tests

Parametric tests are used when data is:

• Continuous and numerical
• Normally distributed
  • Remember that due to the central limit theorem - large data sets (n > 100) are typically amenable to parametric analysis, as sample means will follow a normal distribution
  • Non-normal data can be transformed so that they follow a normal distribution
• Samples are taken randomly
• Samples have the same variance
• Observations within the group are independent
  Independent results are those when one value is not expected to influence another value.
  • A common example is repeated measures: when serial measures are taken from a patient or a hospital, the results cannot be treated as independent
  • Paired tests are used when two dependent samples are compared
  • Unpaired test are used when two independent samples are compared

Tests may be one-tailed or two-tailed:

• A two-tailed test evaluates whether the sample mean is significantly greater or less than the population mean
• A one-tailed test only evaluates the relationship in one direction
  This doubles the power of the test to detect a difference, but should only be performed if there is a very good reason that the effect could only occur in one direction.

Common parametric tests include:

Z test

Used to test whether the mean of a particular sample (x̄) differs from the population mean (μ) by random variation.

Assumptions:

• Large sample
  n > 100.
• Data is normally distributed
• Population standard deviation is known

Student's T Test

This is a variant of the Z test, used when the population standard deviation is not known.

• The results from T test approximate the results of the Z test when n > 100

F Test

Compares the ratio of variances ($Var_1 \over Var_2$) for two samples. If F deviates significantly from 1, then there is a significant difference in group variances.

Analysis of Variance (ANOVA)

ANOVA tests for significant differences between means of multiple groups, in a more efficient manner than multiple comparisons (doing lots of T tests).

There are several types of ANOVA tests used in different situations.

Non-Parametric Tests

Non-parametric tests are used when the assumptions for parametric tests are not met. Non-parametric tests:

• Do not assume the data follows any particular distribution
  This is required when:
  • Non-normality is obvious
    e.g. Multiple observations of 0
  • Possible non-normality
    Typically small sample sizes.
  • Data is ordinal
• Are not as powerful as parametric tests (a larger sample size is required to achieve the same error rate)
• Are more broadly applicable than parametric tests as they do not require the same assumptions

Non-parametric tests still require that data:

• Is continuous or ordinal
• Within-group observations are independent
• Samples are taken randomly

In general, non-parametric tests;

• Take each result and rank them
• Calculations are then performed on each rank to find the test statistic

Common non-parametric tests include:

Mann-Whitney U Test/Wilcoxon Rank Sum Test

Alternative to the unpaired T-test for non-parametric data.

Process:

• Data from both groups are combined, ordered, and given ranks
  • Tied data are given identical ranks, where that rank is equal to the average rank of the tied observations
• The data are then separated into their original group
• Ranks in each group are added to give a test statistic for each group
• A statistical test is performed to see if the sum of ranks in one group is different to another

Wilcoxon Signed Ranks Test

Alternative to the paired T-test for non-parametric data.

Process:

• As above (for the Wilcoxon Rank Sum Test), except absolute difference between paired observations are ranked
  The sign (i.e. positive or negative) is preserved.
• The sum of positive ranks is then compared with the sum of negative ranks
• If there is no difference between groups, we would expect the net value to be 0

References

1. Myles PS, Gin T. Statistical methods for anaesthesia and intensive care. 1st ed. Oxford: Butterworth-Heinemann, 2001.