The Sign Test

Introduction to Statistical Testing and the Sign Test

Descriptive statistics summarise data; inferential statistics allow researchers to make probabilistic inferences about the population from which a sample was drawn. The fundamental question of inferential testing is whether an observed result — a difference between conditions or a relationship between variables — reflects a genuine population effect or could plausibly have arisen by chance.

The Logic of Significance Testing

Statistical hypothesis testing operates from the null hypothesis (H₀) — the assumption that there is no genuine effect in the population and that any observed result is due to chance sampling variation. A statistical test calculates the probability of obtaining the observed result (or a more extreme one) if the null hypothesis were true. This probability is the p-value.

If the p-value is sufficiently small — below a pre-specified significance level (α) — the null hypothesis is rejected and the result is declared statistically significant. The conventional significance level in psychology is p < 0.05: if the probability of the observed result under H₀ is less than 5%, H₀ is rejected.

A p-value of 0.05 means there is a 5% chance of obtaining results at least as extreme as those observed if H₀ is true — a 1-in-20 risk of a Type I error (false positive: rejecting H₀ when it is actually true). Stricter criteria (p < 0.01, p < 0.001) reduce this risk but increase the risk of a Type II error (false negative: failing to detect a genuine effect).

One-Tailed and Two-Tailed Tests

A directional (one-tailed) test is used when the hypothesis predicts the specific direction of an effect (e.g. 'Group A will score higher than Group B'). A non-directional (two-tailed) test is used when the hypothesis predicts a difference but not its direction. One-tailed tests are more powerful (more sensitive to detecting an effect in the predicted direction) but can only be used when the direction is genuinely predicted in advance.

The Sign Test

The sign test is the simplest non-parametric inferential test. It is used to determine whether there is a significant difference between two conditions in a repeated measures design when data are at least at the ordinal level (the researcher can judge which of two scores is higher for each participant).

Procedure:

1. For each participant, record which condition produced the higher score. Assign a '+' if Condition A is higher, a '−' if Condition B is higher. Participants with identical scores in both conditions (ties) are excluded.
2. Count the number of '+' and '−' signs. Let S = the smaller of these two counts.
3. Let N = total number of non-tied participants.
4. Compare S to the critical value from the sign test table for the given N and significance level. The result is significant if S ≤ the critical value.

Example: 10 participants rate anxiety before and after therapy. After excluding two ties (N=8), signs are: +, +, +, +, +, −, +, +. Count: 7 pluses, 1 minus. S = 1 (smaller count). At N=8, α=0.05, one-tailed, critical value = 1. Since S = 1 ≤ 1, the result is significant — anxiety is significantly higher before therapy.

The sign test is the least powerful common non-parametric test because it discards magnitude information — it records only the direction of each difference, not its size. The Wilcoxon signed-rank test (A-Level) addresses this by incorporating magnitude, making it more powerful.