How to do a Power Analysis

To determine the number of replicates needed to detect a ‘true’ difference between sample means use the following formula (Sokal and Rohlf, 1981 Biometry: The Principles and Practice of Statistics in Biological Research. W.H. Freeman & Co., New York. p 262, Box 9.13):

N = 2(s/d)2 * {tα [v] + t2(1-P)[v]}2

where

  • N=number of replications
  • s= true Standard deviation
  • d=smallest true difference that is desired to be detected (note: one only need know the ratio of s/d, not their actual values)
  • v=degrees of freedom (df) with ‘a’ groups and ‘n’ replicates per group
  • α=significance level
  • P=desired probability that a significant difference will be found (power of the test)

To determine how many animals you will need (Nstable), you need some idea of what the variability is for the parameters you will be measuring.  Nstable is obtained through an interative method.  To calculate Nstable, you need the df (which is a function of N).  Use an estimate of N (Nintitial) to obtain an estimate of df (dfinitial ) which is then used to calculate  N2.  This new estimate of N (N2) is then used to calculate a new df, (df2), which is used, in turn, for the calculation of a new estimate of N (N3). This method is repeated until a ‘stable’ N (Nstable) is found.

Example

In the following example, we want to measure how chemicals affect CYP1A enzyme activity in fish. We look up the data we have (or the data of others) on the variability of this enzyme in fish:

Control fish: 269 ± 49 pmol product/min/mg protein, n = 3 individual fish

Treated fish: 1,453 ± 139 pmol product/min/mg protein, n =3 individual fish

If unknown, the term s/d can be replaced by CV/D were CV is the Coefficient of Variation (in %) and D is d in %.  The coefficient of variation for these activities varied between ~ 10 – 18% (that is, 49/269 * 100 = 18.2%; 139/1453*100 = 9.6%); in other studies we conducted, the CV for this enzyme varied between 34 – 55%. We would like to detect at least a 50% difference between means. Using an average coefficient variation of 30%, an α level = 0.05, a desired probability of P = 0.8 and 32 degrees of freedom1:

N2 = 2 (30/50)2 * {t0.05[32] + t2(1-0.8)[32]}2

= 0.72 * {2.037 + 0.853}2

= 6

Note: Since the enzyme activity could go either up or down with treatment, look up the ‘t’ values in a Two-tailed Student’s t Table. The value of ‘t’ for an α of 0.05 and 32 df (t0.05[32]) = 2.037; the value of ‘ t′ for a P of 0.8 and 32 df (t2(1-0.8)[32]) = (t0.4[32]) = 0.853.

The second ‘round’ of calculations (using N2 = 6) leads to a df2 of 40 and a new N3 of 6, it is then stable and Nstable = 6. So, 6 fish per group is the number required to detect a significant difference of at least 50 % between the treatments (at a 0.05 level α), with a likelihood of detecting this difference 80% of the time, if this difference truly exists (this is the power of the test, P).

G*Power

G*Power (external link) is a free software to estimate the sample size and to conduct statistical power analysis.

 

1 Each fish experiment consists of 8 groups (2 sites x 2 sexes x 2 treatments = 8 groups) with 5 replicates per group (Nintitial for the calculation of the dfinitial), yielding v = 8 * (5 – 1) = 32 df.