Looking for a gravitational wave is like trying to hear a single bee in a hive. It is easy for the single buzz – the gravitational wave – to get lost in the overall hum of the hive – the noise of the universe. It’s also easy to mistake another sound, such as someone starting a chainsaw nearby, for the signal.
Statistical analysis is our most powerful tool for avoiding both of these unhappy outcomes and initial reports on new discoveries in physics are, therefore, often expressed in terms of their statistical significance. Saying something is statistically significant is the same as saying it’s unlikely to have occurred by chance.
In physics, statistical significance is usually expressed in units of the standard deviation, or σ (sigma), from the average value.
For instance, consider an experiment where one flips a coin 100 times. The expected outcome is 50 heads and the standard deviation of such an experiment is five. If you get 55 heads, that’s a one‐sigma effect. If you get 60 heads, that’s a two‐sigma effect; 65 is three-sigma; 70 is four-sigma; 75 is five-sigma, and so on.
Your intuition probably tells you that 55 is probably just chance, 60 is odd, 65 is startling, and more than that means there’s something up with the coin. So it is in physics.
Five-sigma corresponds to a p-value, or probability, of 3x107, or about one in 3.5 million. That is, there’s less than one chance in 3.5 million that the effect being seen is due to random chance.
The number of sigma we assign to an effect expresses how much confidence we have that the signal is not the result of random chance – that we have not mistaken a chainsaw for a bee. A five‐sigma effect is the gold standard of proof of a new discovery in physics.
A normal distribution curve illustrating standard deviations, or sigmas. (Image by mwtoews on Wikimedia Commons.)