The Intuition Behind Hypothesis Testing

When I was a student studying hypothesis testing for the first time, I admit I didn’t understand anything about it. I simply followed the required steps that allowed me to answer a question correctly. Now that I’ve become a TA and actually have to teach the topic, I had to understand the logic behind hypothesis testing. So here is the intuition of hypothesis testing clearly explained!

---

What is Hypothesis Testing?

Hypothesis Testing is a test on an assumption about a population parameter. (a population parameter is just any measure that describes a population eg. population mean). For example, if a university claims that the mean GPA of all students (population) is 2.8, we can perform a hypothesis testing that tests whether this claim is true.

But how exactly do we test this claim?

The simplest way is to take a sample from the population, calculate the sample mean, and compare it to the claimed population mean.

Assuming that the sample is selected through valid sampling methods (meaning the sample is representative of the population), the sample mean should be somewhat close to the value of the claimed population mean. If the sample mean is too far off the claimed population mean, then we have reason to believe that the actual population mean is not the claimed population mean.

Sampling Distribution

There is, however, a certain level of variance in calculating the sample mean. When we select a sample, the mean of the sample will vary slightly depending on the observations in the sample. The sampling distribution describes the distribution of the sample mean. Before you advance, I recommend you take a deeper look at sampling distribution first, because hypothesis testing is built upon the sampling distribution. Here is another post of mine that clearly explains sampling distribution.

Statistics 9: Sampling Distribution Clearly Explained!

If you read my post about sampling distribution, you should know by now that sampling distribution is the distribution of many many many sample means. Let’s come back to our example on GPA. Assume the claim that the population mean GPA is 2.8 is true (and let’s also assume that we know the population standard deviation is 0.4). With n=30 (sample size), we can find the mean and standard deviation of the sampling distribution.

We also know that the sampling distribution is normal regardless of the shape of the population distribution because n is greater than 30 (central limit theorem). The sampling distribution will look somewhat like this:

We can see that, based on our assumption of the claim, most sample mean GPAs center around 2.80. The area under the curve of the sampling distribution will be the probability of obtaining a sample with a sample mean within a certain range. For example, the shaded area in yellow will be the probability of obtaining a sample with a sample mean that is greater than 2.90. The total area under the entire curve is 1.

We can see that, based on our assumption that the population GPA is 2.8, it’s unlikely to obtain samples with a sample mean of 2.6 and 2.95. Although not entirely impossible, both of these values are located on the tails of the sampling distribution with very low likelihood.

Now let’s say we randomly select a sample (n=30) and we find out that it has a sample mean GPA of 3.0. According to sampling distribution (which is derived based on the assumption that population mean GPA=2.80), the probability of obtaining a sample with a sample mean greater than 3.0 is extremely low (area in red).

Again, although the probability of this happening is not impossible, the probability is so low that I am starting to have doubts about this sampling distribution (which is again, derived based on the assumption that the population mean GPA=2.80).

Obtaining a sample with a sample mean of 3.0 would make a lot more sense and be much more probable if the center of the sampling distribution is near 3.0 (which is deducted from the fact that the population mean GPA should also be near 3.0).

So based on my sample and sample mean of 3.0, I have reason to believe that the true population mean is not what the university claims at 2.8, but a higher value.

This is the idea of hypothesis testing. Given a certain assumption about the population parameter (population mean GPA is 2.8 in our example), we can test this assumption using sample statistics. If the likelihood of obtaining the sample statistic is too low, then we have good reason to believe that the assumption could be incorrect. Therefore, we should “reject” the assumption about the population parameter. On the other hand, if the likelihood of obtaining the sample statistic is at a reasonable level, then the assumption about the population parameter is probably correct. Therefore, we “do not reject” the assumption about the population parameter.

---

Conclusion

Perfect! Now that we've understood the intuition behind hypothesis testing, let's advance to the actual implementation of the test. In the next story, we will run over the critical steps of performing hypothesis testing.