Probability vs. Likelihood

Benjamin Chen
2022年1月21日
讀畢需時 2 分鐘

Ok, let me ask you a question. What is the difference between probability and likelihood? The more you think about it, the more confused you are, right? Don't worry! These are two very closely related concepts and many people mix them up. This story will try to break down the difference between the two so that you'll never mix them up again!

The best way to differentiate the two terms is to visualize them using a distribution. Let's use a normal distribution to demonstrate (note that this works for all continuous distributions).

Probability

You may recall from introductory statistics that probability, in the context of distribution, is measured by the area under the curve. If you need a recall, you may refer to this post here. Say we have a variable X that is normally distributed with a mean of 10 and a standard deviation of 2.

The probability that variable X is greater than 10 but less than 12 would be the area in red (below).

In mathematical notation, we would denote the probability as:

In other words, we are saying that this is the probability that X will be between 10 and 12 given the normal distribution (with mean 10 and std 2). So far so good?

Now, when we want to find the probability to a different X, all we have to do is change the left-side of the notation. For example, the notation for the probability that X is greater than 12 but less than 14 would be:

Likelihood

Alright! Probability is easy. What about likelihood? In short, the likelihood is the y-axis value of the graph. For instance, the likelihood that X is 12 would be 0.12.

I understand that, at this point, you might be super confused, but hold on. Everything will become clear once we take a look at the mathematical notation of likelihood.

The likelihood that X is 12 can be denoted as follow:

We can see that the two sides of the notation were flipped when compared to the probability. Take some time to digest what this likelihood notation means. Likelihood refers to the 'likelihood' of a distribution given the data. So if we were to change our distribution, we will obtain a different likelihood for the same data. The likelihood that the normal distribution has a mean of 12 and a standard deviation of 2 will be 0.20 when X is 12.