The terms probability and likelihood are often used interchangeably in day-to-day conversation. They have specific meanings in the world of statistics, however, and understanding the difference is helpful in understanding statistical methods.
We’ll use examples to start. Take a coin flip: if you flip a coin, and you know it’s fair; a lifetime of experience gives you a model describing the behavior of the coin: half the time a flip will result in heads. You use this probability of 0.5 to decide whether you want to take a bet on the outcome of that coin.
If, on the other hand, you wanted to test whether that coin was fair, you might flip it many times. Say you flip it 1000 times, and you observe 505 heads and 495 tails. Now you want to know: is this coin fair? Is my model of the coin’s behavior correct? Now you are talking about a likelihood; what is the likelihood that this is a fair coin?
In short, a probability quantifies how often you observe a certain outcome of a test, given a certain understanding of the underlying data. A likelihood quantifies how good one’s model is, given a set of data that’s been observed. Probabilities describe test outcomes, while likelihoods describe models.
–Eleanor