Likelihood function

In the English language, the word likelihood means the same as, or almost the same as, the word 'probability'.

In probability theory the technical term likelihood is a concept related to, but distinct from, that of probability. (Confusingly, likelihood is also used in its informal English sense in some discussions of the interpretation of mathematical probability.)

Unlike probability, which estimates the frequency of or degree of belief in unknown consequences of known causes, likelihood works backwards, from observed results to hypothetical models and parameters.

The likelihood function is the function which specifies the probability of the sample observed on the basis of a known model, in terms of the model's parameters.

Note: This is not the same as the probability of those parameters being the right ones, given the observed sample.

For example, if I toss a coin, with a probability p_H of landing heads up ('H'), the probability of getting two heads in two trials ('HH') is p_H². If p_H = 0.5, then the probability of seeing two heads is 0.25.

In symbols, we can say the above as

p('HH' | p_H = 0.5) = 0.25

Another way of saying this is that the likelihood of p_H = 0.5 given the observation 'HH' is 0.25. But this is not the same as saying that the probability of p_H = 0.5 given the observation is 0.25.

In fact, since the parameter p_H is continuous, we should think of the likelihood function as a likelihood density function.

The likelihood function does not necessarily follow all the axioms of probability: for example, the integral over the likelihood density function does not in general sum to 1.

In this example, the integral of the likelihood density over the interval [0, 1] in p_H is 1/3, clearly demonstrating that the likelihood density function cannot be interpreted as a probability density function for p_H.