Bayesian probability

Bayesianism is the philosophical tenet that the mathematical theory of probability applies to the degree of plausibility of statements, or to the degree of belief of rational agents in the truth of statements.

The terms subjective probability, personal probability, epistemic probability and logical probability are used to describe what Bayesians believe in. Not all of these terms are synonymous, however.

Advocates of "logical probability" would like to codify techniques whereby if two people have the same prior information relevant to the truth of an uncertain proposition, then they would assign the same probability. No one has any idea how to do that except in simple cases, and then the validity of proposed methods is subject to philosophical controversy.

"Subjective probability" is supposed to measure how sure someone is of an uncertain proposition.

The Bayesian approach is in contrast to frequency probability where probability is held to be derived from observed or imagined frequency distributions or proportions of populations. The difference has many implications for the methods by which statistics is practiced when following one model or the other.

History of Bayesian probability

Bayesianism is named after Thomas Bayes, the originator of Bayes' theorem. This theorem is often used to update the plausibility of a given statement in light of new evidence. Laplace (1812) rediscovered the theorem and put it to good use in solving problems in celestial mechanics, medical statistics and, by some accounts, even jurisprudence.

For instance, he estimated the mass of Saturn, given orbital data that were available to him from various astronomical observations. He presented the result together with an indication of its uncertainty, stating it like this: `It is a bet of 11000 to 1 that the error in this result is not within 1/100th of its value'. He would have won the bet, as another 150 years' accumulation of data has changed the estimate by only 0.63%. According to the frequency probability definition, however, we are not permitted to use probability theory to tackle this problem. This is because the mass of Saturn is a constant and not a random variable, therefore, it has no frequency distribution and so probability theory cannot be used.

The Bayesian concepts were championed by L. J. Savage and Bruno de Finetti and others such as Frank P. Ramsey. They created the idea of defining rational belief as an abstraction of betting behavior subject to the constraint that one doesn't want to be inconsistent in his behavior. A series of critiques of statistical methods was based on this concept and formed the basis of debate from the 1950s and statisticians remain divided on the issue.

Applications of Bayesian probability

Today, there are a variety of applications of personal probability that have gained wide acceptance. Some schools of thought emphasise Cox's theorem and the Principle of Maximum Entropy as cornerstones of the theory, while others may claim that Bayesian methods are more general and give better results in practice than frequency probability.