Logistic regression

Logistic regression is a statistical regression model for binary dependent variables. It can be considered as a generalized linear model that utilizes the logit as its link function, and has binomially distributed errors.

The model takes the form

$\operatorname {logit} (p_{i})=\ln \left({\frac {p_{i}}{1-p_{i}}}\right)=\alpha +\beta _{1}x_{1,i}+\cdots +\beta _{k}x_{k,i},$

$i=1,\dots ,n,\,$

where

$p_{i}=\Pr(Y_{i}=1).\,$

The logarithm of the odds (probability divided by one minus the probability) of the outcome is modelled as a linear function of the explanatory variables, $X_{1}$ to $X_{k}$. This can be written equivalently as

$p_{i}=\Pr(Y_{i}=1|X)={\frac {e^{\alpha +\beta _{1}x_{1,i}+\cdots +\beta _{k}x_{k,i}}}{1+e^{\alpha +\beta _{1}x_{1,i}+\cdots +\beta _{k}x_{k,i}}}}.$

The interpretation of the $\beta$ parameter estimates is as a multiplicative effect on the odds ratio. In the case of a dichotomous explanatory variable, for instance sex, $e^{\beta }$ (the antilog of $\beta$) is the estimate of the odds-ratio of having the outcome for, say, males compared with females.

The parameters $\alpha ,\beta _{1},...,\beta _{k}$ are usually estimated by maximum likelihood.

Extensions of the model exist to cope with multi-category dependent variables and ordinal dependent variables.

See also

• Data mining
• Linear discriminant analysis
• Perceptron
• Probit model

References

• Agresti, Alan: Categorical Data Analysis. New York: Wiley, 1990.
• Amemiya, T., 1985, Advanced Econometrics, Harvard University Press.
• Hosmer, D. W. and S. Lemeshow: Applied logistic regression. New York; Chichester, Wiley, 2000.