Algorithms for calculating variance - Revision history

139.47.111.8: /* Naïve algorithm */

2025-04-29T14:32:50Z

Naïve algorithm

← Previous revision		Revision as of 14:32, 29 April 2025
Line 6:		Line 6:
	A formula for calculating the variance of an entire [[statistical population\|population]] of size ''N'' is:		A formula for calculating the variance of an entire [[statistical population\|population]] of size ''N'' is:

	:<math>\sigma^2 = \overline{(x^2)} - \bar x^2 = \frac{\sum_{i=1}^N x_i^2}{N} - \frac{(\sum_{i=1}^N x_i~~)^2~~}{N}.</math>		:<math>\sigma^2 = \overline{(x^2)} - \bar x^2 = \frac{\sum_{i=1}^N x_i^2}{N} - \left(\frac{\sum_{i=1}^N x_i}{N}\right)^2</math>

	Using [[Bessel's correction]] to calculate an [[estimator bias\|unbiased]] estimate of the population variance from a finite [[statistical sample\|sample]] of ''n'' observations, the formula is:		Using [[Bessel's correction]] to calculate an [[estimator bias\|unbiased]] estimate of the population variance from a finite [[statistical sample\|sample]] of ''n'' observations, the formula is:

139.47.111.8: /* Naïve algorithm */

2025-04-29T14:30:23Z

Naïve algorithm

← Previous revision		Revision as of 14:30, 29 April 2025
Line 6:		Line 6:
	A formula for calculating the variance of an entire [[statistical population\|population]] of size ''N'' is:		A formula for calculating the variance of an entire [[statistical population\|population]] of size ''N'' is:

	:<math>\sigma^2 = \overline{(x^2)} - \bar x^2 = \frac {\sum_{i=1}^N x_i^2 - (\sum_{i=1}^N x_i)^2/N}{N}.</math>		:<math>\sigma^2 = \overline{(x^2)} - \bar x^2 = \frac{\sum_{i=1}^N x_i^2}{N} - \frac{(\sum_{i=1}^N x_i)^2}{N}.</math>

	Using [[Bessel's correction]] to calculate an [[estimator bias\|unbiased]] estimate of the population variance from a finite [[statistical sample\|sample]] of ''n'' observations, the formula is:		Using [[Bessel's correction]] to calculate an [[estimator bias\|unbiased]] estimate of the population variance from a finite [[statistical sample\|sample]] of ''n'' observations, the formula is:

Azmisov: Revert prior edit which removed reliability weights calculation; Bessel correction for reliability weights are well known and defined, and sources have been included

2025-01-18T18:05:40Z

Revert prior edit which removed reliability weights calculation; Bessel correction for reliability weights are well known and defined, and sources have been included

← Previous revision		Revision as of 18:05, 18 January 2025
Line 173:		Line 173:
	population_variance = S / w_sum		population_variance = S / w_sum
	# Bessel's correction for weighted samples		# Bessel's correction for weighted samples
	# ~~for integer frequency~~ weights		# Frequency weights
	sample_frequency_variance = S / (w_sum - 1)		sample_frequency_variance = S / (w_sum - 1)

			# Reliability weights
			sample_reliability_variance = S / (1 - w_sum2 / (w_sum**2))
	</syntaxhighlight>		</syntaxhighlight>

Citation bot: Removed parameters. | Use this bot. Report bugs. | Suggested by Dominic3203 | Linked from User:Talgalili/sandbox | #UCB_webform_linked 42/2730

2025-01-12T05:09:36Z

Removed parameters. | Use this bot. Report bugs. | Suggested by Dominic3203 | Linked from User:Talgalili/sandbox | #UCB_webform_linked 42/2730

← Previous revision		Revision as of 05:09, 12 January 2025
Line 104:		Line 104:
	</syntaxhighlight>		</syntaxhighlight>

	This algorithm is numerically stable if ''n'' is small.<ref name="Einarsson2005"/><ref>{{cite book \|last=Higham \|first=Nicholas J. \|url=https://epubs.siam.org/doi/book/10.1137/1.9780898718027 \|title=Accuracy and Stability of Numerical Algorithms \|publisher=Society for Industrial and Applied Mathematics \|year=2002 \|edition=2nd \|publication-place=Philadelphia, PA \|chapter=Problem 1.10 \|doi= 10.1137/1.9780898718027\|isbn=978-0-898715-21-7 \|id=e{{ISBN\|978-0-89871-802-7}}, 2002075848 ~~\|postscript=~~}} Metadata also listed at [https://dl.acm.org/doi/10.5555/579525 ACM Digital Library].</ref> However, the results of both of these simple algorithms ("naïve" and "two-pass") can depend inordinately on the ordering of the data and can give poor results for very large data sets due to repeated roundoff error in the accumulation of the sums. Techniques such as [[compensated summation]] can be used to combat this error to a degree.		This algorithm is numerically stable if ''n'' is small.<ref name="Einarsson2005"/><ref>{{cite book \|last=Higham \|first=Nicholas J. \|url=https://epubs.siam.org/doi/book/10.1137/1.9780898718027 \|title=Accuracy and Stability of Numerical Algorithms \|publisher=Society for Industrial and Applied Mathematics \|year=2002 \|edition=2nd \|publication-place=Philadelphia, PA \|chapter=Problem 1.10 \|doi= 10.1137/1.9780898718027\|isbn=978-0-898715-21-7 \|id=e{{ISBN\|978-0-89871-802-7}}, 2002075848 }} Metadata also listed at [https://dl.acm.org/doi/10.5555/579525 ACM Digital Library].</ref> However, the results of both of these simple algorithms ("naïve" and "two-pass") can depend inordinately on the ordering of the data and can give poor results for very large data sets due to repeated roundoff error in the accumulation of the sums. Techniques such as [[compensated summation]] can be used to combat this error to a degree.

	==Welford's online algorithm==		==Welford's online algorithm==

Graham87: 6 revisions imported: import old edits from "Variance/Algorithm" in the August 2001 database dump

2024-12-23T11:11:59Z

6 revisions imported: import old edits from "Variance/Algorithm" in the August 2001 database dump

← Previous revision	Revision as of 11:11, 23 December 2024
(No difference)

Graham87: 1 revision imported: import old edit from nost:Algorithms for calculating variance

2024-12-23T11:05:16Z

1 revision imported: import old edit from nost:Algorithms for calculating variance

← Previous revision	Revision as of 11:05, 23 December 2024
(No difference)

2A02:3100:4763:5B00:BD51:9F82:D847:6814: /* Weighted incremental algorithm */"reliability weights" does not even appear to be well-defined? Refer to the "further information" page only, to not have different versions.

2024-10-24T07:23:02Z

Weighted incremental algorithm: "reliability weights" does not even appear to be well-defined? Refer to the "further information" page only, to not have different versions.

← Previous revision		Revision as of 07:23, 24 October 2024
Line 173:		Line 173:
	population_variance = S / w_sum		population_variance = S / w_sum
	# Bessel's correction for weighted samples		# Bessel's correction for weighted samples
	# ~~Frequency~~ weights		# for integer frequency weights
	sample_frequency_variance = S / (w_sum - 1)		sample_frequency_variance = S / (w_sum - 1)

	# Reliability weights
	sample_reliability_variance = S / (1 - w_sum2 / (w_sum**2))
	</syntaxhighlight>		</syntaxhighlight>

2A02:3100:4763:5B00:BD51:9F82:D847:6814: stack exchange is not a reliable source. read West 1979 and the linked Weighted_arithmetic_mean 1252910133 by Svalorzen (talk)

2024-10-24T07:16:49Z

stack exchange is not a reliable source. read West 1979 and the linked Weighted_arithmetic_mean 1252910133 by Svalorzen (talk)

← Previous revision		Revision as of 07:16, 24 October 2024
Line 158:		Line 158:

	==Weighted incremental algorithm==		==Weighted incremental algorithm==
	The algorithm can be extended to handle unequal sample weights, replacing the simple counter ''n'' with the sum of weights seen so far <ref>~~https://stats~~.~~stackexchange~~.~~com~~/~~questions/47325/bias-correction-in-weighted~~-~~variance~~</ref>, ~~with~~ this [[incremental computing\|incremental ~~approach~~]]:		The algorithm can be extended to handle unequal sample weights, replacing the simple counter ''n'' with the sum of weights seen so far. West (1979)<ref>{{cite journal \|last=West \|first=D. H. D. \|year=1979 \|title=Updating Mean and Variance Estimates: An Improved Method \|journal=[[Communications of the ACM]] \|volume=22 \|issue=9 \|pages=532–535 \|doi=10.1145/359146.359153 \|s2cid=30671293 \|doi-access=free}}</ref> suggests this [[incremental computing\|incremental algorithm]]:

	<syntaxhighlight lang="python">		<syntaxhighlight lang="python">
Line 175:		Line 175:
	# Frequency weights		# Frequency weights
	sample_frequency_variance = S / (w_sum - 1)		sample_frequency_variance = S / (w_sum - 1)

	# Reliability weights		# Reliability weights
	sample_reliability_variance = S / (~~(w_sum**2~~ - w_sum2) / w_sum)		sample_reliability_variance = S / (1 - w_sum2 / (w_sum**2))
	</syntaxhighlight>		</syntaxhighlight>

Svalorzen: The previously reported algorithm for reliability weights was wrong, as was the variance measure reported in the previously cited paper. You can see discussion about this in the stackexchange reference added, where the paper's approach (called B) is specifically pointed out as wrong. The new code for reliability weights correctly implements approach C in the same post, which is correct.

2024-10-23T14:00:19Z

The previously reported algorithm for reliability weights was wrong, as was the variance measure reported in the previously cited paper. You can see discussion about this in the stackexchange reference added, where the paper's approach (called B) is specifically pointed out as wrong. The new code for reliability weights correctly implements approach C in the same post, which is correct.

← Previous revision		Revision as of 14:00, 23 October 2024
Line 158:		Line 158:

	==Weighted incremental algorithm==		==Weighted incremental algorithm==
	The algorithm can be extended to handle unequal sample weights, replacing the simple counter ''n'' with the sum of weights seen so far~~. West~~ ~~(1979)~~<ref>~~{{cite journal \|last=West \|first=D~~. ~~H. D. \|year=1979 \|title=Updating Mean and Variance Estimates: An Improved Method \|journal=[[Communications of the ACM]] \|volume=22 \|issue=9 \|pages=532–535 \|doi=10~~.~~1145~~/~~359146.359153 \|s2cid=30671293 \|doi~~-~~access=free}}~~</ref> ~~suggests~~ this [[incremental computing\|incremental ~~algorithm~~]]:		The algorithm can be extended to handle unequal sample weights, replacing the simple counter ''n'' with the sum of weights seen so far <ref>https://stats.stackexchange.com/questions/47325/bias-correction-in-weighted-variance</ref>, with this [[incremental computing\|incremental approach]]:

	<syntaxhighlight lang="python">		<syntaxhighlight lang="python">
Line 176:		Line 176:
	sample_frequency_variance = S / (w_sum - 1)		sample_frequency_variance = S / (w_sum - 1)
	# Reliability weights		# Reliability weights
	sample_reliability_variance = S / (w_sum - w_sum2 / w_sum)		sample_reliability_variance = S / ((w_sum**2 - w_sum2) / w_sum)
	</syntaxhighlight>		</syntaxhighlight>

Softlavender: Reverted edit by Akaibu (talk) to last version by 212.185.66.16

2024-09-13T05:11:33Z

Reverted edit by Akaibu (talk) to last version by 212.185.66.16

← Previous revision		Revision as of 05:11, 13 September 2024
Line 1:		Line 1:
	{{Short description\|Important algorithms in numerical statistics}}		{{Short description\|Important algorithms in numerical statistics}}
	{{Lead rewrite\|date=September 2024}}
	{{Use dmy dates\|date=July 2020}}		{{Use dmy dates\|date=July 2020}}
	'''Algorithms for calculating variance''' play a major role in [[computational statistics]]. A key difficulty in the design of good [[algorithm]]s for this problem is that formulas for the [[variance]] may involve sums of squares, which can lead to [[numerical instability]] as well as to [[arithmetic overflow]] when dealing with large values.		'''Algorithms for calculating variance''' play a major role in [[computational statistics]]. A key difficulty in the design of good [[algorithm]]s for this problem is that formulas for the [[variance]] may involve sums of squares, which can lead to [[numerical instability]] as well as to [[arithmetic overflow]] when dealing with large values.