Range coding - Revision history

Belbury: /* How range coding works */ enlarge diagram

2025-01-13T13:20:41Z

How range coding works: enlarge diagram

← Previous revision		Revision as of 13:20, 13 January 2025
Line 4:		Line 4:

	== How range coding works ==		== How range coding works ==
	[[File:Range encoding.jpg\|thumb\|~~right~~\|Graphical representation of the coding process. The message being encoded here is "AABA<EOM>"]]		[[File:Range encoding.jpg\|thumb\|upright=1.5\|Graphical representation of the coding process. The message being encoded here is "AABA<EOM>"]]
	Range coding conceptually encodes all the symbols of the message into one number, unlike [[Huffman coding]] which assigns each symbol a bit-pattern and concatenates all the bit-patterns together. Thus range coding can achieve greater compression ratios than the one-bit-per-symbol lower bound on Huffman coding and it does not suffer the inefficiencies that Huffman does when dealing with probabilities that are not an exact [[power of two]].		Range coding conceptually encodes all the symbols of the message into one number, unlike [[Huffman coding]] which assigns each symbol a bit-pattern and concatenates all the bit-patterns together. Thus range coding can achieve greater compression ratios than the one-bit-per-symbol lower bound on Huffman coding and it does not suffer the inefficiencies that Huffman does when dealing with probabilities that are not an exact [[power of two]].

IznoRepeat: /* External links */ add WP:TEMPLATECAT to remove from template; genfixes

2024-12-06T00:53:37Z

External links: add WP:TEMPLATECAT to remove from template; genfixes

← Previous revision		Revision as of 00:53, 6 December 2024
Line 232:		Line 232:
	[[Category:Entropy coding]]		[[Category:Entropy coding]]
	[[Category:Articles with example C code]]		[[Category:Articles with example C code]]
			[[Category:Data compression]]

KinthermStopenfi at 05:34, 22 September 2024

2024-09-22T05:34:46Z

← Previous revision		Revision as of 05:34, 22 September 2024
Line 107:		Line 107:
	One problem that can occur with the <code>Encode</code> function above is that <code>range</code> might become very small but <code>low</code> and <code>low+range</code> still have differing first digits. This could result in the interval having insufficient precision to distinguish between all of the symbols in the alphabet. When this happens we need to fudge a little, output the first couple of digits even though we might be off by one, and re-adjust the range to give us as much room as possible.		One problem that can occur with the <code>Encode</code> function above is that <code>range</code> might become very small but <code>low</code> and <code>low+range</code> still have differing first digits. This could result in the interval having insufficient precision to distinguish between all of the symbols in the alphabet. When this happens we need to fudge a little, output the first couple of digits even though we might be off by one, and re-adjust the range to give us as much room as possible.

	For example, imagine the input stream has led the encoder to the right-open interval [59888, 60188), that is, <code>low = 59888</code> and <code>range = 300</code>. The trick is to narrow down the interval to [59888, 60000) = ['''59'''888, '''59'''999], which allows the encoder to emit two of the left-most digits of <code>low</code>, and ~~re-ajust~~ the interval to [88800, 99999] = [88800, 100000), that is, <code>low = 88800</code> and <code>range = 100000 - low</code>.		For example, imagine the input stream has led the encoder to the right-open interval [59888, 60188), that is, <code>low = 59888</code> and <code>range = 300</code>. The trick is to narrow down the interval to [59888, 60000) = ['''59'''888, '''59'''999], which allows the encoder to emit two of the left-most digits of <code>low</code>, and readjust the interval to [88800, 99999] = [88800, 100000), that is, <code>low = 88800</code> and <code>range = 100000 - low</code>.

	The decoder will be following the same steps so it will know when it needs to do this to keep in sync.		The decoder will be following the same steps so it will know when it needs to do this to keep in sync.

2804:D41:A71B:2F00:A559:D54C:A79:B1BF: /* How range coding works */

2024-03-29T12:24:39Z

How range coding works

← Previous revision		Revision as of 12:24, 29 March 2024
Line 105:		Line 105:
	</syntaxhighlight>		</syntaxhighlight>

	One problem that can occur with the <code>Encode</code> function above is that <code>range</code> might become very small but <code>low</code> and <code>low+range</code> still have differing first digits. This could result in the interval having insufficient precision to distinguish between all of the symbols in the alphabet. When this happens we need to fudge a little, output the first couple of digits even though we might be off by one, and re-adjust the range to give us as much room as possible~~. The decoder will be following the same steps so it will know when it needs to do this to keep in sync~~.		One problem that can occur with the <code>Encode</code> function above is that <code>range</code> might become very small but <code>low</code> and <code>low+range</code> still have differing first digits. This could result in the interval having insufficient precision to distinguish between all of the symbols in the alphabet. When this happens we need to fudge a little, output the first couple of digits even though we might be off by one, and re-adjust the range to give us as much room as possible.

			For example, imagine the input stream has led the encoder to the right-open interval [59888, 60188), that is, <code>low = 59888</code> and <code>range = 300</code>. The trick is to narrow down the interval to [59888, 60000) = ['''59'''888, '''59'''999], which allows the encoder to emit two of the left-most digits of <code>low</code>, and re-ajust the interval to [88800, 99999] = [88800, 100000), that is, <code>low = 88800</code> and <code>range = 100000 - low</code>.

			The decoder will be following the same steps so it will know when it needs to do this to keep in sync.

	<syntaxhighlight lang="csharp">		<syntaxhighlight lang="csharp">

109.144.217.218: /* External links */ Category change

2023-08-17T16:09:56Z

External links: Category change

← Previous revision		Revision as of 16:09, 17 August 2023
Line 226:		Line 226:

	{{DEFAULTSORT:Range Coding}}		{{DEFAULTSORT:Range Coding}}
	[[Category:~~Lossless~~ ~~compression algorithms~~]]		[[Category:Entropy coding]]
	[[Category:Articles with example C code]]		[[Category:Articles with example C code]]

Richgel999: /* External links */

2023-04-27T12:24:10Z

External links

← Previous revision		Revision as of 12:24, 27 April 2023
Line 221:		Line 221:
	* [http://www.ezcodesample.com/reanatomy.html "Anatomy of Range Encoder" by Andrew Polar]		* [http://www.ezcodesample.com/reanatomy.html "Anatomy of Range Encoder" by Andrew Polar]
	* [https://github.com/jkbonfield/rans_static Fast implementation of range coding and rANS of James K. Bonfield]		* [https://github.com/jkbonfield/rans_static Fast implementation of range coding and rANS of James K. Bonfield]
			* [https://github.com/richgel999/sserangecoding A fast open source implementation of a 24-bit SSE 4.1 interleaved Range Coder]

	{{Compression Methods}}		{{Compression Methods}}

Michael Hardy: /* See also */

2023-01-08T01:41:44Z

Furrykef: /* How range coding works */ rm redundant link

2022-04-20T10:47:58Z

How range coding works: rm redundant link

← Previous revision		Revision as of 10:47, 20 April 2022
Line 5:		Line 5:
	== How range coding works ==		== How range coding works ==
	[[File:Range encoding.jpg\|thumb\|right\|Graphical representation of the coding process. The message being encoded here is "AABA<EOM>"]]		[[File:Range encoding.jpg\|thumb\|right\|Graphical representation of the coding process. The message being encoded here is "AABA<EOM>"]]
	Range coding conceptually encodes all the symbols of the message into one number, unlike [[Huffman coding]] which assigns each symbol a bit-pattern and concatenates all the bit-patterns together. Thus range coding can achieve greater compression ratios than the one-bit-per-symbol lower bound on [[Huffman coding]] and it does not suffer the inefficiencies that Huffman does when dealing with probabilities that are not an exact [[power of two]].		Range coding conceptually encodes all the symbols of the message into one number, unlike [[Huffman coding]] which assigns each symbol a bit-pattern and concatenates all the bit-patterns together. Thus range coding can achieve greater compression ratios than the one-bit-per-symbol lower bound on Huffman coding and it does not suffer the inefficiencies that Huffman does when dealing with probabilities that are not an exact [[power of two]].

	The central concept behind range coding is this: given a large-enough range of [[integer]]s, and a probability estimation for the symbols, the initial range can easily be divided into sub-ranges whose sizes are proportional to the probability of the symbol they represent. Each symbol of the message can then be encoded in turn, by reducing the current range down to just that sub-range which corresponds to the next symbol to be encoded. The decoder must have the same probability estimation the encoder used, which can either be sent in advance, derived from already transferred data or be part of the compressor and decompressor.		The central concept behind range coding is this: given a large-enough range of [[integer]]s, and a probability estimation for the symbols, the initial range can easily be divided into sub-ranges whose sizes are proportional to the probability of the symbol they represent. Each symbol of the message can then be encoded in turn, by reducing the current range down to just that sub-range which corresponds to the next symbol to be encoded. The decoder must have the same probability estimation the encoder used, which can either be sent in advance, derived from already transferred data or be part of the compressor and decompressor.

86.11.13.144: fix inconsistent indentation

2022-02-04T11:54:08Z

fix inconsistent indentation

← Previous revision		Revision as of 11:54, 4 February 2022
Line 193:		Line 193:
	int GetValue(int total)		int GetValue(int total)
	{		{
	return (code - low) / (range / total);		return (code - low) / (range / total);
	}		}
	</syntaxhighlight>		</syntaxhighlight>

Uzume: cleanup after rename

2021-09-19T17:15:30Z

cleanup after rename

← Previous revision		Revision as of 17:15, 19 September 2021
Line 1:		Line 1:
	'''Range encoding''' is an [[entropy coding]] method defined by G. Nigel N. Martin in a 1979 paper,<ref name=compressconsult>[http://www.compressconsult.com/rangecoder/#download G. Nigel N. Martin, ''Range encoding: An algorithm for removing redundancy from a digitized message''], Video & Data Recording Conference, [[Southampton]], UK, July 24–27, 1979.</ref> which effectively rediscovered the FIFO arithmetic code first introduced by Richard Clark Pasco in 1976.<ref>"Source coding algorithms for fast data compression" Richard Clark Pasco, Stanford, CA 1976</ref> Given a stream of symbols and their probabilities, a range coder produces a space-efficient stream of bits to represent these symbols and, given the stream and the probabilities, a range decoder reverses the process.		'''Range coding''' (or '''range encoding''') is an [[entropy coding]] method defined by G. Nigel N. Martin in a 1979 paper,<ref name=compressconsult>[http://www.compressconsult.com/rangecoder/#download G. Nigel N. Martin, ''Range encoding: An algorithm for removing redundancy from a digitized message''], Video & Data Recording Conference, [[Southampton]], UK, July 24–27, 1979.</ref> which effectively rediscovered the FIFO arithmetic code first introduced by Richard Clark Pasco in 1976.<ref>"Source coding algorithms for fast data compression" Richard Clark Pasco, Stanford, CA 1976</ref> Given a stream of symbols and their probabilities, a range coder produces a space-efficient stream of bits to represent these symbols and, given the stream and the probabilities, a range decoder reverses the process.

	Range coding is very similar to [[arithmetic ~~encoding~~]], except that ~~encoding~~ is done with digits in any base, instead of with bits, and so it is faster when using larger bases (e.g. a [[byte]]) at small cost in compression efficiency.<ref>"[http://people.xiph.org/~tterribe/notes/range.html On the Overhead of Range Coders]", Timothy B. Terriberry, Technical Note 2008</ref> After the expiration of the first (1978) arithmetic coding patent,<ref>{{US patent\|4122440}} ~~—~~ (IBM) Filed March 4, 1977, Granted 24 October 1978 (Now expired)</ref> range ~~encoding~~ appeared to clearly be free of patent encumbrances. This particularly drove interest in the technique in the [[open source]] community. Since that time, patents on various well-known arithmetic coding techniques have also expired.		Range coding is very similar to [[arithmetic coding]], except that coding is done with digits in any base, instead of with bits, and so it is faster when using larger bases (e.g. a [[byte]]) at small cost in compression efficiency.<ref>"[http://people.xiph.org/~tterribe/notes/range.html On the Overhead of Range Coders]", Timothy B. Terriberry, Technical Note 2008</ref> After the expiration of the first (1978) arithmetic coding patent,<ref>{{US patent\|4122440}} — (IBM) Filed March 4, 1977, Granted 24 October 1978 (Now expired)</ref> range coding appeared to clearly be free of patent encumbrances. This particularly drove interest in the technique in the [[open source]] community. Since that time, patents on various well-known arithmetic coding techniques have also expired.

	== How range ~~encoding~~ works ==		== How range coding works ==
	[[File:Range encoding.jpg\|thumb\|right\|Graphical representation of the ~~encoding~~ process. The message being encoded here is "AABA<EOM>"]]		[[File:Range encoding.jpg\|thumb\|right\|Graphical representation of the coding process. The message being encoded here is "AABA<EOM>"]]
	Range ~~encoding~~ conceptually encodes all the symbols of the message into one number, unlike [[Huffman coding]] which assigns each symbol a bit-pattern and concatenates all the bit-patterns together. Thus range ~~encoding~~ can achieve greater compression ratios than the one-bit-per-symbol lower bound on [[Huffman ~~encoding~~]] and it does not suffer the inefficiencies that Huffman does when dealing with probabilities that are not exact [[power ~~of two\|powers~~ of two]].		Range coding conceptually encodes all the symbols of the message into one number, unlike [[Huffman coding]] which assigns each symbol a bit-pattern and concatenates all the bit-patterns together. Thus range coding can achieve greater compression ratios than the one-bit-per-symbol lower bound on [[Huffman coding]] and it does not suffer the inefficiencies that Huffman does when dealing with probabilities that are not an exact [[power of two]].

	The central concept behind range ~~encoding~~ is this: given a large-enough range of [[integer]]s, and a probability estimation for the symbols, the initial range can easily be divided into sub-ranges whose sizes are proportional to the probability of the symbol they represent. Each symbol of the message can then be encoded in turn, by reducing the current range down to just that sub-range which corresponds to the next symbol to be encoded. The decoder must have the same probability estimation the encoder used, which can either be sent in advance, derived from already transferred data or be part of the compressor and decompressor.		The central concept behind range coding is this: given a large-enough range of [[integer]]s, and a probability estimation for the symbols, the initial range can easily be divided into sub-ranges whose sizes are proportional to the probability of the symbol they represent. Each symbol of the message can then be encoded in turn, by reducing the current range down to just that sub-range which corresponds to the next symbol to be encoded. The decoder must have the same probability estimation the encoder used, which can either be sent in advance, derived from already transferred data or be part of the compressor and decompressor.

	When all symbols have been encoded, merely identifying the sub-range is enough to communicate the entire message (presuming of course that the decoder is somehow notified when it has extracted the entire message). A single integer is actually sufficient to identify the sub-range, and it may not even be necessary to transmit the entire integer; if there is a sequence of digits such that every integer beginning with that prefix falls within the sub-range, then the prefix alone is all that's needed to identify the sub-range and thus transmit the message.		When all symbols have been encoded, merely identifying the sub-range is enough to communicate the entire message (presuming of course that the decoder is somehow notified when it has extracted the entire message). A single integer is actually sufficient to identify the sub-range, and it may not even be necessary to transmit the entire integer; if there is a sequence of digits such that every integer beginning with that prefix falls within the sub-range, then the prefix alone is all that's needed to identify the sub-range and thus transmit the message.
Line 44:		Line 44:
	And since <EOM> is our final symbol, our final range is <nowiki>[25056, 25920)</nowiki>. Because all five-digit integers starting with "251" fall within our final range, it is one of the three-digit prefixes we could transmit that would unambiguously convey our original message. (The fact that there are actually eight such prefixes in all implies we still have inefficiencies. They have been introduced by our use of [[decimal\|base 10]] rather than [[Binary numeral system\|base 2]].)		And since <EOM> is our final symbol, our final range is <nowiki>[25056, 25920)</nowiki>. Because all five-digit integers starting with "251" fall within our final range, it is one of the three-digit prefixes we could transmit that would unambiguously convey our original message. (The fact that there are actually eight such prefixes in all implies we still have inefficiencies. They have been introduced by our use of [[decimal\|base 10]] rather than [[Binary numeral system\|base 2]].)

	The central problem may appear to be selecting an initial range large enough that no matter how many symbols we have to encode, we will always have a current range large enough to divide into non-zero sub-ranges. In practice, however, this is not a problem, because instead of starting with a very large range and gradually narrowing it down, the encoder works with a smaller range of numbers at any given time. After some number of digits have been encoded, the leftmost digits will not change. In the example after ~~encoding~~ just three symbols, we already knew that our final result would start with "2". More digits are shifted in on the right as digits on the left are sent off. This is illustrated in the following code:		The central problem may appear to be selecting an initial range large enough that no matter how many symbols we have to encode, we will always have a current range large enough to divide into non-zero sub-ranges. In practice, however, this is not a problem, because instead of starting with a very large range and gradually narrowing it down, the encoder works with a smaller range of numbers at any given time. After some number of digits have been encoded, the leftmost digits will not change. In the example after coding just three symbols, we already knew that our final result would start with "2". More digits are shifted in on the right as digits on the left are sent off. This is illustrated in the following code:

	<syntaxhighlight lang="csharp">		<syntaxhighlight lang="csharp">
Line 200:		Line 200:

	== Relationship with arithmetic coding ==		== Relationship with arithmetic coding ==
	[[Arithmetic coding]] is the same as range ~~encoding~~, but with the integers taken as being the numerators of [[Fraction (mathematics)\|fractions]]. These fractions have an implicit, common denominator, such that all the fractions fall in the range <nowiki>[0,1)</nowiki>. Accordingly, the resulting arithmetic code is interpreted as beginning with an implicit "0". As these are just different interpretations of the same coding methods, and as the resulting arithmetic and range codes are identical, each arithmetic coder is its corresponding range encoder, and vice versa. In other words, arithmetic coding and range ~~encoding~~ are just two, slightly different ways of understanding the same thing.		[[Arithmetic coding]] is the same as range coding, but with the integers taken as being the numerators of [[Fraction (mathematics)\|fractions]]. These fractions have an implicit, common denominator, such that all the fractions fall in the range <nowiki>[0,1)</nowiki>. Accordingly, the resulting arithmetic code is interpreted as beginning with an implicit "0". As these are just different interpretations of the same coding methods, and as the resulting arithmetic and range codes are identical, each arithmetic coder is its corresponding range encoder, and vice versa. In other words, arithmetic coding and range coding are just two, slightly different ways of understanding the same thing.

	In practice, though, so-called range ''encoders'' tend to be implemented pretty much as described in Martin's paper,<ref name=compressconsult/> while arithmetic coders more generally tend not to be called range encoders. An often noted feature of such range encoders is the tendency to perform renormalization a byte at a time, rather than one bit at a time (as is usually the case). In other words, range encoders tend to use bytes as ~~encoding~~ digits, rather than bits. While this does reduce the amount of compression that can be achieved by a very small amount, it is faster than when performing renormalization for each bit.		In practice, though, so-called range ''encoders'' tend to be implemented pretty much as described in Martin's paper,<ref name=compressconsult/> while arithmetic coders more generally tend not to be called range encoders. An often noted feature of such range encoders is the tendency to perform renormalization a byte at a time, rather than one bit at a time (as is usually the case). In other words, range encoders tend to use bytes as coding digits, rather than bits. While this does reduce the amount of compression that can be achieved by a very small amount, it is faster than when performing renormalization for each bit.

	== See also ==		== See also ==
Line 224:		Line 224:
	{{Compression Methods}}		{{Compression Methods}}

	{{DEFAULTSORT:Range ~~Encoding~~}}		{{DEFAULTSORT:Range Coding}}
	[[Category:Lossless compression algorithms]]		[[Category:Lossless compression algorithms]]
	[[Category:Articles with example C code]]		[[Category:Articles with example C code]]

← Previous revision		Revision as of 01:41, 8 January 2023
Line 206:		Line 206:
	== See also ==		== See also ==
	* [[Arithmetic coding]]		* [[Arithmetic coding]]
	* [[Asymmetric ~~Numeral~~ ~~Systems~~]]		* [[Asymmetric numeral systems]]
	* [[Data compression]]		* [[Data compression]]
	* [[Entropy encoding]]		* [[Entropy encoding]]