Nearest-neighbor chain algorithm - Revision history

Headbomb: /* Background */ | Altered journal. | Use this tool. Report bugs. | #UCB_Gadget

2025-06-06T00:34:51Z

Background: | Altered journal. | Use this tool. Report bugs. | #UCB_Gadget

← Previous revision		Revision as of 00:34, 6 June 2025
Line 36:		Line 36:
	\| arxiv = cs.DS/9912014		\| arxiv = cs.DS/9912014
	\| issue = 1		\| issue = 1
	\| journal = J. Experimental Algorithmics		\| journal = ACM Journal of Experimental Algorithmics
	\| pages = 1–23		\| pages = 1–23
	\| publisher = ACM		\| publisher = ACM

David Eppstein: Undid revision 1275193820 by 2A02:3100:3C9B:9500:C880:B670:2E09:30C5 (talk) unsourced

2025-02-11T17:46:28Z

Undid revision 1275193820 by 2A02:3100:3C9B:9500:C880:B670:2E09:30C5 (talk) unsourced

← Previous revision		Revision as of 17:46, 11 February 2025
Line 100:		Line 100:
	For the same reason, the total time used by the algorithm outside of these distance calculations is {{math\|O(''n''<sup>2</sup>)}}.<ref name="murtagh-tcj"/>		For the same reason, the total time used by the algorithm outside of these distance calculations is {{math\|O(''n''<sup>2</sup>)}}.<ref name="murtagh-tcj"/>

	Since the only data structure is the set of active clusters and the stack containing a subset of the active clusters, the space required is linear in the number of input points.<ref name="murtagh-tcj"/> However, an implementation using a pairwise distance matrix is often smaller, as every distance then is only computed once, and Lance-Williams-Equations can be used to reduce the cost of computing distances between clusters, at the cost of increasing memory requirements to quadratic memory. For the special case of squared Euclidean distance, cluster centers can replace the set of data points and also reduce the number of distance computations, while keepimg the memory usage linear.		Since the only data structure is the set of active clusters and the stack containing a subset of the active clusters, the space required is linear in the number of input points.<ref name="murtagh-tcj"/>
	Using index structures to accelerate nearest neighbor search - instead of scanning all other clusters - the algorithm can be used on many real data sets in subquadratic time with linear memory, however the theoretical worst case complexity does not improve and remains quadratic.

	==Correctness==		==Correctness==

2A02:3100:3C9B:9500:C880:B670:2E09:30C5: /* Time and space analysis */Complexity of variants

2025-02-11T16:41:35Z

Time and space analysis: Complexity of variants

← Previous revision		Revision as of 16:41, 11 February 2025
Line 100:		Line 100:
	For the same reason, the total time used by the algorithm outside of these distance calculations is {{math\|O(''n''<sup>2</sup>)}}.<ref name="murtagh-tcj"/>		For the same reason, the total time used by the algorithm outside of these distance calculations is {{math\|O(''n''<sup>2</sup>)}}.<ref name="murtagh-tcj"/>

	Since the only data structure is the set of active clusters and the stack containing a subset of the active clusters, the space required is linear in the number of input points.<ref name="murtagh-tcj"/>		Since the only data structure is the set of active clusters and the stack containing a subset of the active clusters, the space required is linear in the number of input points.<ref name="murtagh-tcj"/> However, an implementation using a pairwise distance matrix is often smaller, as every distance then is only computed once, and Lance-Williams-Equations can be used to reduce the cost of computing distances between clusters, at the cost of increasing memory requirements to quadratic memory. For the special case of squared Euclidean distance, cluster centers can replace the set of data points and also reduce the number of distance computations, while keepimg the memory usage linear.
			Using index structures to accelerate nearest neighbor search - instead of scanning all other clusters - the algorithm can be used on many real data sets in subquadratic time with linear memory, however the theoretical worst case complexity does not improve and remains quadratic.

	==Correctness==		==Correctness==

David Eppstein: Mauricio Resende

2024-06-13T00:35:21Z

Mauricio Resende

← Previous revision		Revision as of 00:35, 13 June 2024
Line 71:		Line 71:
	\| editor1-last = Abello \| editor1-first = James M.		\| editor1-last = Abello \| editor1-first = James M.
	\| editor2-last = Pardalos \| editor2-first = Panos M.		\| editor2-last = Pardalos \| editor2-first = Panos M.
	\| editor3-last = Resende \| editor3-first = Mauricio G. C.		\| editor3-last = Resende \| editor3-first = Mauricio G. C. \| editor3-link = Mauricio Resende
	\| contribution = Clustering in massive data sets		\| contribution = Clustering in massive data sets
	\| isbn = 978-1-4020-0489-6		\| isbn = 978-1-4020-0489-6

Citation bot: Add: s2cid, doi. | Use this bot. Report bugs. | Suggested by Whoop whoop pull up | Linked from User:David_Eppstein | #UCB_webform_linked 59/118

2023-01-27T00:58:10Z

Add: s2cid, doi. | Use this bot. Report bugs. | Suggested by Whoop whoop pull up | Linked from User:David_Eppstein | #UCB_webform_linked 59/118

← Previous revision		Revision as of 00:58, 27 January 2023
Line 42:		Line 42:
	\| url = http://www.jea.acm.org/2000/EppsteinDynamic/		\| url = http://www.jea.acm.org/2000/EppsteinDynamic/
	\| volume = 5		\| volume = 5
	\| year = 2000\| bibcode = 1999cs.......12014E }}.</ref><ref name="day-edels">{{citation		\| year = 2000\| doi = 10.1145/351827.351829 \| bibcode = 1999cs.......12014E \| s2cid = 1357701 }}.</ref><ref name="day-edels">{{citation
	\| last1 = Day \| first1 = William H. E.		\| last1 = Day \| first1 = William H. E.
	\| last2 = Edelsbrunner \| first2 = Herbert \| author2-link = Herbert Edelsbrunner		\| last2 = Edelsbrunner \| first2 = Herbert \| author2-link = Herbert Edelsbrunner
Line 52:		Line 52:
	\| url = http://www.cs.duke.edu/~edels/Papers/1984-J-05-HierarchicalClustering.pdf		\| url = http://www.cs.duke.edu/~edels/Papers/1984-J-05-HierarchicalClustering.pdf
	\| volume = 1		\| volume = 1
			\| year = 1984\| s2cid = 121201396
	~~\| year = 1984~~}}.</ref> The nearest-neighbor chain algorithm uses a smaller amount of time and space than the greedy algorithm by merging pairs of clusters in a different order. In this way, it avoids the problem of repeatedly finding closest pairs. Nevertheless, for many types of clustering problem, it can be guaranteed to come up with the same hierarchical clustering as the greedy algorithm despite the different merge order.<ref name="murtagh-tcj"/>		}}.</ref> The nearest-neighbor chain algorithm uses a smaller amount of time and space than the greedy algorithm by merging pairs of clusters in a different order. In this way, it avoids the problem of repeatedly finding closest pairs. Nevertheless, for many types of clustering problem, it can be guaranteed to come up with the same hierarchical clustering as the greedy algorithm despite the different merge order.<ref name="murtagh-tcj"/>

	==The algorithm==		==The algorithm==

Bruce1ee: fixed lint errors – file options; size is ignored when using frame

2022-05-16T06:09:43Z

fixed lint errors – file options; size is ignored when using frame

← Previous revision		Revision as of 06:09, 16 May 2022
Line 55:		Line 55:

	==The algorithm==		==The algorithm==
	[[File:Nearest-neighbor chain algorithm animated.gif\|frame~~\|300px~~\|alt=Animated execution of Nearest-neighbor chain algorithm\|Animation of the algorithm using Ward's distance. Black dots are points, grey regions are larger clusters, blue arrows point to nearest neighbors, and the red bar indicates the current chain. For visual simplicity, when a merge leaves the chain empty, it continues with the recently merged cluster.]]		[[File:Nearest-neighbor chain algorithm animated.gif\|frame\|alt=Animated execution of Nearest-neighbor chain algorithm\|Animation of the algorithm using Ward's distance. Black dots are points, grey regions are larger clusters, blue arrows point to nearest neighbors, and the red bar indicates the current chain. For visual simplicity, when a merge leaves the chain empty, it continues with the recently merged cluster.]]
	Intuitively, the nearest neighbor chain algorithm repeatedly follows a chain of clusters {{math\|''A'' → ''B'' → ''C'' → ...}} where each cluster is the nearest neighbor of the previous one, until reaching a pair of clusters that are mutual nearest neighbors.<ref name="murtagh-tcj">{{citation		Intuitively, the nearest neighbor chain algorithm repeatedly follows a chain of clusters {{math\|''A'' → ''B'' → ''C'' → ...}} where each cluster is the nearest neighbor of the previous one, until reaching a pair of clusters that are mutual nearest neighbors.<ref name="murtagh-tcj">{{citation
	\| last = Murtagh \| first = Fionn		\| last = Murtagh \| first = Fionn

David Eppstein: proper short description this time

2022-01-25T06:31:47Z

proper short description this time

← Previous revision		Revision as of 06:31, 25 January 2022
Line 1:		Line 1:
			{{Short description\|Stack-based method for clustering}}
	{{good article}}		{{good article}}
	In the theory of [[cluster analysis]], the '''nearest-neighbor chain algorithm''' is an [[algorithm]] that can speed up several methods for [[agglomerative hierarchical clustering]]. These are methods that take a collection of points as input, and create a hierarchy of clusters of points by repeatedly merging pairs of smaller clusters to form larger clusters. The clustering methods that the nearest-neighbor chain algorithm can be used for include [[Ward's method]], [[complete-linkage clustering]], and [[single-linkage clustering]]; these all work by repeatedly merging the closest two clusters but use different definitions of the distance between clusters. The cluster distances for which the nearest-neighbor chain algorithm works are called ''reducible'' and are characterized by a simple inequality among certain cluster distances.		In the theory of [[cluster analysis]], the '''nearest-neighbor chain algorithm''' is an [[algorithm]] that can speed up several methods for [[agglomerative hierarchical clustering]]. These are methods that take a collection of points as input, and create a hierarchy of clusters of points by repeatedly merging pairs of smaller clusters to form larger clusters. The clustering methods that the nearest-neighbor chain algorithm can be used for include [[Ward's method]], [[complete-linkage clustering]], and [[single-linkage clustering]]; these all work by repeatedly merging the closest two clusters but use different definitions of the distance between clusters. The cluster distances for which the nearest-neighbor chain algorithm works are called ''reducible'' and are characterized by a simple inequality among certain cluster distances.

David Eppstein: Revert old vandalism

2022-01-25T06:30:20Z

Revert old vandalism

← Previous revision		Revision as of 06:30, 25 January 2022
Line 1:		Line 1:
			{{good article}}
	{{Short description\|Yuvraj Singh is an Indian manister and writers}}{{good article}}
	In the theory of [[cluster analysis]], the '''nearest-neighbor chain algorithm''' is an [[algorithm]] that can speed up several methods for [[agglomerative hierarchical clustering]]. These are methods that take a collection of points as input, and create a hierarchy of clusters of points by repeatedly merging pairs of smaller clusters to form larger clusters. The clustering methods that the nearest-neighbor chain algorithm can be used for include [[Ward's method]], [[complete-linkage clustering]], and [[single-linkage clustering]]; these all work by repeatedly merging the closest two clusters but use different definitions of the distance between clusters. The cluster distances for which the nearest-neighbor chain algorithm works are called ''reducible'' and are characterized by a simple inequality among certain cluster distances.		In the theory of [[cluster analysis]], the '''nearest-neighbor chain algorithm''' is an [[algorithm]] that can speed up several methods for [[agglomerative hierarchical clustering]]. These are methods that take a collection of points as input, and create a hierarchy of clusters of points by repeatedly merging pairs of smaller clusters to form larger clusters. The clustering methods that the nearest-neighbor chain algorithm can be used for include [[Ward's method]], [[complete-linkage clustering]], and [[single-linkage clustering]]; these all work by repeatedly merging the closest two clusters but use different definitions of the distance between clusters. The cluster distances for which the nearest-neighbor chain algorithm works are called ''reducible'' and are characterized by a simple inequality among certain cluster distances.

Line 92:		Line 92:

	==Time and space analysis==		==Time and space analysis==
	Each iteration of the loop performs a single search for the nearest neighbor of a cluster, and either adds one cluster to the stack or removes two clusters from it. ~~Each~~ cluster is only ever added once to the stack, because when it is removed again it is immediately made inactive and merged. There are a total of {{math\|2''n'' − 2}} clusters that ever get added to the stack: {{math\|''n''}} single-point clusters in the initial set, and {{math\|''n'' − 2}} internal nodes other than the root in the binary tree representing the clustering. Therefore, the algorithm performs {{math\|2''n'' − 2}} pushing iterations and {{math\|''n'' − 1}} popping iterations.<ref name="murtagh-tcj"/>		Each iteration of the loop performs a single search for the nearest neighbor of a cluster, and either adds one cluster to the stack or removes two clusters from it. Every cluster is only ever added once to the stack, because when it is removed again it is immediately made inactive and merged. There are a total of {{math\|2''n'' − 2}} clusters that ever get added to the stack: {{math\|''n''}} single-point clusters in the initial set, and {{math\|''n'' − 2}} internal nodes other than the root in the binary tree representing the clustering. Therefore, the algorithm performs {{math\|2''n'' − 2}} pushing iterations and {{math\|''n'' − 1}} popping iterations.<ref name="murtagh-tcj"/>

	Each of these iterations may spend time scanning as many as {{math\|''n'' − 1}} inter-cluster distances to find the nearest neighbor.		Each of these iterations may spend time scanning as many as {{math\|''n'' − 1}} inter-cluster distances to find the nearest neighbor.
Line 202:		Line 202:

	===Distances sensitive to merge order===		===Distances sensitive to merge order===
	The above presentation explicitly disallowed distances sensitive to merge order. Indeed, allowing such distances can cause problems. In particular, there exist order-sensitive cluster distances which satisfy reducibility, but for which the above algorithm will return a hierarchy with suboptimal costs. Therefore, when cluster distances are defined by a recursive formula (as some of the ones discussed above are), care must be taken so that they do not use the hierarchy in a way which is sensitive to merge order.<ref>{{citation		The above presentation explicitly disallowed distances sensitive to merge order. Indeed, allowing such distances can cause problems. In particular, there exist order-sensitive cluster distances which satisfy reducibility, but for which the above algorithm will return a hierarchy with suboptimal costs. Therefore, when cluster distances are defined by a recursive formula (as some of the ones discussed above are), care must be taken that they do not use the hierarchy in a way which is sensitive to merge order.<ref>{{citation
	\| last=Müllner		\| last=Müllner
	\| first=Daniel		\| first=Daniel

Neelgajare: grammar

2021-12-24T07:27:07Z

grammar

← Previous revision		Revision as of 07:27, 24 December 2021
Line 202:		Line 202:

	===Distances sensitive to merge order===		===Distances sensitive to merge order===
	The above presentation explicitly disallowed distances sensitive to merge order. Indeed, allowing such distances can cause problems. In particular, there exist order-sensitive cluster distances which satisfy reducibility, but for which the above algorithm will return a hierarchy with suboptimal costs. Therefore, when cluster distances are defined by a recursive formula (as some of the ones discussed above are), care must be taken that they do not use the hierarchy in a way which is sensitive to merge order.<ref>{{citation		The above presentation explicitly disallowed distances sensitive to merge order. Indeed, allowing such distances can cause problems. In particular, there exist order-sensitive cluster distances which satisfy reducibility, but for which the above algorithm will return a hierarchy with suboptimal costs. Therefore, when cluster distances are defined by a recursive formula (as some of the ones discussed above are), care must be taken so that they do not use the hierarchy in a way which is sensitive to merge order.<ref>{{citation
	\| last=Müllner		\| last=Müllner
	\| first=Daniel		\| first=Daniel

Neelgajare: word choice

2021-12-24T07:26:17Z

word choice

← Previous revision		Revision as of 07:26, 24 December 2021
Line 92:		Line 92:

	==Time and space analysis==		==Time and space analysis==
	Each iteration of the loop performs a single search for the nearest neighbor of a cluster, and either adds one cluster to the stack or removes two clusters from it. ~~Every~~ cluster is only ever added once to the stack, because when it is removed again it is immediately made inactive and merged. There are a total of {{math\|2''n'' − 2}} clusters that ever get added to the stack: {{math\|''n''}} single-point clusters in the initial set, and {{math\|''n'' − 2}} internal nodes other than the root in the binary tree representing the clustering. Therefore, the algorithm performs {{math\|2''n'' − 2}} pushing iterations and {{math\|''n'' − 1}} popping iterations.<ref name="murtagh-tcj"/>		Each iteration of the loop performs a single search for the nearest neighbor of a cluster, and either adds one cluster to the stack or removes two clusters from it. Each cluster is only ever added once to the stack, because when it is removed again it is immediately made inactive and merged. There are a total of {{math\|2''n'' − 2}} clusters that ever get added to the stack: {{math\|''n''}} single-point clusters in the initial set, and {{math\|''n'' − 2}} internal nodes other than the root in the binary tree representing the clustering. Therefore, the algorithm performs {{math\|2''n'' − 2}} pushing iterations and {{math\|''n'' − 1}} popping iterations.<ref name="murtagh-tcj"/>

	Each of these iterations may spend time scanning as many as {{math\|''n'' − 1}} inter-cluster distances to find the nearest neighbor.		Each of these iterations may spend time scanning as many as {{math\|''n'' − 1}} inter-cluster distances to find the nearest neighbor.