Quadratic unconstrained binary optimization: Difference between revisions

Quadratic unconstrained binary optimization (QUBO), also known as unconstrained binary quadratic programming (UBQP), is a combinatorial optimization problem with a wide range of applications from finance and economics to machine learning. QUBO is an NP hard problem, and for many classical problems from theoretical computer science, like maximum cut, graph coloring and the partition problem, embeddings into QUBO have been formulated. Embeddings for machine learning models include support-vector machines, clustering and probabilistic graphical models. Moreover, due to its close connection to Ising models, QUBO constitutes a central problem class for adiabatic quantum computation, where it is solved through a physical process called quantum annealing.

Definition

Let $\mathbb {B} =\lbrace 0,1\rbrace$ the set of binary digits (or bits), then $\mathbb {B} ^{n}$ is the set of binary vectors of fixed length $n\in \mathbb {N}$. Given a symmetric upper triangular matrix ${\boldsymbol {Q}}\in \mathbb {R} ^{n\times n}$, whose entries $Q_{ij}$ define a weight for each pair of indices $i,j\in \lbrace 1,\dots ,n\rbrace$ within the binary vector, we can define the function $f_{\boldsymbol {Q}}:\mathbb {B} ^{n}\rightarrow \mathbb {R}$ that assigns a value to each binary vector ${\boldsymbol {x}}$ through

$f_{\boldsymbol {Q}}({\boldsymbol {x}})={\boldsymbol {x}}^{\intercal }{\boldsymbol {Qx}}=\sum _{i=1}^{n}\sum _{j=1}^{n}Q_{ij}x_{i}x_{j}.$

Intuitively, the weight $Q_{ij}$ is added if both $x_{i}=1$ and $x_{j}=1$ for all $i,j\in \lbrace 1,\dots ,n\rbrace$. The QUBO problem consists of finding a binary vector ${\boldsymbol {x}}^{*}$ that is minimal with respect to $f_{\boldsymbol {Q}}$, namely

$\forall {\boldsymbol {x}}\in \mathbb {B} ^{n}:~f_{\boldsymbol {Q}}({\boldsymbol {x}}^{*})\leq f_{\boldsymbol {Q}}({\boldsymbol {x}})$

In general, ${\boldsymbol {x}}^{*}$ is not unique, meaning there may be a set of minimizing vectors with equal value w.r.t. $f_{\boldsymbol {Q}}$. The complexity of QUBO arises from the number of candidate binary vectors to be evaluated, as $\left|\mathbb {B} ^{n}\right|=2^{n}$ grows exponentially in $n$.

Sometimes, QUBO is defined as the problem of maximizing $f_{\boldsymbol {Q}}$, which is equivalent to minimizing $f_{-{\boldsymbol {Q}}}=-f_{\boldsymbol {Q}}$.

Properties

QUBO is scale invariant for positive factors $\alpha >0$, which leave the optimum ${\boldsymbol {x}}^{*}$ unchanged:

$f_{\alpha {\boldsymbol {Q}}}({\boldsymbol {x}})={\boldsymbol {x}}^{\intercal }(\alpha {\boldsymbol {Q}}){\boldsymbol {x}}=\alpha ({\boldsymbol {x}}^{\intercal }{\boldsymbol {Qx}})=\alpha f_{\boldsymbol {Q}}({\boldsymbol {x}})$.

In its general form, QUBO is NP-hard and cannot be solved efficiently by any polynomial-time algorithm. However, there are polynomially-solvable special cases, where ${\boldsymbol {Q}}$ has certain properties, for example:

• If all coefficients are positive, the optimum is trivially ${\boldsymbol {x}}^{*}=(0,\dots ,0)^{\intercal }$. Similarly, if all coefficients are negative, the optimum is ${\boldsymbol {x}}^{*}=(1,\dots ,1)^{\intercal }$.
• If ${\boldsymbol {Q}}$ is diagonal, the bits can be optimized independently, and the problem is solvable in ${\mathcal {O}}(n)$. The optimal variable assignments are simply $x_{i}^{*}=1$ if $Q_{ii}<0$, and $x_{i}^{*}=0$ otherwise.
• If all off-diagonal elements of ${\boldsymbol {Q}}$ are non-positive, the corresponding QUBO problem is solvable in polynomial time.

QUBO can be solved using integer linear programming solvers like CPLEX or Gurobi Optimizer. This is possible since QUBO can be reformulated as a linear constrained binary optimization problem. To achieve this, substitute the product $x_{i}x_{j}$ by an additional binary variable $z_{ij}\in \mathbb {B}$ and add the constraints $x_{i}\geq z_{ij}$, $x_{j}\geq z_{ij}$ and $x_{i}+x_{j}-1\leq z_{ij}$. Note that $z_{ij}$ can also be relaxed to continuous variables within the bounds zero and one.

Applications

QUBO is a structurally simple, yet computationally hard optimization problem. It can be used to encode a wide range of optimization problems from various scientific areas.

Cluster Analysis

Binary Clustering with QUBO

A bad cluster assignment.

A good cluster assignment.

Visual representation of a clustering problem with 20 points: Circles of the same color belong to the same cluster. Each circle can be understood as a binary variable in the corresponding QUBO problem.

As an illustrative example of how QUBO can be used to encode an optimization problem, we consider the problem of cluster analysis. Here, we are given a set of $N=20$ points in 2D space, described by a matrix ${\boldsymbol {X}}\in \mathbb {R} ^{N\times 2}$, where each row contains two cartesian coordinates. We want to assign each point to one of two classes or clusters, such that points in the same cluster are similar to each other. For two clusters, we can assign a binary variable $x_{i}\in \mathbb {B}$ to the point corresponding to the $i$-th row in ${\boldsymbol {X}}$, indicating whether it belongs to the first ($x_{i}=0$) or second cluster ($x_{i}=1$). Consequently, we have 20 binary variables, which form a binary vector ${\boldsymbol {x}}\in \mathbb {B} ^{20}$ that corresponds to a cluster assignment of all points (see figure).

One way to derive a clustering is to consider the pairwise distances between points. Given a cluster assignment ${\boldsymbol {x}}$, the expression $x_{i}x_{j}+(1-x_{i})(1-x_{j})$ evaluates to 1 if points $i$ and $j$ are in the same cluster. Similarly, $x_{i}(1-x_{j})+(1-x_{i})x_{j}=1$ indicates that they are in different clusters. Let $d_{ij}>0$ denote the Euclidean distance between the points $i$ and $j$, i.e.,

$d_{ij}={\sqrt {{\boldsymbol {X}}_{i}^{\intercal }{\boldsymbol {X}}_{j}}}$,

where ${\boldsymbol {X}}_{i}$ is the $i$-th row of ${\boldsymbol {X}}$.

In order to define a cost function to minimize, when points $i$ and $j$ are in the same cluster we add their positive distance $d_{ij}$, and subtract it when they are in different clusters. This way, an optimal solution tends to place points which are far apart into different clusters, and points that are close into the same cluster.

Let ${\boldsymbol {D}}\in \mathbb {R} ^{N\times N}$ with $D_{ij}=d_{ij}/2$ for all $i,j\in \lbrace 1,\dots ,n\rbrace$. Given an assignment ${\boldsymbol {x}}\in \mathbb {B} ^{N}$, such a cost function is given by

${\begin{aligned}f({\boldsymbol {x}})&={\boldsymbol {x}}^{\intercal }{\boldsymbol {Dx}}-{\boldsymbol {x}}^{\intercal }{\boldsymbol {D}}({\boldsymbol {1}}-{\boldsymbol {x}})-({\boldsymbol {1}}-{\boldsymbol {x}})^{\intercal }{\boldsymbol {Dx}}+({\boldsymbol {1}}-{\boldsymbol {x}})^{\intercal }{\boldsymbol {D}}({\boldsymbol {1}}-{\boldsymbol {x}})\\&=4{\boldsymbol {x}}^{\intercal }{\boldsymbol {D}}{\boldsymbol {x}}-4{\boldsymbol {1}}^{\intercal }{\boldsymbol {D}}{\boldsymbol {x}}+{\boldsymbol {1}}^{\intercal }{\boldsymbol {D1}},\end{aligned}}$

where ${\boldsymbol {1}}=(1,1,\dots ,1)^{\intercal }\in \mathbb {R} ^{N}$.

From the second line we can see that this expression can be re-arranged to a QUBO problem by defining

${\boldsymbol {Q}}=4{\boldsymbol {D}}-4\operatorname {diag} [{\boldsymbol {D1}}]$

and ignoring the constant term ${\boldsymbol {1}}^{\intercal }{\boldsymbol {D1}}$. Using these parameters, a binary vector minimizing this QUBO instance ${\boldsymbol {Q}}$ will correspond to an optimal cluster assignment w.r.t. above cost function.

Connection to Ising models

QUBO is very closely related and computationally equivalent to the Ising model, whose Hamiltonian function is defined as

$H({\boldsymbol {\sigma }})={\boldsymbol {\sigma }}^{\intercal }{\boldsymbol {J}}{\boldsymbol {\sigma }}+{\boldsymbol {h}}^{\intercal }{\boldsymbol {\sigma }}=\sum _{i,j}J_{ij}\sigma _{i}\sigma _{j}+\sum _{j}h_{j}\sigma _{j}$

with real-valued parameters $h_{j},J_{ij},\mu$ for all $i,j$. The spin variables $\sigma _{j}$ are binary with values from $\lbrace -1,+1\rbrace$ instead of $\mathbb {B}$. Moreover, in the Ising model the variables are typically arranged in a lattice where only neighboring pairs of variables $\langle i~j\rangle$ can have non-zero coefficients.

Applying the identity $\sigma =1-2x$ yields an equivalent QUBO problem

${\begin{aligned}&{\boldsymbol {\sigma }}^{\intercal }{\boldsymbol {J}}{\boldsymbol {\sigma }}+{\boldsymbol {h}}^{\intercal }{\boldsymbol {\sigma }}\\&=({\boldsymbol {1}}-2{\boldsymbol {x}})^{\intercal }{\boldsymbol {J}}({\boldsymbol {1}}-2{\boldsymbol {x}})+{\boldsymbol {h}}^{\intercal }({\boldsymbol {1}}-2{\boldsymbol {x}})\\&=4{\boldsymbol {x}}^{\intercal }{\boldsymbol {J}}{\boldsymbol {x}}-4{\boldsymbol {1}}^{\intercal }{\boldsymbol {Jx}}+{\boldsymbol {1}}^{\intercal }{\boldsymbol {J1}}-2{\boldsymbol {h}}^{\intercal }{\boldsymbol {x}}+{\boldsymbol {h}}^{\intercal }{\boldsymbol {1}}\\&={\boldsymbol {x}}^{\intercal }(4{\boldsymbol {J}}){\boldsymbol {x}}-(4{\boldsymbol {J}}^{\intercal }{\boldsymbol {1}}+2{\boldsymbol {h}})^{\intercal }{\boldsymbol {x}}+\underbrace {{\boldsymbol {1}}^{\intercal }{\boldsymbol {J1}}+{\boldsymbol {h}}^{\intercal }{\boldsymbol {1}}} _{\text{const.}},\end{aligned}}$

whose weight matrix ${\boldsymbol {Q}}$ is given by

${\boldsymbol {Q}}=4{\boldsymbol {J}}-\operatorname {diag} [4{\boldsymbol {J}}^{\intercal }{\boldsymbol {1}}+2{\boldsymbol {h}}],$

again ignoring the constant term, which does not affect the minization. Using the identity $x=(1-\sigma )/2$, a QUBO problem with matrix ${\boldsymbol {Q}}$ can be converted to an equivalent Ising model using the same technique, yielding

${\begin{aligned}{\boldsymbol {J}}&={\boldsymbol {Q}}/4,&{\boldsymbol {h}}&=-({\boldsymbol {Q1}}+{\boldsymbol {Q}}^{\intercal }{\boldsymbol {1}})/4,\end{aligned}}$

and a constant offset of ${\boldsymbol {1}}^{\intercal }{\boldsymbol {Q1}}/4$.

External links

• QUBO Benchmark (Benchmark of software packages for the exact solution of QUBOs; part of the well-known Mittelmann benchmark collection)

• Endre Boros, Peter L Hammer & Gabriel Tavares (April 2007). "Local search heuristics for Quadratic Unconstrained Binary Optimization (QUBO)". Journal of Heuristics. 13 (2). Association for Computing Machinery: 99–132. doi:10.1007/s10732-007-9009-3. S2CID 32887708. Retrieved 12 May 2013.
• Di Wang & Robert Kleinberg (November 2009). "Analyzing quadratic unconstrained binary optimization problems via multicommodity flows". Discrete Applied Mathematics. 157 (18). Elsevier: 3746–3753. doi:10.1016/j.dam.2009.07.009. PMC 2808708. PMID 20161596.
• Hiroshima University and NTT DATA Group Corporation : "QUBO++ with ABS2 GPU QUBO Solver" # Software.