Nonnegative rank four boundaries
Matrices of nonnegative rank at most form a semialgebraic set. This semialgebraic set is understood for . In this paper, we study boundaries of the set of matrices of nonnegative rank at most four using notions from the rigidity theory of frameworks. In the nonnegative rank three case, all boundaries are trivial or consist of matrices that have only infinitesimally rigid factorizations. For arbitrary nonnegative rank, we give a necessary condition on zero entries of a nonnegative factorization for the factorization to be infinitesimally rigid, and we show that in the case of matrices of nonnegative rank four, there exists an infinitesimally rigid realization for every zero pattern that satisfies this necessary condition. However, the nonnegative rank four case is much more complicated than the nonnegative rank three case, and there exist matrices on the boundary that have factorizations that are not infinitesimally rigid. We discuss two such examples.
Nonnegative rank of a matrix is the smallest such that can be factorized as where and . One of the first applications of nonnegative rank was introduced by Yannakakis in the context of combinatorial optimization where it measures the complexity of a linear program . In the same paper, Yannakakis showed a connection to communication complexity theory, and subsequently, nonnegative rank has been studied from the perspective of polyhedral geometry , topology , machine learning , algebraic statistics [6, 10], and complexity theory [18, 12].
Matrices of size and nonnegative rank at most form a semialgebraic set, which we denote by . This means that the set of matrices of nonnegative rank at most is a finite union of sets that are defined by finitely polynomial equations and inequalities. These equations and inequalities are easy to describe when . Namely, all entries of a matrix have to be nonnegative and all -minors need to vanish. Kubjas, Robeva and Sturmfels gave a semialgebraic description for matrices of nonnegative rank at most three . A semialgebraic description of the set allows one to check directly whether a matrix has nonnegative rank at most without constructing a nonnegative factorization of the matrix. It is known from the work of Vavasis that checking whether nonnegative rank of a matrix equals its rank is NP-hard  and the best known algorithm for deciding whether an matrix has nonnegative rank at most runs in time by the work of Moitra . Vandaele, Gillis, Glineur and Tuyttens suggest heuristics for computing nonnegative matrix factorizations in practice .
The semialgebraic description for matrices of nonnegative rank at most three builds on the work of Mond, Smith and van Straten on the space of nonnegative factorizations of a nonnegative matrix  (these spaces are known to be universal by the universality theorem of Shitov ). In particular, the first step towards deriving the semialgebraic description in  is a characterization of the boundaries of the semialgebraic set which correspond to critical points of the Morse function in . Minimal generators and Gröbner basis of the ideal of the Zariski closure of the boundary are described in . The goal of this paper is to study boundaries of the set of matrices of rank and nonnegative rank equal to , which we denote by , for . The set is closely related to the set . In particular, these sets differ only by a lower dimensional subset defined by the vanishing of all minors, and hence is dense in . We will focus on the case .
We first show that the boundary components described in  have natural analogs in higher rank. A necessary and sufficient condition for a matrix with positive entries to lie on the boundary of is that all its size three nonnegative factorizations contain seven zeros in special positions. Due to similarity with several notions in rigidity theory of frameworks, in this paper we will adopt some terminology from that field. In this terminology, a matrix with positive entries lies on the boundary of if and only if all its size three nonnegative factorizations are infinitesimally rigid. The main result of our paper is Theorem 4.5, which gives a necessary condition for a nonnegative factorization to be infinitesimally rigid for an arbitrary . The first part of the necessary condition requires the matrix to have at least zeros and the second part requires the zeros to be in a special position. We call the positions of zeros in a nonnegative factorization a zero pattern.
For matrices of nonnegative rank four, we show that for every zero pattern that satisfies the necessary condition in Theorem 4.5, there exists a factorization that realizes the zero pattern and that is infinitesimally rigid. For larger matrices of nonnegative rank four, we are not able to verify this, and it is left open whether this is because of computational challenges or because not every zero pattern that satisfies the conditions in Theorem 4.5 has an infinitesimally rigid realization.
However, it turns out that the situation is much more complicated for nonnegative rank four than for nonnegative rank three and there are matrices with positive entries on the boundary of such that not all their nonnegative factorizations are infinitesimally rigid. As we will demonstrate in Section 5.1, there are matrices with positive entries on the boundary of such that all their size four nonnegative factorizations have less than thirteen zeros. The example we present is a modification of an example by Shitov in , where he shows that nonnegative rank depends on the field (the dependance of nonnegative rank on the field is also studied in [16, 2]). The example we present has exactly one nonnegative factorization; this factorization has twelve zeros and it satisfies a further polynomial constraint.
Another situation that can occur in nonnegative rank four but not three is that there are matrices with positive entries on the boundary of that have a higher dimensional set of nonnegative factorizations, meaning they are not locally rigid (and hence not infinitesimally rigid). This shows that the full geometric characterization of nonnegative rank three boundary matrices given in [13, Lemma 3.10] does not generalize to higher ranks. A counterexample with nonnegative rank four is given in [13, Figure 8]. In Section 5.2 we generalize this example to a construction for producing boundary matrices with non-rigid factorizations for nonnegative rank four or higher out of matrices with lower nonnegative rank.
Almost every aspect of nonnegative matrix factorizations is hard: Nonnegative matrix factorization is polynomial-time equivalent to the existential theory of reals, the factorization spaces are universal  and nonnegative rank depends on the field [16, 2]. Boundaries of the set is yet another addition to this list. Since spaces of nonnegative factorizations are lower-dimensional for matrices on the boundary of and hence the most difficult to find, we hope that better understanding of the boundaries will be also useful for developing better practical nonnegative matrix factorization algorithms.
The outline of our paper is the following. In Section 2.1, we describe the geometric characterization of nonnegative rank via nested polytopes. In Section 2.2, we study nonnegative factorizations through intersections of the space of rank factorizations with a polyhedral cone. In Section 3, we study locally rigid nonnegative factorizations and show that by adding positive rows to and positive columns to , a locally rigid factorization can be made to a boundary factorization. In Section 4, we study a special case of locally rigid factorizations, called infinitesimally rigid factorizations. This section contains our main result, Theorem 4.5, which gives a necessary condition for a factorization to be infinitesimally rigid, and realizations of infinitesimally rigid zero patterns for matrices of nonnegative rank four that satisfy this necessary condition. Finally, in Section 5, we discuss matrices on the boundary that have nonnegative factorizations that are not infinitesimally rigid. Code for computations in this paper is available at
2.1 Geometric characterization via nested polytopes
Nonnegative rank can be characterized geometrically via nested polyhedral cones. We describe two equivalent constructions from the literature for matrices of equal rank and nonnegative rank.
The first description is due to Cohen and Rothblum . It defines as the convex cone spanned by the columns of and as the intersection of and the column span of . Let be a size- rank factorization of , and let be the simplicial cone spanned by the columns of . Since and have the same column span, the cones , and all span the same dimension- subspace of . If is nonnegative, then is contained in the positive orthant, so . If is nonnegative then each column of is a conic combination of columns of with coefficients given by columns of , hence . Conversely, one can construct a size- nonnegative factorization from a dimension- simplicial cone that is nested between and by taking the generating rays of to be the columns of . Therefore the matrix has nonnegative rank if and only if there exists a simplicial cone such that . Gillis and Glineur defined the restricted nonnegative rank of as the smallest number of rays of a cone that can be nested between and , which is an upper bound on the nonnegative rank in the case that the rank and nonnegative rank differ.
The work of Vavasis  presents a second description of the same nested cones up to a linear transformation. Fix a particular rank factorization of (not necessarily nonnegative). All rank factorizations of have the form where is an invertible matrix. Let be the cone spanned by the columns of ; let be the cone spanned by the columns of ; let be the cone that is defined by . The linear map sends these three polyhedral cones to their counterparts in the first construction.
Zeros in a nonnegative factorization correspond to incidence relations between the three cones, , and . In particular, a zero in means that a ray of lies on a facet of . A zero in means that a ray of lies on a facet of .
One often considers nested polytopes instead of nested cones. One gets nested polytopes from nested cones by intersecting the cones with an affine plane, which is usually defined by setting the sum of the coordinates to 1.
Below we present a different geometric picture to help understand when a rank- matrix has nonnegative rank and specifically when it lies on the boundary of the semialgebraic set. We will however at times refer to the nested polytopes .
2.2 Geometry of factorizations
Let be the space of all real valued matrices of rank exactly equal to and let denote the subset of matrices with nonnegative rank . Let
denote the usual matrix multiplication map. Then is the image of the positive orthant . That is if for some with nonnegative entries.
Fix a rank- matrix and a rank factorization with . The fiber of is
This fiber is a real -dimensional smooth irreducible variety. Let
is the graph of the inverse function on matrices. The injective linear map
sends to .
The map is a fiber bundle, with fiber .
A matrix has a set of linearly independent columns. Given a rank factorization of , the same set of columns is linearly independent in . Call the submatrix they form . Then is a rank factorization of with having the submatrix in these columns equal to the identity, and this is the unique factorization of with that property. Let be the subset of of pairs in which has this particular submatrix equal to the identity.
All matrices in have a unique factorization in unless there is linear dependence among the chosen columns. Such exceptions form a lower dimensional subset, so in particular has a neighborhood of matrices with factorizations in . Then has product structure by map
which can be checked is continuous with continuous inverse. This proves the fiber bundle structure of . ∎
A pair represents a nonnegative factorization of if and are both nonnegative. Let denote the columns of and the rows of . The inequality gives linear inequalities on and defines a polyhedral cone in with at most facets which we will denote . Similarly defines a polyhedral cone in with at most facets. The nonnegative factorizations of then correspond to the set . Let , which is itself a polyhedral cone.
is the outer cone and is dual to the inner cone in the second geometric characterization in Section 2.1.
If is a nonnegative factorization of then is as well for any diagonal matrix with positive diagonal entries. We will generally be interested only in factorizations modulo this scaling.
Let be a rank factorization of .
if and only if .
if and only if .
The first statement follows from the definitions of and .
Suppose intersects the interior of and let be a point in the intersection. Then and are both strictly positive full rank matrices which is to say that has an open neighborhood contained in . Since is a fiber bundle, it is an open mapping. Therefore is an open neighborhood of in , so is in the interior.
Suppose does not intersect the interior of . We will construct a rank- matrix arbitrarily close to with . For cone , let denote the dual cone, which consists of all linear functionals that are nonnegative on , and similarly let be the dual cone of . Neither the cone nor contains a line since after a change of coordinates each are a subspace intersected with a positive orthant. Therefore we can choose functionals and in the interiors of and respectively. The functional has the property that for any non-zero , , and similarly for with respect to .
Let be the matrix with in every row, and the matrix with in every column. Choose vectors and in the interiors of and respectively. Let and for chosen small enough so that and are still in the interiors of and respectively. Then contains the point given by copies of and copies of that is in .
Let be any non-zero point on the boundary of , so either for some row of and column of or for some row of and column of . Without loss of generality assume the first case. Letting denote the th row of we have because . This implies is outside of the cone . Since intersects the interior of but not its boundary, it must be contained in the interior of . Since does not intersect the interior of or the origin, it does not intersect . Therefore has .
Note that , which can be made arbitrarily close to in 2-norm by choosing small enough. For sufficiently small , and have full rank since this is an open condition, so . ∎
The set intersects the interior of if and only if contains a nonempty subset that is open in the Euclidean subspace topology on (or equivalently the Zariski closure of is ).
First we show that the set is not contained in any facet hyperplane of . Every facet of is defined by a linear equation strictly involving either the first set of coordinates or the second set. Without loss of generality let be a facet hyperplane defined on the first set of coordinates. Recall that is the graph of the inverse function on matrices. Its projection to the first set of coordinates is a Zariski open set: the complement of the hypersurface defined by the vanishing of the determinant. So then hyperplane is either contained in the determinant hypersurface, in which case and do not intersect, or it intersects in dimension , in which case and intersect transversely as well.
Suppose an open neighborhood of is contained in . If the neighborhood is contained in the boundary of then is contained in the hyperplane of one of the facets since is irreducible. As shown above, this cannot happen so there must be a point on in the interior of . Conversely, if is non-empty then clearly it is open in the subspace topology on . ∎
Suppose , and that is a nonnegative factorization. The point corresponds to the factorization . To understand the possible boundary components of sets of matrices with rank and nonnegative rank equal to , it is sufficient to understand the ways that and can intersect in a neighborhood of . It is not true that if and are disjoint in a neighborhood of , then is on the boundary of ; they may intersect elsewhere. However, Lemma 3.3 demonstrates we can always construct that has as a submatrix, is on the boundary, and for which agrees with in a neighborhood of .
Let denote the tangent space of at the point . We divide the situations in which intersects but not into three broad cases based on how and intersect and on how itself intersects in a neighborhood of . The first case is when has dimension , which is the minimum possible dimension. This case we study in Section 4. The second case is when has strictly larger dimension, and this intersection is contained in . The third case is when does not contain . These two cases are studied in Sections 5.2 and 5.1 respectively.
3 Locally rigid factorizations
In the rest of the paper, we will adopt terminology from the rigidity theory of frameworks . We use the following translation: Dimensions and nonnegative rank of a matrix correspond to a graph, values of the entries of the matrix correspond to edge lengths of the graph and a nonnegative factorization corresponds to a configuration of the vertices of the graph.
A nonnegative factorization is locally rigid if a neighborhood of in has dimension , the minimal possible dimension.
In other words the only nonnegative factorizations near are the trivial ones obtained by scaling the columns of (and rows of ).
By [10, Corollary 4.4], a matrix with positive entries lies on the boundary of if and only if for every nonnegative factorization of the matrix the corresponding geometric configuration satisfies that (i) every vertex of the intermediate triangle lies on an edge of the outer polygon, (ii) every edge of the intermediate triangle contains a vertex of the inner polygon, (iii) a vertex of the intermediate triangle coincides with a vertex of the outer polygon or an edge of the intermediate triangle contains an edge of the inner polygon. Such geometric configurations are isolated for fixed inner and outer polygons, hence the corresponding nonnegative factorizations are locally rigid. Thus if lies on the boundary of , then all its nonnegative factorizations are locally rigid.
In the rest of the section we will show that locally rigid factorizations can be modified so that they lie on the boundary of the set of matrices of rank and nonnegative rank at most .
Let be a nonnegative factorization. For small enough, there exists obtained from by adding at most strictly positive rows and obtained from by adding at most stricly positive columns such that any nonnegative factorization of is in the -neighborhood of for some permutation matrix .
Consider the geometric configuration of cones in corresponding to the factorization . Since is a nonnegative factorization, the intermediate cone is spanned by the unit vectors. We add strictly positive rows to that correspond to hyperplanes at most distance from the facets of the intermediate cone. We add strictly positive columns to that correspond to points that are at most distance from the vertices of the intermediate cone. Neither of these operations changes incidence relations between the three cones. The new outer cone is contained in times larger copy of the intermediate cone and the new inner cone contains a times smaller copy of the intermediate cone. For small enough, there exists such that the only other cones with rays that one can be nested between a larger and smaller copy of the intermediate cone give factorizations that are in the -neighborhood of . ∎
If there is a neighborhood of a factorization such that all factorizations in the neighborhood are on the boundary of , then by adding at most strictly positive rows to and at most strictly positive columns to , one can get a matrix that is on the boundary.
Given a locally rigid nonnegative factorization , then by adding at most strictly positive rows to and at most strictly positive columns to , one can get a matrix that is on the boundary.
However, we do not expect the converse to be true. In Section 5.2, we will see a matrix on the boundary that has factorizations that are not locally rigid.
4 Infinitesimally rigid factorizations
In this section, we study a subclass of locally rigid factorizations called infinitesimally rigid factorizations.
4.1 General theory
We first consider the tangent space of at , and how it intersects . The elements of the tangent space can be represented by parametrized curves in through given by for each . The first two terms of the Taylor expansion of such a curve are
Therefore the tangent space of at has a simple description as
Let denote the cone of tangent directions such that the line stays in for for some . For convenience, we will consider as its projection to the first factor.
The tangent directions along the diagonal matrices always lie in .
A nonnegative factorization is infinitesimally rigid if consists only of the diagonal matrices.
A infinitesimally rigid factorization has all non-trivial tangent directions blocked by the facets of . This implies that also has dimension only in a neighborhood of .
If nonnegative factorization is infinitesimally rigid, then it is locally rigid.
If has full dimension , then is in the interior of .
Cone is the subset of the tangent space that is cut out by the set of facets of that pass through . The facets of come from the rows of . The th column of is which is on the boundary of the halfspace defined by if or in other words if has a zero in the th entry. For each , let be the set of entries of that are zero. A vector in must have satisfying these inequalities, meaning that for each , the th column of , denoted must satisfy . Equivalently , where denotes entry-wise inner product on matrices.
On the other hand, a column of defines an inequality on the th row of , denoted , if and only if has a zero in the th coordinate. For each , let be the set of entries of that are zero. For , has inequality or equivalently .
We have described in terms of its facet inequalities, but it will often be easier to work with the dual cone . For let and for let . Then
The dimension of is equal to the codimension of the largest subspace contained in . As shown above, every functional in is zero along the diagonal.
The trivial components of the boundary of consist of matrices with a zero entry. Note that if has a zero in entry and is a nonnegative factorization of , then row of and column of have zeros in complementary positions so that . Since the support of is contained in the set of columns for which is zero, it is possible to form the outer product matrix as a nonnegative combination of the dual vectors coming from . Similarly, the matrix can be expressed as a nonnegative combination of the dual vectors coming from . Therefore contains a line, and so has less than full dimension. Therefore the factorizations of are not infinitesimally rigid. From here we will consider only matrices with strictly positive entries.
If is an infinitesimally rigid nonnegative factorization then
and have at least zeros in total and
for every distinct pair taken from , there must be a row of with a zero in position and not in position . Similarly for the columns of .
A nonnegative factorization being infinitesimally rigid is equivalent to . To express as the convex cone of a finite number of vectors requires at least vectors. The size of the generating set defining is equal to the total number of zeros in and .
The vectors coming from are nonnegative and the ones from are nonpositive. If , for each coordinate there must be at least one vector with a strictly positive value there, and one with a strictly negative value. To get a positive value in coordinate requires to have a row with zero in the th entry and a non-zero value in the th entry. Similarly for columns of . ∎
Continuing Example 3.2, a matrix with positive entries lies on the boundary of if and only if every nonnegative factorization of has up to permuting rows of , permuting columns of , simultaneously permuting columns of and rows of , and switching and the following form
The analysis of Mond, Smith and van Straten for translates into the following statement.
Proposition 4.7 (Lemma 4.3 of ).
A rank-3 matrix with positive entries is on the boundary of if and only if all nonnegative factorizations of are infinitesimally rigid.
Thus for rank 3, infinitesimally rigid factorizations characterize all non-trivial boundary components of . We will see in Section 5 that this is no longer the case for higher rank.
4.2 Properties of infinitesimally rigid factorizations
If is an infinitesimally rigid nonnegative factorization, then there is at least one zero in every column of and in every row of .
It follows directly from Theorem 4.5. ∎
If is an infinitesimally rigid nonnegative factorization with zeros, then there are at most zeros in every column of and in every row of .
If is infinitesimally rigid, then the dual cone is equal to the space of matrices with zero diagonal of dimension . The zeros of and correspond to the elements of a distinguished generating set of as described above. A generating set of size is minimal, so the only linear relation among the generators must be among all . If there were zeros in the same column of , then there would be generators of contained in a dimensional subspace, implying a smaller linear relation which is impossible. Similarly for the case of zeros in a row of . ∎
If is an infinitesimally rigid nonnegative factorization of a strictly positive matrix, then there are at most zeros in every row of and in every column of .
Since is strictly positive, no row of or column of can contain only zeros. If a row of contains zeros, then there has to be a row of that does not contain any zero, because otherwise would have a zero entry. This contradicts Lemma 4.8. ∎
Let be a locally rigid factorization. Let be a factorization that is obtained from by erasing all rows of and columns of that do not contain any zero entries. Then is locally rigid.
By contradiction, assume that every neighborhood of in contains a pair where is not diagonal. This implies that there is a row of with positive entries and a column of such that or there is a column of with positive entries and a row of such that . Let be the maximal entry of and ; let be the minimal non-zero entry of and . Consider the -neighborhood of where . For any in this neighborhood, every non-diagonal entry of is greater than and every diagonal entry is greater than . Since and are nonnegative, we have and similarly for all . ∎
Finally we also state a general result about the maximal number of zeros in a nonnegative factorization of any matrix in .
Any matrix in has a nonnegative factorization with at least zeros.
We can push every vertex of the intermediate simplex to the boundary of the outer polytope by choosing a point in the simplex and moving the vertices along the rays through the point until they meet the boundary of the outer polytope. Fix a vertex of the new simplex and push all the edges incident to the vertex inside until all facets incident to the vertex touch the inner polytope. Extend the edges of the simplex until the boundary of the outer polytope and take the convex hull. This configuration gives incidence relations and hence the same number of zeros in the corresponding factorization. ∎
4.3 Infinitesimally rigid factorizations for matrices of nonnegative rank four
In this subsection, we will present infinitesimally rigid factorizations for matrices with positive entries and of nonnegative rank four. In particular, we will show that for every zero pattern with zeros satisfying the conditions of Theorem 4.5, there exists an infinitesimally rigid nonnegative factorization realizing this zero pattern.
Constructing the zero patterns requires three different computations. Also, we consider zero patterns up to the action that permutes the rows of , simultaneously permutes the columns of and the rows of , permutes the rows of and transposes . As the first step, we use Macaulay2  to construct an orbit representative under this action for all zero patterns with zeros satisfying the conditions of Theorem 4.5. There are 15 such orbit representatives.
The second step is a computation in MATLAB: We fix a zero pattern and construct random nonnegative factorizations realizing this zero pattern by choosing non-zero entries uniformly at random between and . Then we numerically find a factorization of nonnegative rank four for each matrix using a program by Vandaele, Gillis, Glineur and Tuyttens . If the program does not find a factorization with specified accuracy or it finds a factorization with 13 zeros, then the factorization is a candidate to be an infinitesimally rigid factorization.
Finally, we use Normaliz  to check if these matrices are infinitesimally rigid based on Definition 4.1. For each of the zero patterns, we are able to construct an infinitesimally rigid realization (for some zero patterns, we had to construct additional random realizations to find infinitesimally rigid factorizations):