Similar Elements and Metric Labeling on Complete Graphs

# Similar Elements and Metric Labeling on Complete Graphs

Pedro F. Felzenszwalb
Brown University
Providence, RI, USA
pff@brown.edu

We consider a problem that involves finding similar elements in a collection of sets. The problem is motivated by applications in machine learning and pattern recognition (see, e.g. [2]). Intuitively we would like to discover something in common among a collection of sets, even when the sets have empty intersection. A solution involves selecting an element from each set such that the selected elements are close to each other under an appropriate metric. We formulate an optimization problem that captures this notion and give an efficient approximation algorithm that finds a solution within a factor of 2 of the optimal solution.

The similar elements problem is a special case of the metric labeling problem defined in [1] and we also give an efficient 2-approximation algorithm for the metric labeling problem on complete graphs. Metric labeling on complete graphs generalizes the similar elements problem to include costs for selecting elements in each set.

Beyond producing solutions with good theoretical guarantees, the algorithms described here are also practical. The algorithm for the similar elements problem has been implemented and used to find similar objects in a collection of photographs [3].

## 1 Similar Elements

Let be a (possibly infinite) set and be a metric on . Let be finite subsets of . The goal of the similar elements problem is to select an element from each set such that the selected elements are close to each other under the metric . One motivation is for discovering something in common among the sets even when they have empty intersection.

We formalize the problem as the minimization of the sum of pairwise distances among selected elements. Let with . Define the similar elements objective as,

 c(x)=∑1≤i,j≤nd(xi,xj). (1)

Let be an optimal solution for the similar elements problem.

Optimizing appears to be difficult, but we can define easier problems if we ignore some of the pairwise distances in the objective. In particular we define different “star-graph” objective functions as follows. For each define the objective to account only for the terms in involving ,

 cr(x)=∑j≠rd(xr,xj). (2)

Let be an optimal solution for the optimization problem defined by . We can compute efficiently using a simple form of dynamic programming, by first computing and then computing for .

 xrr=argminxr∈Sr∑j≠rminxj∈Sjd(xr,xj), (3)
 xrj=argminxj∈Sjd(xrr,xj). (4)

Each of the “star-graph” objective functions leads to a possible solution. We then select from among the solutions by evaluating the original objective,

 ^x=argmin1≤r≤nc(xr). (5)
###### Theorem 1.

The algorithm described above finds a 2-approximate solution to the similar elements problem. That is,

 c(^x)≤2c(x∗).
###### Proof.

Since the minimum of a set of values is at most the average,

 c(^x)=min1≤r≤nc(xr)≤1nn∑r=1c(xr).

By the triangle inequality we have

 c(x)=∑1≤i,j≤nd(xi,xj)≤∑1≤i,j≤n(d(xi,xr)+d(xr,xj))=2nn∑l=1d(xr,xl)=2ncr(x).

Therefore

 c(^x)≤2n∑r=1cr(xr).

Note that following (3) and (4),

 cr(xr)=minxr∈Sr∑j≠rminxj∈Sjd(xr,xj).

And therefore

 n∑r=1cr(xr)=n∑r=1minxr∈Sr∑j≠rminxj∈Sjd(xr,xj)≤minx∑1≤r,j≤nd(xr,xj)=c(x∗).

We conclude . ∎

To analyze the running time of the approximation algorithm we assume the distances between pairs of elements in are either pre-computed and given as part of the input, or they can each be computed in time.

Let . The first stage of the algorithm involves optimization problems that can be solved in time each. The second stage of the algorithm involves evaluating for solutions, and takes time.

###### Remark 2.

If each of the sets has size at most the running time of the approximation algorithm for the similar elements problem is .

## 2 Metric Labeling on Complete Graphs

Let be an undirected simple graph on nodes . Let be a finite set of labels with and be a metric on . For let be a non-negative function mapping labels to real values. The unweighted metric labeling problem on is to find a labeling minimizing

 c(x)=∑i∈Vmi(xi)+∑{i,j}∈Ed(xi,xj). (6)

Let . This optimization problem can be solved in polynomial time using dynamic programming if is a tree. Here we consider the case when is the complete graph and give an efficient 2-approximation algorithm based on the solution of several metric labeling problems on star graphs.

For each define a different objective function, , corresponding to a metric labeling problem on a star graph with vertex set rooted at ,

 cr(x)=∑i∈Vmi(xi)n+∑j∈V∖{r}d(xr,xj)2. (7)

Let . We can solve this optimization problem in time using a simple form of dynamic programming. First compute an optimal label for the root vertex using one step of dynamic programming,

 xrr=argminxr∈L⎛⎝mr(xr)n+∑j∈V∖{r}minxj∈L(mj(xj)n+d(xr,xj)2)⎞⎠. (8)

Then compute for ,

 xrj=argminxj∈L(mj(xj)n+d(xrr,xj)2). (9)

Optimizing each separately leads to possible solutions . We select one of them using the original metric labeling objective on the complete graph,

 ^x=argminr∈Vc(xr). (10)
###### Theorem 3.

The algorithm described above finds a 2-approximate solution to the metric labeling problem on a complete graph. That is,

 c(^x)≤2c(x∗).
###### Proof.

Since the minimum of a set of values is at most the average,

 c(^x)=minr∈Vc(xr)≤1n∑r∈Vc(xr).

Since is a metric and is non-negative,

 c(x) = ∑i∈Vmi(xi)+∑{i,j}∈Ed(xi,xj) = ∑i∈Vmi(xi)+∑(i,j)∈V2d(xi,xj)2 ≤ ∑i∈Vmi(xi)+∑(i,j)∈V2(d(xi,xr)2+d(xr,xj)2) = ∑i∈Vmi(xi)+2n∑l∈V∖{r}d(xr,xl)2 ≤ 2n∑i∈Vmi(xi)n+2n∑l∈V∖{r}d(xr,xl)2 = 2ncr(x).

Therefore

 c(^x)≤2∑r∈Vcr(xr).

Note that following (8) and (9),

 cr(xr)=minxr∈L⎛⎝mr(xr)n+∑j∈V∖{r}minxj∈L(mj(xj)n+d(xr,xj)2)⎞⎠.

And therefore

 ∑r∈Vcr(xr) = ∑r∈Vminxr∈L⎛⎝mr(xr)n+∑j∈V∖{r}minxj∈L(mj(xj)n+d(xr,xj)2)⎞⎠ ≤ minx∑r∈V⎛⎝mr(xr)n+∑j∈V∖{r}(mj(xj)n+d(xr,xj)2)⎞⎠ = minx⎛⎝n∑r∈Vmr(xr)n+∑r∈V∑j∈V∖{r}d(xr,xj)2⎞⎠ = minx⎛⎜⎝∑i∈Vmi(xi)+∑(i,j)∈V2d(xi,xj)2⎞⎟⎠ = c(x∗).

We conclude . ∎

The first stage of the algorithm involves optimization problems that can be solved in time each. The second stage of the algorithm involves evaluating solutions, and takes time.

###### Remark 4.

The running time of the approximation algorithm for the metric labeling problem on complete graphs is .

### Acknowledgments

We thank Caroline Klivans, Sarah Sachs, Robert Kleinberg and Yang Yuan for helpful discussions about the contents of this report. This material is based upon work supported by the National Science Foundation under Grant No. 1447413.

## References

• [1] Jon Kleinberg and Eva Tardos. Approximation algorithms for classification problems with pairwise relationships: Metric labeling and markov random fields. Journal of the ACM, 49(5):616–639, 2002.
• [2] Oded Maron and Aparna Lakshmi Ratan. Multiple-instance learning for natural scene classification. In International Conference on Machine Learning, volume 98, pages 341–349, 1998.
• [3] Sarah Sachs. Similar-part approximation using invariant feature descriptors. Undergraduate Honors Thesis, Brown University, 2016.
You are adding the first comment!
How to quickly get a good reply:
• Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
• Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
• Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters