# An Optimal Algorithm for the Separating Common Tangents of two
Polygons^{1}^{1}1A preliminary version of this paper appeared at
SoCG 2015 [1].
In the case where the convex hulls of the polygons are not disjoint,
it is not clear that the algorithm for separating common tangents terminates
within the given bound on the running time.
Here, we give a correct algorithm and simplify the proof of correctness slightly.

###### Abstract

We describe an algorithm for computing the separating common tangents of two simple polygons using linear time and only constant workspace. A tangent of a polygon is a line touching the polygon such that all of the polygon lies to the same side of the line. A separating common tangent of two polygons is a tangent of both polygons where the polygons are lying on different sides of the tangent. Each polygon is given as a read-only array of its corners. If a separating common tangent does not exist, the algorithm reports that. Otherwise, two corners defining a separating common tangent are returned. The algorithm is simple and implies an optimal algorithm for deciding if the convex hulls of two polygons are disjoint or not. This was not known to be possible in linear time and constant workspace prior to this paper.

An outer common tangent is a tangent of both polygons where the polygons are on the same side of the tangent. In the case where the convex hulls of the polygons are disjoint, we give an algorithm for computing the outer common tangents in linear time using constant workspace.

## 1 Introduction

The problem of computing common tangents of two given polygons has received some attention in the case where the polygons are convex. For instance, it is necessary to compute outer common tangents of disjoint convex polygons in the classic divide-and-conquer algorithm for the convex hull of a set of points in the plane by Preparata and Hong [13]. They give a naÃ¯ve linear time algorithm for outer common tangents since that suffices for an time convex hull algorithm. The problem is also considered in various dynamic convex hull algorithms [6, 9, 12]. Overmars and van Leeuwen [12] give an time algorithm for computing an outer common tangent of two disjoint convex polygons when a separating line is known, where each polygon has at most corners. Kirkpatrick and Snoeyink [10] give an time algorithm for the same problem, but without using a separating line. Guibas et al. [8] give an lower bound on the time required to compute an outer common tangent of two intersecting convex polygons, even if it is known that they intersect in at most two points. They also describe an algorithm achieving that bound.

Touissaint [14] considers the problem of computing separating common tangents of convex polygons and notes that the problem occurs in problems related to visibility, collision avoidance, range fitting, etc. He gives a linear time algorithm. Guibas et al. [8] give an time algorithm for the same problem.

All the here mentioned works make use of the convexity of the polygons. If the polygons are not convex, one can use a linear time algorithm to compute the convex hulls before computing the tangents [7, 11]. However, if the polygons are given in read-only memory, it requires extra bits to store the convex hulls. In this paper, we also obtain linear time while using only constant workspace, i.e. bits. For the outer common tangents, we require the convex hulls of the polygons to be disjoint. There has been some recent interest in constant workspace algorithms for geometric problems, see for instance [2, 3, 4, 5].

The problem of computing separating common tangents is of special interest because these only exist when the convex hulls of the polygons are disjoint, and our algorithm detects if they are not. Thus, we also provide an optimal algorithm for deciding if the convex hulls of two polygons are disjoint or not. This was to the best of our knowledge not known to be possible in linear time and constant workspace prior to our work.

### 1.1 Notation and some basic definitions

Given two points and in the plane, the closed line segment with endpoints and is written . When , the line containing and which is infinite in both directions is written .

Define the dot product of two points and as , and let be the counterclockwise rotation of by the angle . Now, for three points , , and , we define , where sgn is the sign function. is if is to the left of the directed line from to , if , , and are collinear, and if is to the right of the directed line from to . We see that

We also note that if and are on the line and appear in the same order as and , i.e., , then for every point .

The left half-plane is the closed half plane with boundary lying to the left of directed line from to , i.e., all the points such that . The right half-plane is just .

Assume for the rest of this paper that and are two simple polygons in the plane with and corners, respectively, where is defined by its corners in clockwise or counterclockwise order, . Indices of the corners are considered modulo , so that and are the same corner when .

We assume that the corners are in general position in the sense that and have no common corners and the union of corners contains no three collinear corners.

A tangent of is a line such that and are not disjoint and such that is contained in one of the closed half-planes defined by . The line is a common tangent of and if it is a tangent of both and . A common tangent is an outer common tangent if and are on the same side of the tangent, and otherwise the tangent is separating. See Figure 1.

For a simple polygon , we let be the convex hull of . The following lemma is a well-known fact about .

###### Lemma 1.

For a simple polygon , is a convex polygon and the corners of appear in the same cyclic order as they do on .

The following lemma states folklore properties of tangents of polygons.

###### Lemma 2.

A line is a tangent of a polygon if and only if it is a tangent of . Under our general position assumptions, the following holds: If one of and is completely contained in the other, there are no outer common tangents of and . Otherwise, there are two or more. There are exactly two if and are disjoint. If and are not disjoint, there are no separating common tangents of and . Otherwise, there are exactly two.

## 2 Computing separating common tangents

In this section, we assume that the corners of and are both given in counterclockwise order. We prove that Algorithm 1 returns a pair of indices such that the line is a separating common tangent with contained in for . If the tangent does not exist, the algorithm returns NULL. The other separating common tangent can be found by a similar algorithm if the corners of the polygons are given in clockwise order and ‘’ is changed to ‘’ in line 1.

The algorithm traverses the polygons in parallel one corner at a time using the indices and . We say that the indices define a temporary line, which is the line . We update the indices and until the temporary line is the separating common tangent. At the beginning of an iteration of the loop at line 1, we traverse one corner of , . If the corner happens to be on the wrong side of the intermediate line, we make the temporary line pass through that corner by updating to and we reset to . The reason for resetting is that a corner of which was on the correct side of the old temporary line can be on the wrong side of the new line and thus needs be traversed again.

We show that if the temporary line is not a separating common tangent after each polygon has been traversed twice, then the convex hulls of the polygons are not disjoint. Therefore, if a corner is found to be on the wrong side of the temporary line when a polygon is traversed for the third time, no separating common tangent can exist and NULL is returned. Let be the value of after iterations, . We always have due to the initialization of . See Figure 2.

Assume that is updated in line 1 in iteration . The point is in the half-plane , but not on the line . Therefore, we have the following observation.

###### Observation 3.

When is updated, the temporary line is rotated counterclockwise around by an angle less than .

Assume in the following that the convex hulls of and are disjoint so that separating common tangents exist. Let be the indices that define the separating common tangent such that is contained in , i.e., is the result we are going to prove that the algorithm returns.

Since is convex, the temporary line always divides into two convex parts. If we follow the temporary line from in the direction towards , we enter at some point and thereafter leave again at some point . We clearly have if and only if the temporary line is a tangent to , since if and the line was no tangent, would only be a line segment. The part of the boundary of counterclockwise from to is in whereas the part from to is on . We therefore have the following observation.

###### Observation 4.

Let be the index of the corner of strictly after in counterclockwise order. There exists a corner of such that if and only if .

Let be the index of the first corner of when following in counterclockwise order from , . If is itself a corner of , we have . By Observation 4 we see that with equality if and only if . Let be when only line 1 has been executed. Consider now the value of after iterations. Let and add to until . This gives a non-decreasing sequence of indices of the first corner of in . Actually, we prove in the following that we need to add to at most once before . If we add to . Thus we have .

The following lemma intuitively says that the algorithm does not “jump over” the correct solution and it expresses the main idea in our proof of correctness.

###### Lemma 5.

###### Proof.

We prove the lemma for . From the definition of , we get that . Since the sequence is non-decreasing, the inequality is true for every .

Now, assume inductively that and consider what happens during iteration . If neither nor is updated, the statement is trivially true from the induction hypothesis, so assume that an update happens.

By the old temporary line we mean the temporary line defined by and the new temporary line is the one defined by . The old temporary line enters at some point and exits at some point when followed from . Likewise, let be the point where the new temporary line exits when followed from . The point exists since the convex hulls are disjoint.

Assume first that the variable in the algorithm is , i.e., a corner of the polygon is traversed. In this case .

We now prove . Assume that . The situation is depicted in Figure 3. In this case . Hence, the update happens when is traversed or earlier, so . Assume now that . We cannot have since , therefore . Consider the corner on following in counterclockwise order, . Due to the minimality of , we have . By Observation 4, . Therefore, must be updated when is traversed or earlier, so .

For the inequality , consider the new temporary line in the direction from to . We prove that is in the part of from counterclockwise to . The point is in the polygon defined by the segment together with the part of from counterclockwise to . Therefore, the new temporary line enters and exits . It cannot exit through the segment , since the old and new temporary lines intersect at , which is in . Therefore, must be on the part of from to . If is on the part of from counterclockwise to , then is on the part from to as we wanted.

Otherwise, assume for contradiction that the points appear in the order , , , counterclockwise along , where . The endpoints of the segment are on different sides of the tangent defined by , so the segment intersects the tangent at a point . The part of from to and the segments and form a simple polygon , see Figure 3 for an example. The new temporary line enters at the point , so it must leave after . The line cannot cross after since is convex. It also cannot cross the segment at a point after since the old and the new temporary line cross before , namely at . The tangent defined by and the new temporary line intersect before since the endpoints of the segment are on different sides of the tangent. Therefore, the line cannot cross the segment at a point after . Hence, the line cannot exit . That is a contradiction.

Therefore, is on the part of from to and hence the first corner of after must be before or coincident with , so that .

Assume now that in the beginning of iteration , i.e., a corner of the other polygon is traversed. In that case, we have , and we need only prove . Observation 3 gives that is in the part of from to , since the new temporary line is obtained by rotating the old temporary line counterclockwise around by an angle less than . That appears before on counterclockwise from follows from exactly the same arguments as in the case .

We have nowhere used the test at line 1 to conclude that . Hence, the test is never positive. This completes the proof. ∎

We are now ready to prove that Algorithm 1 has the desired properties.

###### Theorem 6.

If the polygons and have separating common tangents, Algorithm 1 returns a pair of indices defining a separating common tangent such that is contained in for . If no separating common tangents exist, the algorithm returns NULL. The algorithm runs in linear time and uses constant workspace.

###### Proof.

Assume first that the algorithm returns . We know that for each , since we never update to values as large as . Therefore, we have that for each and each . Hence the pair indeed defines the separating common tangent.

Assume now that there exists a separating common tangent. By Lemma 5, a pair is returned. As we already saw, this means that defines the separating common tangent.

If an update happens in iteration , the sum is increased by at least , where was the previous iteration where an update happened. Inductively, we see that when the final update of and happens, there has been at most iterations. After the final update, at most iterations follow. In total, the algorithm performs iterations. ∎

## 3 Computing outer common tangents

In this section, we assume that two polygons and are given such that their convex hulls are disjoint. We assume that the corners of are given in counterclockwise order and the corners of are given in clockwise order. We say that the orientation of and is counterclockwise and clockwise, respectively. We prove that Algorithm 2 returns two indices that define an outer common tangent such that and are both contained in .

As in the case of separating common tangents, we define as the value of after iterations of the loop at line 2 of Algorithm 2. See Figure 4. For this algorithm, we get a slightly different analogue to Observation 3:

###### Observation 7.

When is updated, the temporary line is rotated around in the orientation of by an angle less than .

Let be the point where the temporary line enters when followed from and the point where it exits . We have the following analogue of Observation 4.

###### Observation 8.

Let be the index of the corner of strictly after following the orientation of . There exists a corner of such that if and only if .

Let be the index of the first corner of after following the orientation of , where if is itself a corner of . By Observation 8, we have with equality if and only if . Define a non-decreasing sequence of the value of after iterations as we did for separating tangents. Also, let the indices define the outer common tangent that we want the algorithm to return such that . We can now state the analogue to Lemma 5 for outer common tangents.

###### Lemma 9.

After each iteration and for each we have

###### Proof.

Assume and the induction hypothesis . The inequality can be proven exactly as in the proof of Lemma 5. Therefore, consider the inequality and assume that an update happens in iteration .

Let the old temporary line and the new temporary line be the lines defined by the indices and , respectively. Let and be the points where the old temporary line enters and exits followed from , respectively, and let be the point where the new temporary line enters . The points and exist since the convex hulls of and are disjoint.

Assume first that the variable in the algorithm equals when the update happens. We prove that is in the part of from to following the orientation of , which is counterclockwise. The point is in the simple polygon bounded the part of from counterclockwise to and the segment . Therefore, the new temporary line must enter to get to . It cannot enter through , since the old and new temporary line cross at which is not in by assumption. Therefore, it must enter through the part of from to , so is in this part. If is not in the part of from to , it is clearly true that is in the part from to . Otherwise, assume for contradiction that the points appear on in the order and . Let be the half-line starting at following the tangent away from , and let be the half-line starting at following the old temporary line away from . The part of from to and the half-lines and define a possibly unbounded area outside , see Figure 5. We follow the new temporary line from towards . The point is not in and the new temporary line exits at since it enters at , so it must enter somewhere at a point on the segment . It cannot enter through since is convex. It cannot enter through since and are on the same side of the outer common tangent. It cannot enter through since the old and new temporary line intersect in , which is not in . That is a contradiction, so is on the part of from to . Hence, the first corner after is coincident with or before , i.e., .

Assume now that in the beginning of iteration so that a corner of the polygon is traversed. Observation 7 gives that is on the part of from counterclockwise to . It follows that appears before on counterclockwise from from exactly the same arguments as in the case . ∎

###### Lemma 10.

If or , then for some and some index .

###### Proof.

Assume that for , since otherwise, we are done. Likewise, assume that all of the part of from to is in . The part separates from in the set . Since the part of from to cannot cross or , it must exit and enter through points on when followed from , and hence the claim is true. ∎

We can now prove the stated properties of Algorithm 2.

###### Theorem 11.

If the polygons and have disjoint convex hulls, Algorithm 2 returns a pair of indices defining an outer common tangent such that and are contained in . The algorithm runs in linear time and uses constant workspace.

###### Proof.

Assume that the pair does not define the outer common tangent. By Lemma 10, an update of or happens when or is traversed or before. By Lemma 9, the algorithm does not terminate before and has been traversed. Hence, when the algorithm terminates, defines the outer common tangent.

Like in the proof of Theorem 6, we see inductively that when the final update of and happens, there has been at most iterations. After that, at most iterations follow. Hence, the algorithm terminates after at most iterations. ∎

## 4 Concluding Remarks

We have described an algorithm for computing the separating common tangents of two simple polygons in linear time using constant workspace. We have also described an algorithm for computing outer common tangents using linear time and constant workspace when the convex hulls of the polygons are disjoint. Figure 6 shows an example where Algorithm 2 does not work when applied to two disjoint polygons with overlapping convex hulls. In fact, if there was no bound on the values and in the loop at line 2, the algorithm would update and infinitely often and never find the correct tangent. An obvious improvement is to find an equally fast and space efficient algorithm which does not require the convex hulls to be disjoint. An algorithm for computing an outer common tangent of two polygons, when such one exists, also decides if one convex hull is completely contained in the other. Together with the algorithm for separating common tangents presented in Section 2, we would have an optimal algorithm for deciding the complete relationship between the convex hulls: if one is contained in the other, and if not, whether they are disjoint or not. However, keeping in mind that it is harder to compute an outer common tangent of intersecting convex polygons than of disjoint ones [8], it would not be surprising if it was also harder to compute an outer common tangent of general simple polygons than simple polygons with disjoint convex hulls when only constant workspace is available.

## Acknowledgments

We would like to thank Mathias Tejs Bæk Knudsen for pointing out the error in the algorithm for separating common tangents in the preliminary version of the paper [1].

## References

- [1] M. Abrahamsen. An optimal algorithm for the separating common tangents of two polygons. In 31st International Symposium on Computational Geometry (SoCG 2015), pages 198–208.
- [2] M. Abrahamsen. An optimal algorithm computing edge-to-edge visibility in a simple polygon. In Proceedings of the 25th Canadian Conference on Computational Geometry, CCCG, pages 157–162, 2013.
- [3] T. Asano, K. Buchin, M. Buchin, M. Korman, W. Mulzer, G. Rote, and A. Schulz. Memory-constrained algorithms for simple polygons. Computational Geometry: Theory and Applications, 46(8):959–969, 2013.
- [4] T. Asano, W. Mulzer, G. Rote, and Y. Wang. Constant-work-space algorithms for geometric problems. Journal of Computational Geometry, 2(1):46–68, 2011.
- [5] L. Barba, M. Korman, S. Langerman, and R.I. Silveira. Computing the visibility polygon using few variables. In Proceedings of the 22nd International Symposium on Algorithms and Computation, ISAAC, volume 7014 of Lecture Notes in Computer Science, pages 70–79. Springer, 2011.
- [6] G.S. Brodal and R. Jacob. Dynamic planar convex hull. In Proceedings of the 43rd annual IEEE Symposium on Foundations of Computer Science, FOCS, pages 617–626, 2002.
- [7] R.L. Graham and F.F. Yao. Finding the convex hull of a simple polygon. Journal of Algorithms, 4(4):324–331, 1983.
- [8] Leonidas Guibas, John Hershberger, and Jack Snoeyink. Compact interval trees: A data structure for convex hulls. International Journal of Computational Geometry & Applications, 1(1):1–22, 1991.
- [9] J. Hershberger and S. Suri. Applications of a semi-dynamic convex hull algorithm. BIT Numerical Mathematics, 32(2):249–267, 1992.
- [10] D. Kirkpatrick and J. Snoeyink. Computing common tangents without a separating line. In Proceedings of the 4th International Workshop on Algorithms and Data Structures, WADS, volume 955 of Lecture Notes in Computer Science, pages 183–193. Springer, 1995.
- [11] A.A. Melkman. On-line construction of the convex hull of a simple polyline. Information Processing Letters, 25(1):11–12, 1987.
- [12] M.H. Overmars and J. van Leeuwen. Maintenance of configurations in the plane. Journal of Computer and System Sciences, 23(2):166–204, 1981.
- [13] F.P. Preparata and S.J. Hong. Convex hulls of finite sets of points in two and three dimensions. Communications of the ACM, 20(2):87–93, 1977.
- [14] G.T. Toussaint. Solving geometric problems with the rotating calipers. In Proceedings of the IEEE Mediterranean Electrotechnical Conference, MELECON, pages A10.02/1–4, 1983.