Constructing minimal phylogenetic networks from softwired clusters is fixed parameter tractable
Here we show that, given a set of clusters on a set of taxa , where , it is possible to determine in time whether there exists a level- network (i.e. a network where each biconnected component has reticulation number at most ) that represents all the clusters in in the softwired sense, and if so to construct such a network. This extends a polynomial time result from elusiveness (). By generalizing the concept of “level- generator” to general networks, we then extend this fixed parameter tractability result to the problem where refers not to the level but to the reticulation number of the whole network.
Keywords:Phylogenetics Fixed Parameter Tractability Directed Acyclic Graphs
1.1 Phylogenetic networks and softwired clusters
The traditional model for representing the evolution of a set of species (or, more abstractly, a set of taxa) is the rooted phylogenetic tree SempleSteel2003 (); MathEvPhyl (); reconstructingevolution (). Essentially, this is a singly-rooted tree where the leaves are bijectively labelled by and the edges are directed away from the root. In recent years there has been a growing interest in extending this model to also incorporate non-treelike evolutionary phenomena such as hybridizations, recombinations and horizontal gene transfers. This has subsequently stimulated research into rooted phylogenetic networks which generalize rooted phylogenetic trees by also permitting nodes with indegree two or higher, known as reticulation nodes, or simply reticulations. For detailed background information on phylogenetic networks we refer the reader to HusonRuppScornavacca10 (); Nakhleh2009ProbSolv (); Semple2007 (); husonetalgalled2009 (); twotrees (); surveycombinatorial2011 (). Figure 1 shows an example of a rooted phylogenetic network.
We are interested in the following biologically-motivated optimization problem. We are given a set of clusters on , where a cluster is simply a strict subset of . We wish to construct a phylogenetic network that “represents” all the clusters in such that the amount of reticulation in the network is “minimized”. There are several different definitions of “represents” and “minimized” present in the literature. In this article we will consider only the softwired definition of “represents” husonetalgalled2009 (); cass (); HusonRuppScornavacca10 (); surveycombinatorial2011 (). Most of our formal definitions will be deferred to the preliminaries. Nevertheless, it is helpful to already formally state that a rooted phylogenetic tree on represents a cluster if contains an edge such that is exactly equal to the subset of reachable from by directed paths. A phylogenetic network on , on the other hand, represents a cluster in the softwired sense if there exists some rooted phylogenetic tree on such that represents and is topologically embedded inside . Regarding “minimized”, we consider two closely related, but subtly different, variants of minimality. The first variant, reticulation number minimization, aims at minimizing the total number of reticulation nodes in the network111This is the definition when all reticulation vertices have indegree-2, for more general networks reticulation number is defined slightly differently. See the Preliminaries for more information.. The second, less well-known variant, level minimization JanssonSung2006 (); JanssonEtAl2006 (); lev2TCBB (); reflections (); tohabib2009 (), asks us to minimize the maximum number of reticulation nodes contained in any “tangled” region of the network, which essentially correspond to the non-trivial biconnected components of the underlying undirected graph (see Figure 1). The reticulation number is a global optimality criterion, while the level is a local optimality criterion. In general minimizing for one variant does not induce minimum solutions for the other variant (see e.g. Figure 3 of husonetalgalled2009 ()), although the algorithmic techniques used to tackle these problems are often related elusiveness ().
Both these problems are NP-hard and APX-hard bordewich (); twotrees (). This raises the natural question: is it NP-hard to minimize the reticulation number (respectively, the level) if the number of reticulation nodes in the network (respectively, per tangled region) is fixed? Prior to this article there were only partial answers known to these questions. In elusiveness () it was proven that level-minimization is polynomial-time solvable if the level is fixed. A striking aspect of this proof is that the running time of the algorithm is only polynomial time in a highly theoretical sense: it is too high to be of any practical interest. This exorbitant running time has two causes. Firstly, the exhaustive enumeration of all generators lev2TCBB (), essentially the set of all possible underlying topologies of a network if the taxa are ignored. Secondly, after determining the correct generator, a second wave of exhaustive enumeration determines where a critical subset of should be located within the network, after which all remaining elements of can easily be added without much computational effort.
The question of whether a corresponding positive result would hold for reticulation number minimization was left open, although the emergence of several partial results and practically efficient algorithms husonetalgalled2009 (); elusiveness () suggested that this might well be the case. Furthermore, it was not obvious how the algorithm from elusiveness () could be adapted to yield a fixed parameter tractable algorithm for level minimization – where the parameter is the level of the network – since appears as an exponent of in the running time of the algorithm. (We refer to Flum2006 (); niedermeier2006 (); downey1999 (); Gramm2008 () for an introduction to fixed parameter tractability). Curiously, the main problem is not the enumeration of the generators, because the number of generators is independent of Gambette2009structure (), but the allocation of the critical initial subset of taxa to their correct location in the network.
In this article we settle all these questions by proving for the first time that both level minimization and reticulation number minimization are fixed parameter tractable (where, in the case of reticulation number minimization, the parameter is the reticulation number of the whole network). We give one algorithm for level minimization and one algorithm for reticulation minimization, although the two algorithms have a large common core. The algorithms again rely heavily on generators, which we extend here to also be useful in the context of reticulation number minimization; generators had hitherto only appeared in the level minimization literature. In both algorithms the major non-triviality is showing how the network structure can still be adequately recovered if the parameter is no longer allowed to appear in the exponent of as it was in elusiveness ().
1.2 Beyond softwired clusters: the wider context
We believe that this approach is significant beyond the softwired cluster literature. Other articles discuss the problem of constructing rooted phylogenetic networks not by combining clusters but by combining triplets simplicityAlgorithmica (); reflections (), characters gusfielddecomp2007 (); gusfield2 (); WuG08 (); myers2003 () or entire phylogenetic trees into a network. These models are in general mutually distinct although they do have a significant common overlap which reaches its peak in the case of data derived from two phylogenetic trees. To see this, note that if one takes the union of clusters represented by a set of two or more phylogenetic trees, then the reticulation number (or level) required to represent these clusters is in general less than or equal to the reticulation number (or level) required to topologically embed the trees themselves in the network, and this inequality is often strict. However, in the case of a set comprising exactly two trees the inequality becomes equality twotrees (). Hence for data obtained from two trees one could solve the reticulation number minimization and level minimization problems for clusters by using algorithms developed for the problem of topologically embedding the trees themselves into a network. These algorithms are highly efficient and fixed parameter tractable in a practical, as opposed to solely theoretical sense bordewich2 (); sempbordfpt2007 (); quantifyingreticulation (); whiddenWABI (). However, these tree algorithms do not help us with more general cluster sets, because for more than two trees the optima of the cluster and tree models start to diverge. Indeed, the cluster model often saves reticulations with respect to the tree model by weakening the concept of “above” and “below” in the network, which is exactly why the input tree topologies do not generally survive if one atomizes them into their constituent clusters twotrees (). Moreover, the literature on embedding three or more trees into a network is not yet mature, with articles restricting themselves to preliminary explorations pirnISMB2010 (); huynh (). It therefore seems plausible that the generator approach might be adapted to the tree model (or the other constructive methods mentioned) to yield a unified technique for producing positive complexity results for reticulation number minimization and level minimization, even in the case of many input trees (or data obtained from many input trees).
Consider a set of taxa , where . A rooted phylogenetic network (on ), henceforth network, is a directed acyclic graph with a single node with indegree zero (the root), no nodes with both indegree and outdegree equal to 1, and nodes with outdegree zero (the leaves) bijectively labeled by . In this article we usually identify the leaves with . The indegree of a node is denoted and is called a reticulation if , otherwise is a tree node. An edge is called a reticulation edge if its target node is a reticulation and is called a tree edge otherwise. When counting reticulations in a network, we count reticulations with more than two incoming edges more than once because, biologically, these reticulations represent several reticulate evolutionary events. Therefore, we formally define the reticulation number of a network as
A rooted phylogenetic tree on , henceforth tree, is simply a network that has reticulation number zero. We say that a network on displays a tree if can be obtained from by performing a series of node and edge deletions and eventually by suppressing nodes with both indegree and outdegree equal to 1, see Figure 2 for an example. We assume without loss of generality that each reticulation has outdegree at least one. Consequently, each leaf has indegree one. We say that a network is binary if every reticulation node has indegree 2 and outdegree 1 and every tree node that is not a leaf has outdegree 2.
Proper subsets of are called clusters, and a cluster is a singleton if . We say that an edge of a tree represents a cluster if is the set of leaf descendants of . A tree represents a cluster if it contains an edge that represents . It is well-known that the set of clusters represented by a tree is a laminar family, often called a hierarchy in the phylogenetics literature, and uniquely defines that tree. We say that represents “in the softwired sense” if displays some tree on such that represents , see Figures 2 and 3. In this article we only consider the softwired notion of cluster representation and henceforth assume this implicitly222Alternatively, we say that a network N represents a cluster “in the hardwired sense” if there exists a tree edge of such that is the set of leaf descendants of v.. A network represents a set of clusters if it represents every cluster in (and possibly more). The set of all softwired clusters represented by a network can be obtained as follows. For a network , we say that a switching of is obtained by, for each reticulation node, deleting all but one of its incoming edges. Given a network and a switching of , we say that an edge of represents a cluster w.r.t. if is an edge of and is the set of leaf descendants of in . The set of all softwired clusters represented by , denoted , is the set of clusters represented by all edges of w.r.t. , where ranges over all possible switchings HusonRuppScornavacca10 (). Note that the set of all possible switchings of coincides with the set of all trees displayed by . It is also natural to define that an edge of represents a cluster if there exists some switching of such that represents w.r.t . Note that, in general, an edge of might represent multiple clusters, and a cluster might be represented by multiple edges of .
Given a set of clusters on , throughout the article we assume that, for any taxon , contains at least one cluster containing . For a set of clusters on we define as , we sometimes refer to this as the reticulation number of . The related concept of level requires some more background. A directed acyclic graph is connected (also called “weakly connected”) if there is an undirected path (ignoring edge orientations) between each pair of nodes. A node (edge) of a directed graph is called a cut-node (cut-edge) if its removal disconnects the graph. A directed graph is biconnected if it contains no cut-nodes. A biconnected subgraph of a directed graph is said to be a biconnected component if there is no biconnected subgraph of that contains . A phylogenetic network is said to be a network if each biconnected component has reticulation number less than or equal to .333Note that to determine the reticulation number of a biconnected component, the indegree of each node is computed using only edges belonging to this biconnected component. A network is called simple if the removal of a cut-node or a cut-edge creates two or more connected components of which at most one is non-trivial (i.e. contains at least one edge). A (simple) level- network is called a (simple) level- network if the maximum reticulation number among the biconnected components of is precisely . For example, the network in Figure 1 is a level-2 network (which is not simple), the network in Figure 3 is a simple level-4 network and the network in Figure 3 is a simple level-2 network. Note that a tree is a level-0 network. For a set of clusters on we define , the level of , as the smallest such that there exists a level- network that represents . It is immediate that for every cluster set it holds that , because a level- network always contains at least one biconnected component containing reticulations.
We say that two clusters are compatible
if either or or , and incompatible
otherwise. Consider a set of clusters .
The incompatibility graph
of is the undirected graph that has node set
and edge set
and are incompatible
clusters in }. We say that a set of taxa is compatible with if every cluster is compatible with , and incompatible otherwise.
We say that a set of clusters on is separating if it is incompatible with all sets of taxa such that and .
When we write we mean “some function that only depends on ”. For simplicity we overload to refer to multiple different functions with this property. We write to mean “some function that is polynomial in ”, where . As in the case of , we often overload this expression. Indeed, the goal of this article is not to derive exact expressions for the running time, but to show that it is bounded above by . It should be noted that the that we encounter in this article can be extremely exponential in . Also, can in general be exponentially large as a function of , but (as we shall see in due course) it is reasonable to assume that is bounded above by when the parameter (reticulation number or level) is fixed.
The next lemma ensures that, if our goal is to find a network representing a set of clusters and minimizing the level or the reticulation number, we can restrict our attention to binary networks:
elusiveness () Let be a network on . Then we can transform N into a binary network such that has the same reticulation number and level as and all clusters represented by are also represented by .
Thanks to Lemma 1, we may assume that there exists a binary network with reticulation number (or with level if we are interested in level minimization) that represents . We henceforth restrict our analysis to binary networks and, except in places where it might cause confusion to not be explicit, we will not emphasize again that we only deal with this kind of network.
3 Minimizing level is fixed parameter tractable
The aim of this section is to show that level-minimization is fixed parameter tractable. To compute , we will repeatedly query, “Is ? If so, construct a network with level equal to that represents ” for , where starts at 0 and is incremented by 1 until the query is answered positively. Assuming that the queries are correctly answered, this process will terminate after iterations. Hence, to prove an overall running time of , it is sufficient to show that for each we can correctly answer the query in time at most . Note that if and only if all the clusters in are pairwise mutually compatible, which can be easily checked in time , so we henceforth assume that .
The high-level idea is the following. In cass (); HusonRuppScornavacca10 () it is shown that level- networks can be constructed using a divide and conquer strategy. Informally, the idea is to construct a level- network for each connected component of and then to combine these into a single network. The clusters in each connected component first have to be processed, which creates (for each component) a separating set of clusters. From Lemma 1 of elusiveness (), we know that, if a level- network representing a separating set of clusters on exists, a simple level- network representing has to exist. This network will never have two or more taxa with the same parent elusiveness (). The transformation underpinning Lemma 1 furthermore allows us to assume that this simple level- network is binary. Hence, the divide and conquer strategy essentially reduces to constructing binary simple level- networks for separating sets of clusters (and then combining them into a single network).
In Section 3.1 we show how to construct a simple level- network in time from a separating set of clusters. Subsequently we show in Section 3.2 how to combine these networks in time into a single level- network.
3.1 Constructing simple networks from separating cluster sets
Before proving the main result of this paper, we need to prove some preliminary results.
Given a simple level- network and a set of clusters on , checking whether is represented by can be done in time , where .
Note that there are at most trees displayed by and each tree represents at most clusters. This means that is at most . Since cannot represent if , checking whether takes at most time. ∎
Thus, if , since is assumed to be separating, it is not possible that and we can immediately answer “no” to the query. We thus
henceforth assume that i.e. that contains at most clusters.
If all the leaves of a binary simple level- network are removed and all nodes with both indegree and outdegree equal to 1 are deleted, the resulting structure is called a level- generator as defined in lev2TCBB (). See Figure 4 for the level-1 and level-2 generators. The number of level- generators is bounded by Gambette2009structure ()444Note that the number of level- generators grows rapidly in , lying between and Gambette2009structure ()..
The sides of a level- generator are defined as the union of its edges (the edge sides) and its nodes of indegree-2 and outdegree-0 (the node sides). The number of sides in a generator is bounded by , because the sum of its vertices and edges is linear in reflections ().
The set (for ) is defined as the set of all networks that can be constructed by choosing some level- generator and then applying the following leaf hanging transformation to such that each taxon of appears exactly once in the resulting network. (This is essentially identical to the definition given in reflections (), which is only a superficial refinement of the definition given in lev2TCBB ()).
First, for each pair of vertices in connected by a single edge , replace by a path with internal vertices and, for each such internal vertex , add a new leaf , an edge , and label with some taxon from . All the taxa added in this way are “on side ” where is the side corresponding to the edge . (It is also permitted that the path has zero internal nodes i.e. that the side remains empty).
Second, for each pair of vertices in connected by two edges, treat the two edges as in step 1, but ensure that at least one of the two paths does not have zero internal nodes.
Third, for each vertex of with indegree 2 and outdegree 0 add a new leaf , an edge and label with a taxon ; we say “taxon is on side ” where is the side corresponding to vertex .
The main reason for step 2 in Definition 1 is to ensure that multi-edges in generators do not survive in the final network, because our definition of phylogenetic network does not allow multi-edges. The following lemma follows directly from the results in Section 3.1 of lev2TCBB ():
The set (for ) is equal to the set of all binary simple level- networks.
By Lemma 1 of elusiveness () and Lemmas 1 and 2, we have the following:
Let be a separating set of clusters on , such that . Then there exists a network in such that represents .
Given a taxon set , we call any network resulting from adding all taxa in to sides of a generator (in the sense of Definition 1) a completion of on . Here we call a side that receives taxa a long side, a side that receives 1 taxon a short side and a side that receives 0 taxa an empty side. Figure 3 is thus a completion of generator , where sides and are empty, side is short, and sides are long. Note that node sides (such as in the example) are always short, but not all short sides are node sides i.e. edge sides can be short too.
Given a generator , we call a set of side guesses for , denoted by , a set of guesses about the type of each side of (i.e. whether it is empty, short, or long). A completion of on respects if all sides that are long in receive at least 2 taxa in , sides that are short in one taxon and empty sides zero taxa. Then we have the following result:
Searching in the space of all binary simple level- networks on is equivalent to searching in the space of all completions of a level- generator respecting a set of side guesses , iterating overall all sets of side guesses for a generator and all level- generators.
Let be a level- generator and let be a set of side guesses for . We say that the pair is side-minimal w.r.t. a separating cluster set on and , if there exists a completion of on respecting that is a level- network representing and, amongst all simple level- networks that represent , has a minimum number of long sides, and (to further break ties) amongst those networks it has a minimum number of short sides.
We define an incomplete network as a generator , a set of side guesses , a set of finished sides (i.e. those sides for which we have already decided that no more taxa will placed on them), a set of future sides (i.e. those short and long sides that have had no taxa allocated yet) and at most one long side on which at least one taxon has already been placed but where we might still want to add some more taxa. We call this the active side. A valid completion of an incomplete network is an assignment of the unallocated taxa to the future sides and (possibly) above the taxa already placed on the active side, that respects and such that the resulting network (which we call the result of the valid completion) represents . Informally, the result of a valid completion is any network on respecting and representing that is obtained by respecting all placements of taxa made thus far.
For example, consider again the network in Figure 3. Let be the network in that figure and let be the
network obtained from by deleting taxa and suppressing the resulting vertices with indegree and outdegree both equal to 1. Let be generator 2a,
and let be the set of side-guesses where sides and are empty, side is short, and sides are long. Sides
are finished, is a future side and is the active side. In particular, we can perform a valid completion of this incomplete network by putting
taxa and above taxon on side and then putting on side . In this case, is the result of the completion, although in general an incomplete network
might have many valid completions, or none.
Given a cluster set , we write if and only if every non-singleton cluster in containing , also contains . Then we have the following result.
Given a separating set of clusters on and an ordered set of distinct taxa of such that and for . Then .
If for and , this means that the set is compatible with . Since , we have a contradiction. ∎
The following observations will be useful to prove Lemma 3.
Let be a phylogenetic network on representing a set of clusters on constructed by choosing some level- generator and then applying the leaf hanging transformation described in Definition 1 to . If two taxa and in are on the same side of the generator underlying and the parent of is a descendant of the parent of , then .
Given a simple phylogenetic network , we say that a side is reachable from a side in if there is a directed path in the generator underlying from the head of side to the tail of side .
Let be a phylogenetic network on representing a set of clusters on constructed by choosing some level- generator and then applying the leaf hanging transformation described in Definition 1 to . Moreover, let and be two taxa of on the same side of the generator underlying such that and let be a taxon on a side such that is not reachable from and . Then we have that .
Since , we know that every non-singleton cluster that contains also contains . Now, let such a cluster. is represented by some tree displayed by , so some edge in is such that is the set of all taxa reachable from directed paths from the head of . Now, and are both in , so there a directed path from the head of to and a directed path from the head of to . Since is not reachable from , the only way that such a directed path can reach is via the parent of , hence the fact that . If no cluster containing and exists, since we have that the only cluster containing is the singleton cluster . Then, obviously, too. ∎
Let be a phylogenetic network on representing a set of clusters on constructed by choosing some level- generator and then applying the leaf hanging transformation described in Definition 1 to . Let and be two taxa in on the same side of the generator underlying such that there exists an edge from the parent of to the parent of . Then represents all clusters in containing but not .
Let be a separating cluster set on . Then every size-2 subset of is incompatible with .
Let be a simple phylogenetic network , a taxon and a side of the generator underlying . We denote by the following operation, where we exclude the case
from consideration where is a short side that already has a taxon on it. If is a short side, then is simply the network obtained by putting on side (in the sense of Definition 1).
Otherwise, is a long side, and then is the network obtained by placing “just above” the highest taxon on side . If there are not yet any taxa on side then
we simply let be the first taxon on side . (See Figure 5 for clarification).
We are now ready to analyse Algorithm 1, which is a critical subroutine. Let us assume that we have an incomplete network with an active side (which is by definition long) such that all long sides that are reachable from , are finished. These preconditions will be motivated in due course. Informally, Algorithm 1 lets us decide whether we should continue adding taxa to the top of the active side, or stop and declare it finished. (In fact, the algorithm is rather more complicated than that, because a side-effect of the algorithm is that it sometimes adds taxa to unfinished short sides, irrespective of whether it has chosen to add a taxon to the top of the active side). Algorithm 2 will repeatedly call Algorithm 1 until it finally declares the active side finished. This is all ultimately driven by the main algorithm, Algorithm 3, which - amongst other tasks - then identifies and initialises (i.e. places a first taxon at the bottom of) a new active side.
Let be a separating set of clusters on and let be the first integer for which a level- network representing exists. Let be an incomplete network such that its underlying generator and set of side guesses are such that is side-minimal w.r.t. and , and let be an active side of . Then, if a valid completion for exists, Algorithm 1 computes a set of (incomplete) networks such that this set contains at least one network for which a valid completion exists.
Recall that, from Corollary 1, we can restrict our search to networks in . We write to denote the set of taxa present in a (incomplete) network . For a set of clusters on and a subset , we define the restriction of to as . We start the proof by analyzing the case when (see Algorithm 1 for the definition of , , , etc).
Suppose . If then there are two possibilities. If then clearly no taxon can be placed directly above on , because that would mean , and thus , contradiction. Hence the only correct move is to declare that the side is finished and return . If then, since , we have that, for every there exists some such that and . Clearly the relation is not allowed to create cycles in , because otherwise the set of taxa in the cycle would form a cluster compatible with (see Proposition 2). Suppose we start at an arbitrary taxon in and perform a non-repeating walk on the taxa of by following the relation. Given that is of finite size and this walk cannot visit a taxon of that it has already visited earlier in the walk (thus creating a cycle), we will find a taxon such that there is no such that and , meaning that , contradiction. So the case that but , cannot actually happen. Now, consider the case that . Algorithm 1 will always end the side and return in this case. Indeed, no valid completion of can have some taxon that has not yet been allocated above on side . Suppose this is not true. Clearly, from Observation 2, , so . In this case, all taxa in are either equal to , or underneath and above . Indeed, let be a taxon in and suppose, for the sake of contradiction, that is above on side or on another side . If is above on side , then from Observation 2 we have that . If is on another side , the fact that implies that there is no room under side so, by Observation 3 we have that . Thus, in both cases (i.e. if is above on side or on a different side ) we have that , meaning that , contradiction. We can hence conclude that each taxon in is either equal to , or underneath and above in any completion of where is on . But, however one arranges two or more taxa on one side, at least one taxon will imply another taxon in the sense of the relation. More formally, in any case there exist two taxa and in such that and . This implies that , contradiction. This concludes the correctness of the case .
We now consider the case when . Let be the only taxon in . In this case, Algorithm 1 will return if . Indeed, no valid completion of exists where one or more taxa are placed above on . Suppose this is wrong. In that case, observe that in every valid completion always has to be the taxon directly above . Indeed, if there was some valid completion such that is not directly above , then there would exist some taxon such that (from Observation 2) and (as before, this follows from the fact that and from Observations 2 and 3). This would mean that , contradiction. So we assume that is directly above . Now, since , then there is some cluster in the input that contains , does not contain , and contains some not-yet allocated taxon distinct from . From Observation 4, the only edge that can represent such a cluster is the edge between the parents of and . But all the clusters represented by consist only of already-allocated taxa, because . This means that adding on side will only lead us to construct non-valid completions. Hence we conclude that, if , all valid completions of do not contain any other taxon on and ending the side is the right choice.
Now consider the case and let be restricted to . If does not represent we are definitely correct to declare the side as finished and return . Indeed, all valid completions of N do not contain any other taxon on . Suppose it is not correct. Then there exists a valid completion of where at least one taxon is above on . Again, for the same reasons as above we assume that is always the taxon directly above . Since does not represent , this incompatibility cannot be eliminated by adding more taxa, hence we conclude that there are no valid completions of with taxa above on side . Hence, ending the side is the only correct option. Suppose now that does represent ; Algorithm 1 adds above on side , and does not declare as finished. This conclusion can only be incorrect if all valid completions require that is not directly above . We observe that in any valid completion of there can be no taxon directly above on , because otherwise, as before, since we will have that and hence , contradiction. So all valid completions terminate the side at . Let be an arbitrary valid completion of and denote by the network obtained from by moving , wherever it is, just above . Firstly, we claim that still represents . Recall that , so the only potential problem is with clusters in that contain but do not contain . Let be such a cluster not represented by . Suppose . But in this case we would have in , contradiction. So the only possibility is that . Clearly was in and was thus represented by . Moreover, from Observation 4, the edge that represents in is the edge between the parents of and . Given that , no more taxa can be added “underneath” side and this edge still represents in because is a valid completion of . Hence moving in the way is safe in terms of cluster representation.
Secondly, we claim that moving in this way does not alter the side types i.e. the empty/short/long sides before moving remain empty/short/long after moving . To see this, note that moving from its original location reduces the number of taxa by 1 on some side, and increases the number of taxa of by 1. Side is by assumption already long, so remains so. The side of containing cannot change from being long to being short in , because this lowers the total number of long sides, and by assumption the pair underlying is side-minimal. Similarly it cannot change from being short to being empty, because this leaves the number of long sides the same but reduces the number of short sides, again contradicting the assumption that is side-minimal. Combining these two claims - that moving is safe for cluster representation and does not alter the side types nor the underlying generator - let us conclude that there is a valid completion for in which is placed directly above . Hence it is correct to add above on side , and does not declare as finished.
The case is identical to the corresponding subcase when . This means that in this case it is always correct to declare the side as finished and return .
Consider now the case . Observe firstly that, if some taxon is placed directly above , then all remaining taxa in must be allocated to sides in . To see why this is, note that for every we have that . So, if is placed above on or on a side not in , then, from Observation 2 and 3 we would have that , contradicting the fact that is in . We only need to show that, if a valid completion for exists, then the set contains a network for which there exists a valid completion. Note that contains (line 20) , where is declared as finished, (lines 21-23) all possible networks obtained from by allocating all taxa in to sides in and (lines 24-27) all possible networks obtained from by allocating all taxa in to sides in , iterating over all . The only case that these three sets do not describe, is when every valid completion has a taxon directly above , but at least one taxon is not mapped to . But this implies, similarly to the case , that , so , contradiction. Hence this case cannot happen, and the three sets actually describe all possible outcomes in this situation. So at least one of them will contain a network with a valid completion in the case does have a valid completion.
Consider now the case . We begin with the subcase . Similar to previous arguments we know that, if we place (the only element in ) directly above , all taxa in have to be allocated to . This holds because, from Observation 4, any cluster that contains but not is represented by the edge between the parents of and . If , the set is composed of (line 33) , where is declared as finished, (lines 34-35) all possible networks obtained from by allocating to a side in and (lines 36-38) all possible networks obtained from by allocating all taxa in to sides in . Observe that the only situation that these three guesses do not describe, is when some taxon is placed above and is not mapped to . But in this case we would have that , contradicting the fact that is in . So does again describe all possible outcomes.
This leaves us with the very last subcase, and . The subcase when does not represent restricted to is actually fairly straightforward. It is clear that cannot be placed in this position in a valid completion. Hence the only two situations that line 41 and lines 42-43 do not describe, is when some element is placed directly above , and is not mapped to . But, as before, this implies that , which as we have seen is not possible. So the only remaining subcase is when , and does represent restricted to . Now, consider the network . Informally the dummy taxa in act as “placeholders” for taxa that will only later in the algorithm be mapped to . We do not know exactly what these taxa will be, but we know that they will definitely be there. Consider a cluster . If does not represent then this must be because of the dummy taxa, because we know that did represent restricted to . Note that this holds irrespective of the true identity of the dummy taxa. Hence, will never be represented by any completion of . For this reason we conclude that, if does not represent , it is definitely correct to declare the side finished (line 49) or allocate to a side in (lines 50-51).
Finally, suppose does represent . This is the flip-side of the previous argument. Whatever the true identity of the dummy taxa, every valid completion of will represent every cluster in . Let be an arbitrary valid completion of and denote by the network obtained from by moving , wherever it is, just above . Now, as we did earlier we argue that in this case it is “safe” to put directly above . Indeed, because , the only clusters that might not be represented in are clusters in . But we have shown that when is placed directly above all the clusters in are represented regardless of how we complete the rest of the network. Secondly, we argue just as before that moving in this way cannot alter the side types. So if we choose as there must still exist a valid completion. This concludes the proof of the lemma. ∎
Let be a separating set of clusters on and let be the first integer for which a level- network representing exists. Let be an incomplete network such that its underlying generator and set of side guesses are such that is side-minimal w.r.t. and , and let be an active side of that contains only a single taxon. Then, if a valid completion for exists, Algorithm 2 computes in time a set of (incomplete) networks for which is a finished side, such that contains at least one network for which there exists a valid completion.
The correctness follows from Lemma 3. We now prove the running time.
First, note that the size of the set returned by Algorithm 1 is bounded by . This is evident for the sets constructed on lines 34-35, 42-43 and 50-51 but it holds also for the sets constructed respectively on lines 21-23, 24-27 and 36-38, since these sets are constructed only if, respectively, , or . Since in all other cases , the size of the set returned by Algorithm 1 is indeed bounded by . Moreover, from Proposition 1, it follows that the running time of Algorithm 1 is .
Second, note that, each time that Algorithm 1 returns a set of networks such that , decreases or is declared as finished. Additionally, when and , Algorithm 1 returns only one network per call and we have at most of these calls (because either is declared finished or a new taxon is added to ).
Since the number of sides in a generator is bounded by and is a subset of the short sides of the generator (which follows from the fact that all long sides reachable from are assumed to be finished), we have that is bounded by . Thus, the running time of Algorithm 2 is bounded by . ∎
We will subsequently use the term lowest side to denote a long side that does not yet have all its taxa (i.e. an unfinished long side), and such that there is no other long side with this property that is reachable from .
The following lemma is basically the fixed parameter tractable version of Lemma 3 from elusiveness ():
Let be a separating set of clusters on . Then, for every fixed , Algorithm 3 determines whether a level- network exists that represents , and if so, constructs such a network in time .
Algorithm 3 starts by choosing a level- generator and a set of side guesses . Note that (see lines 1-2) generators and sets of guesses are analyzed in such a way that generators with a smaller number of sides and sets of side guesses with a smaller number of long sides, and (to further break ties) short sides, are analyzed first (this is the meaning of the expression “in increasing side order”). This implies that the side-minimal pair , if any exists, is analyzed before any other pair for which a valid completion exists. This is done to be able to apply Lemma 4.
Then (see lines 4-18), the algorithm constructs a set of complete networks, i.e. simple level- networks where each short side has received a single taxon and each long side at least two, and returns the first of them that represents , if any exists. On line 6, denotes the set of all taxa in that are candidates to be the first taxon on side , which we call . We will discuss this set in more detail shortly.
Note that lines and of the algorithm are only a technical step. Indeed, when we declare a side as finished, we assume that we will never alter that side again. Hence it does not change the analysis if we collapse all the taxa on side into a single meta-taxon. That is, if we have decided that the taxa on the side are we simply replace all these taxa by a single new taxon and replace by in any clusters in that they appear in (line 10). This collapsing step is simply a convenience to ensure that the set of sides reachable from the current lowest side are always empty or short sides. This will be helpful when proving the running time of Algorithm 3, see below. When we are finished allocating all the taxa in and are ready to check whether the resulting final network represents we can simply de-collapse all the i.e. “unfold” all the long sides that we have collapsed (line 14). This means that the correctness of Algorithm 3 follows by Observation 1 and Lemma 4.
We now need to prove the correctness of the running time. First, note that the number of pairs to consider is bounded by since both the number of generators and the number of sides per generator are bounded by .
We now need to prove that the size of is at most for all sides i.e. that the number of taxa that might be the first taxon on side , is not too big. So let be any taxon which can fulfil this role, and let be the taxon directly above on side . (The taxon must exist because we assume that is long). Clearly, . By line 10, we have that the only sides reachable from side are short and empty sides. Moreover, we know from Observation 5 that, because is separating, there is some non-singleton cluster such that but . By Observation 4, such a cluster has to be represented by the edge between the parents of and . Now, any cluster represented by can only contain taxa that are reachable from by a directed path. The only sides that are reachable from side are short and empty sides, so the cluster can only contain at most taxa (because there are at most short sides). So we know that is in some cluster , and that is “small” in the sense that its size is bounded above by . So if we take all “small” clusters, and let be their union, we know that we could simply try taking every element in and guessing that it is equal to . To ensure that we do not use too many guesses, we have to show that is bounded by . To see that this holds, consider the question: how many clusters in contain at most taxa for a constant ? Observe that on every long side only the taxa furthest away from the root are potentially in such clusters. Any taxon closer to the root on a long side cannot possibly be in a cluster of size at most , because if it is in a cluster then so are at least other taxa too. Hence there are at most taxa that can be involved in “small” clusters: the taxa on the short sides and the taxa at the bottom of the long sides. So we have that is bounded by and we can guess with at most guesses.
The collapsing and de-collapsing steps (lines 10 and 14) can be done in time, as well as completing each side (line 7), by Lemma 4. Moreover, by Proposition 1, also checking whether represents takes time. Additionally, the allocation of remaining taxa to the unfinished short sides (line 15) takes a time bounded by . Indeed, if we have correctly guessed all the taxa on the long sides, then there will be at most taxa left that have not yet been assigned to a side, and these will correspond to taxa on (a subset of) the short sides. (Some of the short sides might already have been allocated taxa by Algorithm 1).
This means that, if the size of is bounded by , then the entire algorithm can be executed in time. And this is indeed the case, since and (summing over all iterations) the total number of lowest sides are bounded by and each time that a side is completed, the number of unfinished long sides decreases by 1.
3.2 From simple networks to general networks
To prove the fixed parameter tractability of constructing general level- networks, we need to introduce a few other concepts. The most important is the concept of a decomposable network.
Let be a set of clusters on a taxon set with incompatibility graph and let be a phylogenetic network that represents . is said to be decomposable w.r.t. if and only if there exists a cluster-to-edge mapping such that, for any two clusters , and lie in the same connected component of if and only if the two tree edges and that represent and are contained in the same biconnected component of .
The following observation is straightforward:
Let be a set of clusters on a taxon set with incompatibility graph and let be a phylogenetic network representing that is decomposable w.r.t. . Then, we have that the number of biconnected components of is equal to the number of connected components of