Equivalence Problems for Tree Transducers: A Brief Survey
Abstract
The decidability of equivalence for three important classes of tree transducers is discussed. Each class can be obtained as a natural restriction of deterministic macro tree transducers (mtts): (1) no context parameters, i.e., topdown tree transducers, (2) linear size increase, i.e., mso definable tree transducers, and (3) monadic input and output ranked alphabets. For the full class of mtts, decidability of equivalence remains a longstanding open problem.
1 Introduction
The macro tree transducer (mtt) was invented independently by Engelfriet [21, 30] and Courcelle [14, 15] (see also [38]). As a model of syntaxdirected translations, mtts generalize the attribute grammars of Knuth [47]. Note that one (annoying) issue of attribute grammars is that they can be circular; mtts always terminate. Macro tree transducers are a combination of contextfree tree grammars, invented by Rounds [58] and also known as “macro tree grammars” [35], and the topdown tree transducer of Rounds and Thatcher [59, 68]: the derivation of the grammar is (topdown) controlled by a given input tree. In terms of a topdown transducer, the combination is obtained by allowing nesting of state calls in the rules (similar to the nesting of nonterminals in the productions of contextfree tree grammars). Topdown tree transducers generalize to trees the finite state (string) transducers (also known as “generalized sequential machines”, or gsms, see [39, 9]). In terms of formal languages, compositions of mtts give rise to a large hierarchy of string languages containing, e.g., the io and oi hierarchies (at level one they include the indexed languages of Aho [2]), see [23]. mtts can be applied in many scenarios, e.g., to type check xml transformations (they can simulate the pebble transducers of Milo, Suciu, and Vianu [52]), see [24, 50, 51], or to efficiently implement streaming XQuery transformations [43, 53]. In terms of functional programs, mtts are particularly simple programs that do primitive recursion over an input tree and only produce trees as output. Applications in programming languages exist include [69, 70, 53].
Equivalence of nondeterministic transducers is undecidable, already for restricted string transducers [41]. We therefore only consider deterministic transducers. What is known about the equivalence problem for deterministic macro tree transducers? Unfortunately not much in the general case. Only a few subcases are known to be decidable. Here we describe three of them:

topdown tree transducers

linear size increase transducers

monadic tree transducers.
The first one was solved long ago by Esik [31], but was revived through the “earliest canonical normal form” by Engelfriet, Maneth, and Seidl [27]. The latter implies Ptime equivalence check for total topdown tree transducers. The second one is solved by the decidability of equivalence for deterministic mso tree transducers of Engelfriet and Maneth [26]. The same authors have shown that every mtt of linear size increase is effectively equal to an mso transducer [25]. Hence, decidability of equivalence follows for mtts of linear size increase. Here we give a direct proof using mtts. The third result is about mtts over monadic trees. These are string transducers with copying. The decidability of their equivalence problem follows through a relationship with Lsystems [29]; in particular, with the sequence equivalence problem of hdt0l systems. The latter was first proved decidable by Culik II and Karumhäki [19].
2 Preliminaries
We deal with finite, ordered, ranked trees. In such a tree, each node is labeled by a symbol from a ranked alphabet such that the rank of the symbol is equal to the number of children of the node. Formally, a ranked alphabet consists of a finite set together with a mapping associating to each symbol its rank. We write to denote that the rank of is equal to . By we denote the subset of symbols of that have rank . Let be a ranked alphabet. The set of all trees over , denoted , is the smallest set of strings such that if , , and , then also . For a tree of the form we simply write . For a set , we denote by the set of all trees over such that the rank of each is zero. Let be distinct symbols in and let such that none of the leaves in are labeled by for . Then by we denote the tree substitution that replaces each leaf labeled by the tree . Thus denotes the tree .
Let for some ranked alphabet . We denote the nodes of by their Dewey dotted decimal path and define the set of nodes of as if with , , and . Thus, denotes the root node, and denotes the th child of the node . For a node we denote by its label, and, for a tree we denote by the tree obtained from by replacing its subtree rooted at by the tree . The size of , denoted , is its number of nodes. The height of , denoted , is defined as for , , and .
We fix the set of input variables and the set of formal context parameters . For we define and .
A deterministic finitestate bottomup tree automaton is a tuple where is a finite set of states, is the set of final states, and for every and , is a function from to . The transition function is extended to a mapping from to in the obvious way, and the set of trees accepted by is .
3 Macro Tree Transducers
Definition 1
A (deterministic) macro tree transducer is a tuple such that is a ranked alphabet of states with , and are ranked alphabets of input and output symbols, respectively, is the initial state, and for every , , and , , the set of rules contains at most one rule of the form
where such that if a node in has its label in , then its first child has its label in . A rule as above is called a rule and its righthand side is denoted by .
The translation (often just denoted ) realized by an mtt is a partial function recursively defined as follows. For each state of rank , is the translation of starting in state , i.e., “starting” with a tree where . For instance, for , simply equals . In general, for an input tree , is obtained from by repeatedly replacing a subtree of the form with by the tree . The latter tree will also be written . We define .
By the definition above, all our mtts are deterministic and we refer to them as “macro tree transducers” and denote their class of translations by mtt. An mtt is total if there is exactly one rule of the above form. If each state of an mtt is of rank one, then it is a topdown tree transducer. The class of translations realized by (deterministic) topdown tree transducers is denoted by t. In Lemmas 3 and 5 we make use of nondeterministic topdown tree transducers and mark the corresponding class there by the letter “N”. A transducer is nondeterministic if there are several rules in . An mtt is monadic if its input and output ranked alphabets are monadic, i.e., only contain symbols of rank and . An mtt is of linear size increase if there is a constant such that for every .
As an example, consider the transducer consisting of the rules in Figure 1.
The alphabets of this transducer are , , and . The mtt translates a binary tree into the same tree, but additionally adds under each leaf the reverse (Dewey) path of the node. For instance, is translated into as can be verified in this computation of
Note that is not of linear size increase. Hence, it is not MSO definable. Since the translation is neither topdown nor monadic, it falls into a class of mtts for which we do not know a procedure to decide equivalence. As an exercise, the reader may wonder whether there is an mtt that is similar to , but outputs Dewey paths below leaves (instead of their reverses). In contrast, consider input trees with only exactly one leaf (and all other leaves labeled differently). Then an mtt of linear size increase can output under the unique leaf its reverse Dewey path, i.e., this translation is mso definable. Is it now possible with an mtt to output the nonreversed Dewey path?
One of the most useful properties of mtts is the effective preservation of regular tree languages by their inverses. This is used inside the proofs of several results presented here. For instance, to prove that for every mtt there is an equivalent one which is nondeleting in the parameters (= strict), one can use the above property as follows: given a state of rank and a subset , the language is regular, and hence is the regular set of inputs for which outputs only parameters in the set . Thus, by using regular lookahead we can determine which parameters are used, and can change the rules to call an appropriate state which is only provided the parameters in which it uses. Regular lookahead is explained in Section 4.1.
The following result is stated in Theorem 7.4 of [30]. A similar proof as below is given at the end of [24]. A slightly simpler proof, for a slightly larger class, is presented by Perst and Seidl for macro forest transducers [55].
Lemma 1
Let be an mtt with output alphabet and let be a regular tree language (given by a bottomup tree automaton ). Then is effectively regular. In particular, the domain is effectively regular.
Proof. Let and . We construct the new automaton . The states of are mappings that associate with each state a mapping . The set consists of all such that . For we define such that for every and , . Here, is the extension of to trees in by the rule for all . Now let with . For we define where for every and , . Now, is the extension of of above to symbols by for every .
3.1 Bounded Balance
Many algorithms for deciding equivalence of transducers are based on the notion of bounded balance. Intuitively, two transducers have bounded balance if the difference of their outputs on any “partial” input is bounded by a constant. For instance, for two d0l systems with , Culik II defines [17] the balance of a string as the difference of the lengths of and . He shows that equivalence is decidable for d0l systems that have bounded difference. In a subsequent article [18], Culik II and Fris show that any two equivalent d0l systems in normal form have bounded difference, thus giving the first solution to the famous d0l equivalence problem.
For a tree transducer a partial input is an input tree which contains exactly on distinguished leaf labeled , and is a fresh symbol not in . More precisely, a partial input is a tree where is in the domain of and is a node of . Since the transducer has no rules for , the computation on the input “blocks” at the labeled node. Thus, the tree may contain subtrees of the form where is a state of rank . For instance, consider the transducer shown in the left of Figure 2 and consider the partial input tree . Then
It should be clear that if we replace each by so that is a tree with , then we obtain the output of on input tree . For instance, we may pick above; then we obtain which indeed equals .
Consider the trees and for two mtts and . What is the balance of these two trees? There are two natural notions of balance: either we compare the sizes of , or we compare their heights. Given two trees we define their sizebalance (for short, sbalance) as and their heightbalance (for short, hbalance) as . Two transducers have bounded sbalance (resp. hbalance) if there exists a such that for any partial input the sbalance (resp. hbalance) of and is at most . Obviously, bounded hbalance implies bounded sbalance, but not vice versa.
Let and be equivalent mtts. Do and have bounded sizebalance? To see that this is in general not the case, it suffices to consider two simple topdown tree transducers: translates a monadic partial input of the form into the full binary tree of height containing occurrences of the subtree . The transducer translates into the full binary tree of height with occurrences of the subtree .
The rules of and are shown in Figure 2. Clearly, and are of bounded heightbalance (with constant ). But, their sizebalance is not bounded.
In a similar way it can be seen that equivalent mtts need not have bounded heightbalance. This even holds for monadic mtts: Consider the transducers and with rules shown
in Figure 3. Their heightbalance on input is equal to ; for instance, for the tree we obtain
which are trees of height and , respectively. We may verify that replacing by results in equal trees:
And for we obtain that
Let us now show that equivalent topdown tree transducers have bounded heightbalance.
Lemma 2
Equivalent topdown tree transducers effectively have bounded heightbalance.
Proof. Let be equivalent topdown tree transducers with sets of states , respectively. Note that they have the same domain . Let and . Consider the two trees for . Let be a smallest input tree such that . It should be clear that the height of is bounded by some constant . In fact, let be the height of a smallest tree in the set , for any subsets and . Since is in such a set, its height is at most . This bound can be computed because by Lemma 1 the sets and are effectively regular, and regular tree languages are effectively closed under intersection [12]. In fact, it is not difficult to see that we can choose . Hence, there is a constant such that for any appearing in . Clearly we can take , where is the maximal height of the righthand side of any rule of and . This means that because and the substitutions increase the height of by at most .
If the transducers of Lemma 2 are total, then and is the maximal size of the righthand side of any rule for an input leaf symbol, i.e., .
Let us consider an example of two equivalent topdown tree transducers and with
output paths of different height. The rules of and are given in Figure 4. Let us consider the input tree . We omit some parentheses in monadic input trees. We obtain
Similarly, for the transducer we obtain
As the reader may verify, if is replaced by the leaf , then indeed the output trees and are the same, i.e., the transducers are equivalent. Let us compare the trees and . On the one hand, the transducer is “ahead” of the transducer in the output branch . It has already produced a node at that position, while has not (and is in state at that position). On the other hand, is ahead of at two other positions in the output: at the node the transducer has produced an node already, while at that node is in state , and, at node the transducer has output an node, while also here is in state .
4 Decidable Equivalence Problems
4.1 TopDown Tree Transducers
It was shown by Ésik [31] that the bounded heightdifference of topdown tree transducers can be used to decide equivalence.
Theorem 1
([31]) Equivalence of topdown tree transducers is decidable.
Proof. We follow the version of the proof given by Engelfriet [21].
Consider two equivalent topdown tree transducers and . By Lemma 2 they have bounded heightbalance by some constant . Consider the trees and . An “overlay” of these two trees is shown in Figure 5 (this is a copy of Figure 10 of [21]). At the node where is in state , the transducer has already produced the tree , i.e., at this node is “ahead” of by the amount . Similarly, at the labeled node, is ahead of by the amount . Clearly, the height of and is bounded by . Hence, there are only finitely many such trees and . We can construct a topdown tree automaton which in its states keeps track of all such “difference trees” and , while simulating the runs of and . It checks if the outputs are consistent, and rejects if either the outputs are different or if the height of a difference tree is too large. Finally, we check if accepts the language ; this is decidable because is regular by Lemma 1, and equivalence of regular tree languages is decidable (see [12]).
Note that Ésik [31] shows that even for singlevalued (i.e., functional) nondeterministic topdown tree transducers, equivalence is decidable. It is open whether or not equivalence is decidable for valued nondeterministic topdown tree transducers (but believed to be decidable along the same lines as for bottomup tree transducers [64], cf. the text below Theorem 5). A topdown tree transducer is lettertoletter if the righthand side of each rule contains exactly one output symbol in . It was shown by Andre and Bossut [7] that equivalence is decidable for nondeleting nondeterministic lettertoletter topdown tree transducers. An interesting generalization of Theorem 1 is given by Courcelle and FranchiZannettacci [16]. They show that equivalence is decidable for “separated” attribute grammars which are evaluated in two independent phases: first a phase that computes all inherited attributes, followed by a phase that computes all synthesized attributes (topdown tree transducers are the special case of synthesized attributes only).
Topdown Tree Transducers with Regular LookAhead. Regular lookahead means that the transducer (mtt or t) comes with a (complete) deterministic bottomup automaton (without final states), called the “lookahead automaton of ”. A rule of the lookahead transducer is of the form
and is applicable to an input tree only if the lookahead automaton recognizes in state for all . Given two topdown tree transducers with regular lookahead , we can transform them into ordinary transducers (without lookahead) such that the resulting transducers are equivalent if and only if the original ones are. This is done by changing the input alphabet so that for every original input symbol of rank , it now contains the symbols for all possible lookahead states of and of . Thus, for every , the new input alphabet has many symbols. It is easy to see that Lemma 2 also holds for transducers with lookahead, by additionally requiring that the domain of the is intersected with all input trees that represent correct runs of the lookahead automata. Thus, Theorem 1 also holds for topdown tree transducers with lookahead.
Theorem 2
Equivalence of topdown tree transducers with regular lookahead is decidable.
Canonical Normal Form. Consider two equivalent topdown tree transducers and let be their domain. As the example transducers and with rules in Figure 4 show, for a partial input tree , there may be positions in the output trees where is ahead of , and other positions where is ahead of . Such a scenario is also depicted in Figure 5.
We say that is earlier than , if for every and , the tree is a prefix of the tree . A tree is a prefix of a tree if for every with it holds that . The question arises whether for every topdown translation there is an equivalent unique earliest transducer such that for ; we call such a transducer a canonical transducer. The question was answered affirmative by Engelfriet, Maneth, and Seidl [27]. We only state this result for total transducers.
Theorem 3
Let be a total topdown tree transducer. An equivalent canonical transducer can be constructed in polynomial time.
Proof. The canonical transducers are topdown tree transducers without an initial state, but with an axiom tree . This means that the translation on input tree starts with the tree (instead of for ordinary transducers).
Starting with , we define its axiom . In a first step, an earliest transducer is constructed: if there is a state and an output symbol (of rank ) such that for every input symbol , then is not earliest. Intuitively, the symbol should be produced earlier, at each call of the state . Thus, the construction replaces in all righthand sides (and in the axiom ) by where the are new states. For every , is defined as the th subtree of the root of – beware, this righthand side may have changed due to the replacement above. Finally we remove and its rules. This step is repeated until it cannot be applied anymore. In this case has become earliest and no and exists such that for all input trees of . It should be clear that the earliest step can be carried out in polynomial time. In the second step, equivalent states are merged to obtain the canonical transducer; the corresponding equivalence relation on states is computed using fixed point iteration in cubic time (with respect to the size of ). It is computed in such a way that if , then .
Note that the availability of a canonical (“minimal”) transducer has many advantages. For instance, it makes possible to formulate a MyhillNerode like theorem which, in turn, makes possible Goldstyle learning of topdown tree transducers (in polynomial time), as shown by Lemay, Maneth, and Niehren [48].
Consider the two transducers and with the rules given in Figure 4. To construct a canonical equivalent transducer for according to Theorem 3, we observe that state of is not earliest: the root equals for the righthand sides of all rules. We replace by in the rule, and introduce the two rules and . We remove and have obtained an earliest transducer. The canonical transducer is constructed by realizing that the states and are equivalent and hence can be merged. The rules of the canonical transducer can thus be given as
To construct the canonical transducer for , we observe that state is not earliest: the root equals in both rules. We thus replace everywhere by where are new states. After this replacement, the current rule is:
Thus, new rules are and . The rules are and . The resulting transducer is earliest. We now compute that and that . We merge these pairs of states and obtain the same transducer (up to renaming of states) as the canonical one of above. Hence, and are equivalent.
As a consequence of Theorem 3 we obtain that equivalence of total topdown tree transducers can be decided in polynomial time.
Theorem 4
Equivalence of total topdown tree transducers can be decided in polynomial time.
The earliest normal form has also certain “disadvantages”. For instance, it does not preserve linearity (or nondeletingness) of the transducer. Consider for the rules and . When making earliest, these rules are removed and a rule such as is replaced by which is nonlinear. It is also deleting: .
4.2 BottomUp Tree Transducers
As a corollary of Theorem 2 we obtain that also for deterministic bottomup tree transducers, equivalence is decidable. This follows from the fact that every deterministic bottomup tree transducer can be transformed into an equivalent deterministic topdown tree transducer with regular lookahead [20].
Theorem 5
Equivalence of deterministic bottomup tree transducers is decidable.
Proof. A deterministic bottomup tree transducer is a tuple where is the set of final states and contains for every , , and at most one rule of the form where is a tree in . We construct in linear time a deterministic bottomup tree automaton which for every rule as above has the transition . This automaton serves as the lookahead automaton of a topdown tree transducer with the unique state . For a rule as above, the transducer has the rule
It should be clear that the resulting topdown tree transducer with lookahead (which has only the single state ) is equivalent to the given bottomup tree transducer .
The equivalence problem for bottomup tree transducers was first solved by Zachar [71]. It was shown by Seidl [61] that equivalence can be decided in polynomial time for singlevalued (i.e., functional) nondeterministic bottomup tree transducers. Note that this also follows from Theorem 8 and the (polynomial time) construction in the proof of Theorem 5. This result was extended to finitevalued nondeterministic bottomup tree transducers by Seidl [62]. For nondeterministic lettertoletter bottomup tree transducers, equivalence was shown decidable by Andre and Bossut [6]; such transducers contain exactly one output symbol in the righthand side of each rule. They reduce the problem to the equivalence of bottomup relabelings which was solved by Bozapalidis [10]. For deterministic bottomup tree transducers the effective existence of a canonical normal form, similar in spirit to the earliest normal form of topdown tree transducers, was shown by Friese, Seidl, and Maneth [37]. They show that this normal form can be constructed in polynomial time, if each state of the given transducer produces either none or infinitely many outputs; hence, equivalence is decidable in polynomial time for such transducers. Friese presents in her PhD thesis [36] a MyhillNerode theorem for bottomup tree transducers.
4.3 Linear Size Increase mtts
It was shown by Engelfriet and Maneth [25] that total deterministic mtts of linear size increase characterize the total deterministic mso definable tree translations. In fact, even any composition of total deterministic mtts, when restricted to linear size increase, is equal to an mso definable translation, as shown by Maneth [49]. The mso definable tree translations are a special instance of the mso definable graph translations, introduced by Courcelle and Engelfriet, see [13]. Decidability of equivalence for deterministic mso graphtostring translations on a contextfree graph language was proved by Engelfriet and Maneth [26]. It implies decidable equivalence also for mso tree translations. We present a proof of the latter here that only uses mtts and avoids going through mso.
The idea of the proof stems from Gurari’s proof [42] of the decidability of equivalence for 2dgsm. In a nutshell: the ranges of all the above translations are Parikh. A language is Parikh if its set of Parikh vectors is equal to the set of Parikh vectors of a regular language. Let be an alphabet. The Parikh vector of a string is the tuple of natural numbers such that for , equals the number of occurrences of in . For a language that is Parikh, it is decidable whether or not it contains a string with Parikh vector for some natural number . This property is used to prove equivalence as follows. Given two treetostring transducers we first change to produce a new end marker at the end of each output string. Then, given the regular domain language of and , and two distinct output letters we construct the Parikh language
Here denotes the th letter in the string . We now decide if there is an such that , using the fact that is Parikh. If such an exists, then the transducers are not equivalent. If, for all possible , , no such exists, then we know that the transducers are equivalent.
It was shown by Engelfriet, Rozenberg, and Slutzki in Corollary 3.2.7 of [29] that ranges of nondeterministic finitecopying topdown tree transducers with regular lookahead (for short, s) possess the Parikh property. The nondeterminism of this result is useful for defining the language , because we need to nondeterministically choose and positions and of the output strings. A nondeterministic topdown tree transducer is finitecopying if there is number such that for every and , the number of occurrences of states (more precisely, subtrees such that is a state of ) in the tree is . We denote the class of translations of nondeterministic finitecopying topdown tree transducers by .
For a tree we denote by its yield, i.e., the string of its leaf labels from left to right. For a class of tree translations we denote by the corresponding class of treetoyield translations. The treetoyield translations of topdown tree transducers can be obtained by topdown treetostring transducers which have strings over output symbols and state calls in the righthand sides of their rules. We repeat the argument given in [29]. By regt we denote the class of regular tree languages, i.e., those languages recognized by (deterministic) finitestate bottomup tree automata.
Lemma 3
Languages in are Parikh.
Proof. Let be a transducer and let . A topdown transducer is linear if no appears more than once in any of the righthand sides of its rules. We construct a linear transducer such that and the string is a permutation of the string , for every . The new transducer computes in its states the state sequences of , i.e., the sequence of states that are translating the current input node. Since is finitecopying, there effectively exists a bound on the length of the state sequences. For a new state