Finding Distinct Subpalindromes Online
We exhibit an online algorithm finding all distinct palindromes inside a given string in time over an ordered alphabet and in time over an unordered alphabet. Using a reduction from a dictionary-like data structure, we prove the optimality of this algorithm in the comparison-based computation model.
Keywords: stringology, counting palindromes, subpalindromes, palindromic closure, online algorithm
Institute of Mathematics and Computer Science, Ural Federal University,
A palindrome is a string that is equal to its reversal. Palindromes are among the most interesting text regularities. During the last few decades, many algorithmic problems concerning palindromes were considered. In this paper we solve one problem that remained open.
There is a well known online algorithm by Manacher  that finds all maximal subpalindromes of a string in linear time and linear space (by a “subpalindrome” we mean a substring that is a palindrome). It is known  that every string of length contains at most distinct subpalindromes, including the empty string. The following question arises naturally: can one find all distinct subpalindromes of a string in linear time and space? In , this question was answered in the affirmative, but with an offline algorithm. The authors stated the existence of the corresponding online algorithm as an open problem. Our main contribution is the following result.
Let be a finite unordered (resp., ordered) alphabet. There exists an online algorithm which finds all distinct subpalindromes in a string over in (resp., ) time and linear space. This algorithm is optimal in the comparison based computation model.
As a by-product, we get an online linear time and space algorithm that finds, for all prefixes of a string, the lengths of their maximal suffix-palindromes and of their palindromic closures.
2 Notation and Definitions
An alphabet is a finite set of letters. A string over is a finite sequence of letters. It is convenient to consider a string as a function . A period of is any period of this function. The number is the length of , denoted by . We write for the -th letter of and abbreviate by . A substring of is any string such that for some and . Each occurrence of the substring in is determined by its position . If (resp. ), then is a prefix (resp. suffix) of . A prefix (resp. suffix) of a string is called proper if it is not equal to . The string is the reversal of , denoted by . A string is a palindrome if it coincides with its reversal. A palindrome of even (resp. odd) length is referred to as an even (resp. odd) palindrome. If a substring, a prefix or a suffix of a string is a palindrome, we call it a subpalindrome, a prefix-palindrome, or a suffix-palindrome, respectively. The palindromic closure of a string is the shortest palindrome such that is a prefix of .
Let be a subpalindrome of . The number is the center of , and the number is the radius of . Thus, a single letter and the empty string are palindromes of radius 0. Note that the center of the empty subpalindrome is the previous position of the string.
By an online algorithm for an algorithmic problem concerning strings we mean an algorithm that processes the input string sequentially from left to right, and answers the problem for each prefix of after processing the letter .
3 Distinct subpalindromes
3.1 Suffix-Palindromes and Palindromic Closure
The problem of finding the lengths of palindromic closures for all prefixes of a string is closely related to the problem of finding all distinct subpalindromes of this string. It was conjectured in  that there exists an online linear time algorithm for the former problem.
Let be the maximal suffix-palindrome of . It is easy to see that the palindromic closure of equals to the string . An offline algorithm for finding the maximal suffix-palindromes for each prefix of the string can be found, e. g., in [1, Ch. 8]. Our online algorithm is a modification of Manacher’s algorithm (see ).
We construct a data structure based on Manacher’s algorithm. Let be a boolean flag (needed to distinguish between odd and even palindromes). This data structure contains a string and supports the procedure adding a letter to the end of . The function returns the length of maximal odd/even (according to ) suffix-palindrome of .
Our data structure uses the following internal variables:
, which is the length of ;
, which is the center of the maximal odd/even (according to ) suffix-palindrome of ;
, which is an array of integers such that for any the value is equal to the radius of the maximal odd/even (according to ) subpalindrome with the center . The main property of is expressed in the following lemma (see [1, Lemma 8.1]).
Let be an integer, .
(1) If then ;
(2) if then .
At the beginning, is filled with zeros, , , , where is a special letter that does not appear in the input string111The strange-looking initial value of provides the correct processing of the first letter after (the while loop will be skipped and the correct values for the next iteration will be obtained)..
There exists an online linear time and space algorithm that finds the lengths of the maximal suffix-palindromes of all prefixes of a string.
From the correctness of Manacher’s algorithm (see ) and Lemma 3.1 it follows that the function correctly returns the length of the maximal odd/even suffix palindrome of the processed string. For a string of length , we call the procedure times with the parameter and times with . If one call of the procedure uses iterations of the loop in the lines 4–9, then the value of increases by . Hence, the loop is used at most times in total. Apart from this loop, performs a constant number of operations. This gives us the required time bound. ∎
There exists an online linear time and space algorithm that finds the lengths of palindromic closured of all prefixes of a string.
Let and consider the state of the data structure after the sequence of calls , .
The calls to after each call to return consequently the values for the case and for the case .
3.2 Distinct subpalindromes
We make use of the following
Lemma 3.2 ().
Each subpalindrome of a string is the maximal suffix-palindrome of some prefix of this string.
This lemma implies that the online algorithm designed in Sect. 3.1 finds all subpalindromes of a string. To find all distinct subpalindromes, we have to verify whether the maximal suffix-palindrome of a string has another occurrence in this string. Note that the direct comparison of substrings for this purpose leads to at least quadratic overall time. Instead, we will use a version of suffix tree known as Ukkonen’s tree. To introduce it, we need some definitions.
A trie is a rooted labelled tree in which every edge is labelled with a letter such that all edges leading from a vertex to its children have different labels. Each vertex of the trie is associated with the string labelling the path from the root to this vertex. A trie can be “compressed” as follows: any non-branching descending path is replaced by a single edge labelled by the string equal to the label of this path. The result of this procedure is called a compressed trie. For a set of strings, the compressed trie of is defined by the following two properties: (i) for each string of , there is a vertex associated it and (ii) the trie has the minimal number of vertices among all compressed tries with property (i).
A (compressed) suffix tree is the compressed trie of the set of all suffixes of a string. Ukkonen’s tree is the data structure containing a string and the suffix tree of this string (labels are stored as pairs of positions in the string). Ukkonen’s tree allows one to add a letter to the end of the string (procedure ), updating the suffix tree. We also need the following parameter: the length of the minimal suffix of the processed string such that this suffix occurs in this string only once (function ). Let us recall some implementation details of Ukkonen’s tree for the efficient implementation of .
The update of Ukkonen’s tree is based on the system of suffix links. Such a link connects a vertex associated with a word to the vertex associated with the longest proper suffix of . These links are also defined for “implicit” vertices (the vertices that are not in the compressed trie, but present in the corresponding trie). In particular, Ukkonen’s tree supports the triple such that
|(1)||is a vertex (associated with some string ) of the current suffix tree,|
|(2)||is an edge (labelled by some string ) between and its child,|
|(3)||is an integer between and ,|
with the property that is the longest suffix of the processed string that occurs in this string at least twice. This triple is crucial for fast update of Ukkonen’s tree (for further details, see ).
Lemma 3.3 ().
The procedure performs calls using space and (resp., ) time in the case of ordered (resp., unordered) alphabet.
We modify Ukkonen’s tree, associating with each vertex an additional field to store the length of the string associated with . Maintaining this field requires a constant number of operations at the moment when is created. Thus, this update adds time and space to the total cost of maintaining Ukkonen’s tree. Thus, Lemma 3.3 holds for the modified Ukkonen’s tree as well. It remains to note that .
Theorem 1.1: existence.
The following algorithm solves the problem and has the required complexity. The algorithm uses data structures and , processing the same input string . The next (say, th) symbol of is added to both structures through the procedures and . After this, we call to get the length of the maximal palindromic suffix of and to get the length of the shortest suffix of that never occurred in before. The inequality means the detection of a new palindrome; we get its first and last positions from the structure and output them. In the case of the opposite inequality, there is no new palindrome, and we output “—”.
Consider the string again. We get the following results for :
3.3 Lower bounds
Recall that a dictionary is a data structure containing some set of elements and designed for the fast implementation of basic operations like checking the membership of an element in the set, deleting an existing element, or adding a new element. Below we consider an insert-only dictionary over a set . In each moment, such a dictionary contains a subset of and supports only the operation . This operation checks whether the element is already in the dictionary; if no, it adds to the dictionary.
Suppose that the alphabet consists of indivisible elements, , and the insert-only dictionary over is initially empty. Then the sequence of calls of requires, in the worst case, time if is ordered and if is unordered.
Let be an ordered alphabet. Assume that on some stage all letters with even numbers are in the dictionary, while all elements with odd numbers are not. Consider the next operation. In the comparison-based computation model, a query “?” is answered by some decision tree; each node of this tree is marked by the condition “” for some . To distinguish between and , the tree should contain the nodes for both and . Now note that for any , exactly one of the letters and belongs to . So, to answer correctly all possible queries “?” the decision tree should have nodes for all letters. Then the depth of this tree is . Therefore, for some element the number of comparisons needed to prove that is . After processing , the content of the dictionary remains unchanged. The decision tree can change, but it does not matter: we again choose the next letter to be the one having an even number and requiring comparisons to prove its membership in . Thus, our “bad” sequence of calls is as follows: it starts with , and continues with the “worst” letter, described above, on each next step. Even if the first calls can be performed in time each, the overall time is , as required.
In the case of unordered alphabet all conditions in the decision tree have the form “”. It is clear that if the dictionary contains elements, the maximal number of comparisons equals as well. Choosing the bad sequence of calls in the same way as for the ordered alphabet, we arrive at the required bound . ∎
Before finishing the proof of Theorem 1.1 we mention the following lemma. Its proof is obvious.
Suppose that are two different letters and is a string such that each is a letter different from and . Then all nonempty subpalindromes of are single letters.
Proof of Theorem 1.1: lower bounds.
We prove the required lower bounds reducing the problem of maintaining an insert-only dictionary to counting distinct palindromes in a string. Assume that we have a black box algorithm that processes an input string letter by letter and outputs, after each step, the number of distinct palindromes in the string read so far. The time complexity of this algorithm depends on the length of the string at least linearly, and a linear in algorithm does exist, as we have proved in the Sect. 3.2. Thus, we can assume that the considered black box algorithm works in time , where is the size of the alphabet of the processed string and the function is non-decreasing.
The insert-only dictionary over a set of size can be maintained as follows. We pick up two letters and mark their presence in the dictionary using two boolean variables, and . All other letters are processed with the aid of the mentioned black box. Let us describe how to process a sequence of calls starting from the empty dictionary.
For each call, we first compare the current letter to and . If , then is the answer to the query “?”; after answering the query we set . The case is managed in the same way.
If , we feed the black box with , , and (in this order). Then we get the output of the black box and check whether the number of distinct subpalindromes in its input string increased. By Lemma 3.5, the increase happens if and only if appears in the input string of the black box for the first time. Thus, we can immediately answer the query “?”, and, moreover, is now in the dictionary.
The described algorithm performs the sequence of calls in time plus the time used by the blackbox to process a string of length over . Hence, the overall time bound is . In view of Lemma 3.4 we obtain (resp., ) in the case of ordered (resp., unordered) alphabet . The required lower bounds are proved. ∎
-  M. Crochemore and W. Rytter, Jewels of stringology, World Scientific Publishing Co. Pte. Ltd., 2002.
-  X. Droubay, J. Justin, and G. Pirillo, Episturmian words and some constructions of de luca and rauzy, Theoret. Comput. Sci. 255 (2001), 539–553.
-  R. Groult, E. Prieur, and G. Richomme, Counting distinct palindromes in a word in linear time, Inform. Process. Lett. 110 (2010), 908–912.
-  G. Manacher, A new linear-time on-line algorithm finding the smallest initial palindrome of a string, J. ACM 22 (1975), no. 3, 346–351.
-  E. Ukkonen, On-line construction of suffix trees, Algorithmica 14 (1995), no. 3, 249–260.