From cardinal spline wavelet bases to highly coherent dictionaries
Wavelet families arise by scaling and translations of a prototype function, called the mother wavelet. The construction of wavelet bases for cardinal spline spaces is generally carried out within the multi-resolution analysis scheme. Thus, the usual way of increasing the dimension of the multi-resolution subspaces is by augmenting the scaling factor. We show here that, when working on a compact interval, the identical effect can be achieved without changing the wavelet scale but reducing the translation parameter. By such a procedure we generate a redundant frame, called a dictionary, spanning the same spaces as a wavelet basis but with wavelets of broader support. We characterise the correlation of the dictionary elements by measuring their ‘coherence’ and produce examples illustrating the relevance of highly coherent dictionaries to problems of sparse signal representation.
Canonical Coherent States have played an important role in quantum physics from the early days to the present time [1, 2, 3, 4, 5, 6, 7, 8, 9]. Mathematically they have been studied within the structure of frames [10, 11, 6]. Their equivalent in the context of signal processing were first introduced by Gabor and nowadays frequently appear in some contexts under the name of Gabor frames or Weyl-Heisenberg frames [12, 13]. On the other hand, the particular class of coherent states arising from the affine group on the real line, called wavelets [14, 12, 6, 11, 13, 15], have also been broadly applied in physics [16, 17, 18, 19], as well as in signal processing. The applications of wavelets notably increased when fast techniques, arising from the multi-resolution analysis scheme, became available. Ever since wavelet analysis has been a popular tool among practitioners.
For the most part multi-resolution analysis has been applied for generating orthogonal, semi-orthogonal and biorthogonal wavelet bases. Some redundant frames have been constructed within this framework as well. However, we should recall that countable sets of frame wavelets were first constructed by discretisation of the translation and scaling parameters of affine coherent states. According to the mathematical measure of coherence introduced in  an orthogonal basis has coherence zero and a non-orthogonal basis, although not redundant by definition, has coherence greater than zero. It is the purpose of the present effort to show that, in some finite dimensional spaces, by properly increasing the coherence one can generate spaces of higher dimension.
The success in constructing a number of linearly independent wavelets with good mathematical properties, such as regularity or localisation, is very much appreciated by researchers in different fields (including the authors of this contribution) as a major achievement in the design of wavelets. Nevertheless, in this communication we would like to illustrate, by recourse to a very interesting example, a remarkable property of a class of highly coherent redundant wavelet systems. We will construct such systems for finite dimensional cardinal splines spaces, which are well characterised from a mathematical viewpoint  and will be shown to be useful for the problem of sparse signal representation by non-linear approximation.
The problem of non-linear approximation concerns the representation of a given function (signal) through the selection of waveforms, frequently called atoms, which are taken from a redundant set, called a dictionary . This problem has been the subject of quite recent theoretical work with regard to quasi incoherent dictionaries [23, 20].
The term spline wavelets comprises a number of wavelet systems ranging from Haar piece-wise constant wavelets to wavelet functions of much higher regularity. In our framework all such systems admit an equivalent construction and a particular one is obtained by setting the order of the corresponding B-spline scaling function. We will restrict wavelet systems to an interval and construct spline wavelet dictionaries on the basis of the following result: Let us focus on a cardinal spline space on a compact interval with distance between two adjacent knots. We prove that such a space can be spanned by translating a wavelet taken from a multi-resolution subspace corresponding to a fixed scale, say scale , , as long as the distance between two consecutive functions is reduced to be . This interesting feature provides the foundations for the construction of a large variety of possible dictionaries for the identical space. In particular, multi-resolution-like dictionaries can be constructed by spline wavelets whose support at the finest scale is larger than that corresponding to the finest scale in a multi-resolution analysis of the identical space. This property provides a clear explanation of the ‘power of coherence’. Our results for finite dimensional cardinal spline spaces on a compact interval assert that the benefit of increasing coherence, by decreasing the translation parameter of the functions, is not only a consequence of incorporating redundancy but also of the fact that by such a procedure one may increase the dimension of the space. This phenomenon emerges clearly from our construction.
As will be illustrated by numerical simulations, transforming a spline wavelet basis into highly coherent dictionaries has a significant impact on signal representation by non-linear techniques. It will be shown that some of these dictionaries may yield a significant gain in the sparseness of a representation with respect to the results produced by means of the corresponding basis. This is enhanced by the comparison with other techniques such as Best Basis Selection using Wavelet Packets.
The paper is organised as follows: Section 2 introduces some background on spline multi-resolution analysis on a compact interval relevant for our purpose. Section 3 establishes the fact that, rather than increasing the wavelet scale to provide a representation of a cardinal spline space of higher dimension, the representation can be achieved by appropriate reduction of the distance between two consecutive wavelets. This result is used for constructing a multi-resolution-like redundant dictionary that, through the numerical examples of Section 4, is shown to be useful for sparse signal representation. The conclusions are drawn in Section 5.
2 Cardinal spline multi-resolution analysis on a compact interval
Let us recall that multi-resolution analysis restricted to the interval , involves a sequence of nested spaces satisfying that The complementary wavelet subspaces are constructed in order to fulfil
Without loss of generality we will assume throughout the paper that . Let us consider now that are cardinal spline spaces of order with simple knots at the equidistant partition of the interval , having distance between two adjacent knots. This implies that each consists of piece-wise polynomials of order having continuous derivatives . A basis for arises from the restriction of the functions
to the interval . The corresponding scaling function is the cardinal B-spline of order associated with the uniform simple knot sequence . Such a function is given as
where is equal to if and 0 otherwise.
It should be stressed that different ways of constructing the boundary functions give rise to different bases for . The restriction of the scaling functions to the interval provides one such basis, which is easy to construct. Indeed, considering that the support of is one has:
Without loss of generality, we assume that at least one scaling function from and one wavelet from are completely contained in . This is equivalent to assuming that where is the length of . A basis for can then be constructed by restricting the functions
to the interval and eliminating some redundancy introduced by the cutting process. One can start by considering all the functions having non-trivial intersection with the interval , which restricts the values of the index to . Since the cardinality of such a set is , but , in order to have a basis it is necessary to eliminate boundary functions. A natural choice is to eliminate the first left boundary functions and the last right boundary functions where and , indicate the upper and lower integer part, respectively. A basis for constructed by the cut-off process is thereby given as:
3 Spline wavelet dictionaries
In a previous publication we have shown that a cardinal spline space on a compact interval can be spanned by dictionaries consisting of functions of broader support than the corresponding B-spline basis . In the present notation this entails that for an integer the set of B-splines of order
Note that the support of a function in (8) completely contained in is -times the support of a corresponding basis function from , and we have introduced a new translation parameter which is no longer an integer but .
We call the set a B-spline dictionary and wish to consider now the possibility of creating cardinal spline dictionaries of wavelets. To this end we construct the set as follows: The wavelets are translated using the translation parameter, , defined above. All the functions whose supports have non-trivial intersection with the interval are considered. Thus
Our proposal for constructing cardinal spline wavelet dictionaries stems from the fact that . This result is established in the next theorem. The proof is achieved by using an equivalent technique to that for the proof of Theorem 1 in  and is given in Appendix A.
Let be given as in (10) for a given integer . Then the following relation holds:
The above theorem gives us a tool for designing different wavelet dictionaries for cardinal spline spaces. Notice that simply by setting different values of the index in (10) we obtain dictionaries of wavelets of different support spanning the identical cardinal spline space .
As described below the result of Theorem 1 allows us to create multi-resolution-like dictionaries for the space being considered.
Considering we construct multi-resolution-like dictionaries spanning as
The top left graph of Fig. 1 shows two consecutive scaling functions (dark line) and two consecutive wavelets at the coarsest scale (light line) in a linear spline basis (order ). The right graph shows the equivalent functions in a dictionary constructed by considering . The bottom graphs depict an equivalent example but involving cubic splines, which correspond to order . Let us remark that, since in both examples the dictionaries arise by setting , for representing the same space as the corresponding basis we should consider a coarser scale, i.e, if the finest scale in the basis corresponds to a scaling factor in the dictionary the finest scale corresponds to a scaling factor .
Of course, considering the result of Theorem 1 one could construct many different dictionaries for the identical space. Each such dictionary constitutes a frame for a cardinal spline space of order with distance between knots. As will be illustrated in the next section, particular dictionaries constructed as in (13) may yield a significant gain, as far as sparseness is concerned, in problems of signal representation.
For the examples given in Section 4 we construct specific dictionaries on the interval as follows: We consider cubic splines and use the semi-orthogonal wavelets (Chui-Wang4 family) introduced in  to construct the basis for each by the simple cut off process described in Section 2. In order to construct the dictionary we take a prototype function from each subspace (and also from ) and translate such a function to a distance . Notice that we do not use functions from . This is because by reducing the distance between functions to we need one less scale than the wavelet basis to represent the same space.
In order to characterise the coherence of the dictionaries at hand we use the cumulative coherence function, , introduced in . Given a dictionary , where is the set of indices labelling the dictionary atoms, measures how much a collection of atoms resembles a fixed one. It is defined by
where indicates the cardinality of the set .
The cumulative coherence for the basis and the dictionary , which will be used in the next section, is plotted in Fig. 2. It is clear from this figure that the coherence of the dictionary is much more larger than the coherence of the basis .
An increment in coherence also implies that the construction of the dual frame for the dictionary may become an ill posed problem. However, when the aim is to use dictionaries for sparse signal representation one is not interested in the dual frame for the whole space. On the contrary, in order to produce orthogonal projections onto the approximation subspace, the duals for the particular subspace need to be determined. For the representation to be useful for compression purposes the dimension of the approximation subspace should be considerable smaller that the dimension of the dictionary space. Thus, the construction of the duals in the corresponding subspace is well posed. Recursive techniques for adapting dual functions are discussed in [26, 27, 28].
4 Application to sparse signal representation
In this section we use the multi-resolution-like dictionary characterised in the previous section, corresponding to considering . The central aim is to compare the sparseness of the signal representation achieved by using the basis and the dictionary . Moreover, the results produced by a number of other techniques will be presented for further comparison. The two signals to be represented are the chirp and the seismic signal of Fig. 3.
The piece of seismic signal was taken from the WaveLab802 Toolbox  (it is acknowledged there that such a signal is distributed throughout the seismic industry as a test dataset). Both signals have a good approximation, coinciding with the graphs of Fig. 3, in , the space of cardinal cubic splines with the distance between two adjacent knots. In order to represent this space we will use: The basis and the dictionary . In both cases we apply the same strategy for selecting the atoms to approximate the given signal by the atomic decomposition
where the superscript indicates that the coefficients yield the orthogonal projection of the given signal onto the subspace spanned by the selected atoms .
Our strategy evolves by stepwise minimum residual error selection and is carried out in three stages:
The atoms are selected one by one according to the adaptive pursuit method  in order to reach a predetermined precision in the representation of the signal.
The previous approximation is improved by means of a “swapping procedure” which operates as follows: at each step an atom of the atomic decomposition is replaced by a dictionary atom, provided that the operation improves the residual error .
A backward pursuit method  is applied to disregard some coefficients in order to produce an approximation up to the error of i).
MATLAB codes for constructing the proposed dictionaries and implementing the selection strategy are available at .
|BBS Shannon entropy||315||217|
|BBS Shannon entropy||279||201|
For further comparison we have represented the signals using Fast Wavelet Transform (FWT) with the biorthogonal (cubic spline) family Bior4.4 and the Daubechies orthogonal family Daub10. With these two families we also considered a number of different criteria for Best Basis Selection (BBS) from the corresponding Wavelet Packets (WP) . These techniques were applied by using the Wavekit tool  and iteratively tuning the threshold in order to produce a signal approximation of the same quality (with respect of the error norm) by all the approaches. The results are presented in Table 1. The first column labels the wavelet families. The second one specifies the approach and the third and fourth column the number of atoms needed by those approaches to represent the corresponding signal up to the same precision.
As can be seen in Table 1, to obtain the desired approximation of the chirp of Fig. 3 using the basis we need 254 atoms. Sparseness is improved to 166 atoms by using the proposed redundant dictionary involving functions of the same nature. On the contrary, the number of atoms that are needed to represent the chirp signal by all the other approaches is larger. The same feature is exhibited in the representation of the seismic signal. However, there is an interesting phenomenon that can be observed: In the previous case the FWT (for both the Bior4.4 and Daub10 families) was shown to produce a much poorer sparseness property than the basis . In this case the results yielded by the three wavelet bases are comparable. This is due to the fact that the nature of the chirp near the left boundary is very different from that near the right one. Hence, the periodic boundary conditions in the implementation of FWT have a very negative effect in relation to the spareness of the representation. We believe this is the reason why the cut-off approach performs much better in representing the chirp. However, in the case of the seismic signal for which the two ends of the signal are not so different and of small magnitude, the approximations obtained by all the wavelet bases are comparable. The representation of this signal by selecting atoms from the proposed dictionary is again significantly superior to all the other techniques.
The construction of wavelet dictionaries for cardinal spline spaces on a compact interval was discussed. It was first proved that if is the distance between consecutive knots in a cardinal spline space on a compact interval, such a space can be spanned by translating wavelets at scale a distance . This result can be used for building a large variety of wavelet dictionaries for the identical space. In particular, multi-resolution-like highly coherent dictionaries of wavelets of different supports were constructed. An interesting feature of the proposed construction is that it allows us to span a spline space of given dimension by using wider wavelets than those necessary in multi-resolution analysis. As illustrated by the numerical examples, this feature is relevant to sparse signal representation by non-linear techniques. The simulations presented here clearly show that the proposed multiresolution-like dictionaries, which are easily constructed by simple translations of prototype functions at different scales, may yield a very significant gain in the sparseness of a signal representation by step-wise selection techniques. This outcome could be somewhat ‘expected’, since by decreasing the translation step one also generates more choice so as to chose the suitable functions for representing a given signal. However, what is definitely a very interesting and somewhat surprising result is that, when working on a compact interval, by decreasing the translation step of a wavelet family one can generate higher dimensional spaces. In our mind this is a mathematical characterisation of the ‘power of coherence’.
Support from EPSRC (grants GRR8635501 and EPD0626321) is acknowledged.
Appendix A: Proof of Theorem 1
The inclusion is obvious because for all the spline wavelets in are, by definition, in . Since the distance between two adjacent knots in is , by moving the wavelets such a distance it is ensured that they remain in . We can then build scaling-like equations for all the functions of the dictionary :
The proof of the inclusion is achieved by showing that every function can be written as a linear combination of functions from . For every (corresponding to an inner or right boundary function ) there is a unique index such that and for every . Hence, in the line of , by using (16b) or (16c) the function is substituted by plus a linear combination of , . This recursive process ends with the substitution where .
As for every (corresponding to a left boundary function ) there is also a unique index such that , and for every . Thus using (16a) the function is substituted by plus a linear combination of . The recursive process ends with the substitution where .
Consequently, by back-substitution, we have the decomposition of each function in terms of . ∎
-  R. J. Glauber, Coherent and incoherent states of the radiation field, Physical Review 131 (6) (1963) 2766–2788.
-  I. Daubechies, Coherent states and projective representation of the linear canonical transformations, Journal of Mathematical Physics 21 (1980) 1277–1389.
-  S. K. S. J. R. Klauder, Coherent States, World Scientific, Singapore, 1985.
-  G. Kaiser, Quantum Physics, Relativity and Complex Spacetime Towards a New Synthesis, North-Holland, Amsterdam, 1990.
-  S. T. Ali, J. P. Antoine, J. P. Gazeau, Square integrability of group representation on homogeneous spaces. II. coherent and quasi-coherent states. the case of the Poincaré group, Ann. Inst. Henri Poincaré 55 (1991) 587–890.
-  S. T. Ali, J. P. Antoine, J. P. Gazeau, Coherent States, Wavelets and their generalizations, Springer, London, 2000.
-  A. Vourdas, Analytic representations in quantum mechanics, J. Phys. A: Math. Gen. 39 (7) (2006) R65–R141.
-  M. A. M. d. A. A D Ribeiro, A. F. R. de Toledo Piza, The semiclassical coherent state propagator for systems with spin, J. Phys. A: Math. Gen. 39 (12) (2006) 3085–3097.
-  L. C. dos Santos, M. A. M. de Aguiar, Coherent state path integrals in the weyl representation, J. Phys. A: Math. Gen. 39 (43) (2006) 13465–13482.
-  S. T. Ali, J. P. Antoine, J. P. Gazeau, Continuous frames in Hilbert space, Annals of Physics 222 (1993) 1–37.
-  G. Kaiser, A Friendly Guide to Wavelets, Birkhäuser, Berlin, 1994.
-  C. E. Heil, D. F. Walnut, Continuous and discrete wavelet transforms, SIAM Review 31 (4) (1989) 628–666.
-  I. Daubechies, Ten Lectures on Wavelets, CBMS-NSF Reg. Conf. Series in Applied Math., SIAM, Philadelphia, PA, 1992.
-  E. W. Aslaken, J. R. Klauder, Continuous representation theory using the affine group, Journal of Mathematical Physics 10 (12) (1969) 2267–2275.
-  L. Rebollo-Neira, Frames in two dimensions arising from wavelet transforms, Proceedings of the Royal Society, series A 457 (2013) (2001) 2079–2091.
-  G. Kaiser, Physical wavelets and their sources: real physics in complex spacetime, J. Phys. A: Math. Gen. 36 (30) (2003) R291–R338.
-  M. Andrle, C. Burdik, J.-P. Gazeau, R. Krejcar, Wavelet multiresolution for the Fibonacci chain, Journal of Physics A: Mathematical and General 33 (2000) L47–L51.
-  M. Andrle, P. Kramer, Inflation and wavelets for the icosahedral Danzer tiling, Journal of Physics A: Mathematical and General 37 (2004) 3443–3457.
-  J. C. van den Berg (Ed.), Wavelets in Physics, Cambridge Univerity Press, Cambridge, 1999.
-  J. Tropp, Greed is good: algorithmic results for sparse approximation, IEEE Transactions on Information Theory 50 (10) (2004) 2231–2242.
-  L. L. Schumaker, Spline Functions: Basic Theory, Wiley, New York, 1981.
-  S. Mallat, A wavelet tour of signal processing, Academic Press, London, 1998.
-  R. Gribonval, M. Nielsen, Nonlinear approximation with dictionaries. I. Direct estimates, Journal of Fourier Analysis and Applications 10 (2004) 55–71.
-  M. Andrle, L. Rebollo-Neira, Cardinal B-spline dictionaries on a compact interval, Appl. Comput. Harmon. Anal. 18 (2005) 336–346.
-  C. Chui, J. Wang, On compactly supported spline wavelets and a duality principle, Trans. Amer. Math. Soc. 330 (1992) 903–915.
L. Rebollo-Neira, Recursive bi-orthogonalisation approach and orthogonal
projectors, math-ph/0209026 (2002);
L. Rebollo-Neira, New Topics in Mathematical Physics Research, Nova Science Publishers, New York, 2006, Ch. On non-orthogonal signal representation.
-  L. Rebollo-Neira, Backward adaptive biorthogonalization, International Journal of Mathematics and Mathematical Science 2004 (35) (2004) 1843–1853.
-  L. Rebollo-Neira, Constructive updating/downdating of oblique projectors: generalization of the Gram-Schmidt process, Journal of Physics A: Mathematical and Theoretical 40 (24) (2007) 6381–6394.
-  Wavelab802, web page http://www-stat.stanford.edu/~wavelab.
-  L. Rebollo-Neira, D. Lowe, Optimized orthogonal matching pursuit approach, IEEE Signal Processing Letters 9 (2002) 137–140.
-  M. Andrle, L. Rebollo-Neira, A swapping-based refinement of orthogonal matching pursuit strategies, Signal Processing 86 (2006) 480–495.
-  M. Andrle, L. Rebollo-Neira, E. Sagianos, Backward-optimized orthogonal matching pursuit approach, IEEE Signal Proc. Let. 11 (2004) 705–708.
M. Andrle, L. Rebollo-Neira, Biorthogonal techniques for optimal signal
-  R. R. Coifman, M. V. Wickerhauser, Entropy based algorithms for best basis selection, IEEE Transactions on Information Theory 32 (1992) 712–718.
-  H. Ojanen, Wavekit: a wavelet toolbox for Matlab, web page http://www.math.rutgers.edu/~ojanen/wavekit (1998).