LandweberKaczmarz method in Banach spaces with inexact inner solvers
Abstract.
In recent years Landweber(Kaczmarz) method has been proposed for solving nonlinear illposed inverse problems in Banach spaces using general convex penalty functions. The implementation of this method involves solving a (nonsmooth) convex minimization problem at each iteration step and the existing theory requires its exact resolution which in general is impossible in practical applications. In this paper we propose a version of LandweberKaczmarz method in Banach spaces in which the minimization problem involved in each iteration step is solved inexactly. Based on the subdifferential calculus we give a convergence analysis of our method. Furthermore, using Nesterov’s strategy, we propose a possible accelerated version of LandweberKaczmarz method. Numerical results on computed tomography and parameter identification in partial differential equations are provided to support our theoretical results and to demonstrate our accelerated method.
1. Introduction
Regularization of inverse problems has been considered extensively and significant progress has been made; see [10, 18, 21, 26, 34] and references therein. Due to the demand of capturing special features of the reconstructed objects and the demand of dealing with general noise, regularization in Banach spaces has emerged as a highly active research field and many new regularization methods have been proposed and investigated in recent years; one may refer to [5, 17, 19, 23, 24, 25, 33] and the references therein for recent developments.
Because of its simplicity and relatively small complexity per iteration, Landweber iteration and its Kaczmarz version have received extensive attention in inverse problem community [9, 11, 12, 14, 27]. In recent years, several versions of Landweber iteration has been formulated in Banach spaces, see [5, 20, 22, 31]. When solving illposed system of the form
(1.1) 
consisting of equations, a Kaczmarz version of Landweber iteration in Banach spaces with general uniformly convex penalty functions has been proposed in [22] which cyclically considers each equation in (1.1) in a GaussSeidel manner. For these modern versions of Landweber method, each iteration step essentially requires the computation of next iterate from current iterate via
where is a step size, is a duality mapping, denote one of , denotes the Fréchet derivative of , and is a uniformly convex function. Therefore, the implementation of the Landweber(Kaczmarz) method in Banach spaces requires to solving a minimization problem of the form
(1.2) 
associated with in each iteration step.
The existing convergence theory on Landweber(Kaczmarz) method in Banach spaces requires the exact resolution of the minimization problem (1.2). For some special its exact resolution is possible. However, this minimization problem in general can only be solved inexactly by an iterative procedure which may produce an approximate solution satisfying
(1.3) 
for some small . Furthermore, numerical simulations indicate that solving (1.2) very accurately in every step does not improve the final reconstruction result but wastes a huge amount of computational time. Therefore, it is necessary to formulate a LandweberKaczmarz method with inexact inner solver in each iteration step and to develop the corresponding convergence theory. The inequality (1.3) suggests that the subdifferential calculus might be a useful tool for this purpose.
It is wellknown that Landweber iteration admits the slow convergence property ([10]) which restricts its applications to wide range of problems. To expand its applied range, it is necessary to introduce some acceleration strategy to fasten its convergence speed. In this paper we will use the Nesterov’s strategy in optimization ([29]) to propose an accelerated version of LandweberKaczmarz method in Banach spaces in which some extrapolation steps are incorporated. We do not have a theory to guarantee its acceleration effect at this moment, however, we will provide numerical simulations to support the fast convergence property.
This paper is organized as follows. In section 2 we will provide some preliminaries on Banach spaces and convex analysis and derive some useful results concerning subdifferential. In section 3 we propose the LandweberKaczmarz method with inexact inner solvers, show that it is welldefined, and prove its convergence and regularization property. We then use Nesterov’s strategy to propose an accelerated version. We also discuss how to produce the inexact solvers for solving the inner minimization problem at each iteration step of LandweberKaczmarz method. Finally, we provide numerical simulations to verify the theoretical results and to demonstrate the fast convergence of the accelerated method.
2. Preliminaries
Let be two Banach space whose norm is denoted by . We use to denote its dual spaces. For any and , we write for the duality pairing. Let be another Banach space. By we denote for the space of all bounded linear operators from to . For any we use to denote its adjoint, i.e.
for any and .
For each , the setvalued mapping defined by
is called the duality mapping of with gauge function . When is uniformly smooth in the sense that its modulus of smoothness
satisfies , the duality mapping , for each , is single valued and uniformly continuous on bounded sets.
Given a convex function , we use
to denote its effective domain. It is called proper if . For a proper convex function and , we define for any the set
which is called the subdifferential of at . Any element in is called an subgradient of at . When , the subdifferential of reduces to the subdifferential . It is clear that for some implies . If is lower semicontinuous, then for any , the subdifferential is always nonempty for any , see [35, Theorem 2.4.4]; however, can be empty in general.
For with , we may introduce
which is called the Bregman distance induced by at in the direction . It is clear that
When , the Bregman distance becomes the wellknown Bregman distance [7] which will be denoted by . It should be pointed out that Bregman distance is not a metric distance in general. Nevertheless, as the following result shows, the Bregman distance can be used to detect information under the norm if is convex for some in the sense that there is a constant such that
(2.1) 
for all and , .
Lemma 2.1.
Let be a proper, lower semicontinuous function that is convex in the sense of (2.1). If for some , then
(2.2) 
for any .
Proof.
In convex analysis, the LegendreFenchel conjugate is an important notion. Given a proper convex function , its LegendreFenchel conjugate is defined by
As an immediate consequence of the definition, one can see, for any , that
(2.3) 
If, in addition, is lower semicontinuous, then ([35, Theorem 2.4.4])
(2.4) 
For a proper, lower semicontinuous, convex function, even if it is nonsmooth, its LegendreFenchel conjugate can have enough regularity as the following result indicates.
Lemma 2.2.
Let be a reflexive Banach space and let be a proper, lower semicontinuous function that is convex in the sense of (2.1). Then , is Fréchet differentiable, and its gradient satisfies
(2.5) 
which consequently implies
(2.6) 
for any , where is the number conjugate to , i.e. .
Proof.
See [35, Theorem 3.5.10 and Corollary 3.5.11]. ∎
Finally we conclude this section by providing a result which show that, when is convex, then, for any , the distance from to can be controlled in terms of .
Lemma 2.3.
Let ba a reflexive Banach space and let be a proper, lower semicontinuous function that is convex in the sense of (2.1). If and satisfy for some , then for any there holds
(2.7) 
and hence
(2.8) 
3. The method
We consider the system
(3.1) 
consisting of equations, where, for each , is an operator between two reflexive Banach spaces and . Such systems arise in many practical applications including various tomography problems using multiple exterior measurements.
We will assume that (3.1) has a solution which consequently implies that
In practical applications, instead of we only have noisy data satisfying
(3.2) 
with a small noise level . How to use to produce an approximate solution of (3.1) is an important question. In [22] we proposed a Landweber iteration of Kaczmarz type which makes use of every equation in (3.1) cyclically. In order to capture the feature of the sought solution, general convex functions have been used in [22] as penalty terms.
We will make the following assumption, where .
Assumption 3.1.
According to Assumption 3.1 (c), we can find a constant such that
(3.4) 
Moreover
(3.5) 
for all which shows that is continuous on for each .
The formulation of the Landweber iteration of Kaczmarz type in [22] involves in each iteration step a minimization problem of the form
(3.6) 
for any . The convergence result developed there requires to solving (3.6) exactly. The exact solution of (3.6) can be found for some special . However, this minimization problem in general can only be solved inexactly by iterative procedures. Furthermore, numerical simulations indicate that solving (3.6) very accurately in every step does not improve the final reconstruction result. Therefore, it is necessary to formulate a LandweberKaczmarz method with inexact inner solver in each iteration step and to develop the corresponding convergence theory.
Concerning the inexact resolution of (3.6), we make the following assumption.
Assumption 3.2.
For any given there is a procedure for solving (3.6) such that for any , the element satisfies
(3.7) 
Moreover, for each , the mapping is continuous.
In subsection 3.5 we will discuss how to produce the inexact procedure by using concrete examples of including the total variation like convex penalty functions.
3.1. The method with noisy data
We are ready to formulate our LandweberKaczmarz method with inexact inner solvers. We will take and let denote the duality mapping over with gauge function . Given an integer , we set .
Algorithm 3.1 (LandweberKaczmarz method with noisy data).
Let , and be suitably chosen numbers, and let be a sequence of positive numbers satisfying .

Pick and such that .

Let and . Let and let be a small number. For we define and
We then update
(3.8) where
(3.9) with

Let be the first integer such that and use as an approximate solution.
In Algorithm 3.1, each is determined by completely without involving , and each is defined by the inexact procedure specified in Assumption 3.2 for solving (3.6) which is independent of , . This splitting character can make the implementation of Algorithm 3.1 efficiently. Furthermore, the definition of implies that
which shows that
(3.10) 
We will use this fact in the forthcoming convergence analysis.
We first prove the following basic result which shows that Algorithm 3.1 is welldefined.
Lemma 3.1.
Let and be reflexive Banach spaces with being uniformly smooth, let and , satisfy Assumption 3.1, and let be a sequence of positive numbers satisfying
(3.11) 
Let be a constant such that . If and are chosen such that
(3.12) 
and if is chosen such that , where
(3.13) 
then for Algorithm 3.1 there hold

for all ;

the method terminates after iteration steps;
Proof.
Let be any solution of (3.1) in . We first show that, if for some , then
(3.15) 
To see this, we consider
which can be written as
Since , we have from (2.3) that
By the definition of we also have
Therefore
Because is convex, we may use (2.6) in Lemma 2.2 and the definition of to obtain
(3.16) 
Since , we may use Lemma 2.3 to derive that
Plugging this estimate into (3.1) and using the definition of it follows that
By writing
we may use the condition , Assumption 3.1 (c), and the properties of to obtain
Therefore
(3.17) 
By the definition of we can see that
and
Combining the above two estimates with (3.1) we can obtain
(3.18) 
We next consider the term . We claim that
(3.19) 
where is the constant defined by (3.13). Indeed, this is trivial when . We only need to consider the case that . If , then
If , we may use the inequality for and to derive that
Thus, by using , we have
We therefore obtain (3.19).