Landweber-Kaczmarz method in Banach spaces with inexact inner solvers
In recent years Landweber(-Kaczmarz) method has been proposed for solving nonlinear ill-posed inverse problems in Banach spaces using general convex penalty functions. The implementation of this method involves solving a (nonsmooth) convex minimization problem at each iteration step and the existing theory requires its exact resolution which in general is impossible in practical applications. In this paper we propose a version of Landweber-Kaczmarz method in Banach spaces in which the minimization problem involved in each iteration step is solved inexactly. Based on the -subdifferential calculus we give a convergence analysis of our method. Furthermore, using Nesterov’s strategy, we propose a possible accelerated version of Landweber-Kaczmarz method. Numerical results on computed tomography and parameter identification in partial differential equations are provided to support our theoretical results and to demonstrate our accelerated method.
Regularization of inverse problems has been considered extensively and significant progress has been made; see [10, 18, 21, 26, 34] and references therein. Due to the demand of capturing special features of the reconstructed objects and the demand of dealing with general noise, regularization in Banach spaces has emerged as a highly active research field and many new regularization methods have been proposed and investigated in recent years; one may refer to [5, 17, 19, 23, 24, 25, 33] and the references therein for recent developments.
Because of its simplicity and relatively small complexity per iteration, Landweber iteration and its Kaczmarz version have received extensive attention in inverse problem community [9, 11, 12, 14, 27]. In recent years, several versions of Landweber iteration has been formulated in Banach spaces, see [5, 20, 22, 31]. When solving ill-posed system of the form
consisting of equations, a Kaczmarz version of Landweber iteration in Banach spaces with general uniformly convex penalty functions has been proposed in  which cyclically considers each equation in (1.1) in a Gauss-Seidel manner. For these modern versions of Landweber method, each iteration step essentially requires the computation of next iterate from current iterate via
where is a step size, is a duality mapping, denote one of , denotes the Fréchet derivative of , and is a uniformly convex function. Therefore, the implementation of the Landweber(-Kaczmarz) method in Banach spaces requires to solving a minimization problem of the form
associated with in each iteration step.
The existing convergence theory on Landweber(-Kaczmarz) method in Banach spaces requires the exact resolution of the minimization problem (1.2). For some special its exact resolution is possible. However, this minimization problem in general can only be solved inexactly by an iterative procedure which may produce an approximate solution satisfying
for some small . Furthermore, numerical simulations indicate that solving (1.2) very accurately in every step does not improve the final reconstruction result but wastes a huge amount of computational time. Therefore, it is necessary to formulate a Landweber-Kaczmarz method with inexact inner solver in each iteration step and to develop the corresponding convergence theory. The inequality (1.3) suggests that the -subdifferential calculus might be a useful tool for this purpose.
It is well-known that Landweber iteration admits the slow convergence property () which restricts its applications to wide range of problems. To expand its applied range, it is necessary to introduce some acceleration strategy to fasten its convergence speed. In this paper we will use the Nesterov’s strategy in optimization () to propose an accelerated version of Landweber-Kaczmarz method in Banach spaces in which some extrapolation steps are incorporated. We do not have a theory to guarantee its acceleration effect at this moment, however, we will provide numerical simulations to support the fast convergence property.
This paper is organized as follows. In section 2 we will provide some preliminaries on Banach spaces and convex analysis and derive some useful results concerning -subdifferential. In section 3 we propose the Landweber-Kaczmarz method with inexact inner solvers, show that it is well-defined, and prove its convergence and regularization property. We then use Nesterov’s strategy to propose an accelerated version. We also discuss how to produce the inexact solvers for solving the inner minimization problem at each iteration step of Landweber-Kaczmarz method. Finally, we provide numerical simulations to verify the theoretical results and to demonstrate the fast convergence of the accelerated method.
Let be two Banach space whose norm is denoted by . We use to denote its dual spaces. For any and , we write for the duality pairing. Let be another Banach space. By we denote for the space of all bounded linear operators from to . For any we use to denote its adjoint, i.e.
for any and .
For each , the set-valued mapping defined by
is called the duality mapping of with gauge function . When is uniformly smooth in the sense that its modulus of smoothness
satisfies , the duality mapping , for each , is single valued and uniformly continuous on bounded sets.
Given a convex function , we use
to denote its effective domain. It is called proper if . For a proper convex function and , we define for any the set
which is called the -subdifferential of at . Any element in is called an -subgradient of at . When , the -subdifferential of reduces to the subdifferential . It is clear that for some implies . If is lower semi-continuous, then for any , the -subdifferential is always non-empty for any , see [35, Theorem 2.4.4]; however, can be empty in general.
For with , we may introduce
which is called the -Bregman distance induced by at in the direction . It is clear that
When , the -Bregman distance becomes the well-known Bregman distance  which will be denoted by . It should be pointed out that -Bregman distance is not a metric distance in general. Nevertheless, as the following result shows, the -Bregman distance can be used to detect information under the norm if is -convex for some in the sense that there is a constant such that
for all and , .
Let be a proper, lower semi-continuous function that is -convex in the sense of (2.1). If for some , then
for any .
Since is -convex and , we have for any that
By taking we then obtain (2.2). ∎
In convex analysis, the Legendre-Fenchel conjugate is an important notion. Given a proper convex function , its Legendre-Fenchel conjugate is defined by
As an immediate consequence of the definition, one can see, for any , that
If, in addition, is lower semi-continuous, then ([35, Theorem 2.4.4])
For a proper, lower semi-continuous, -convex function, even if it is non-smooth, its Legendre-Fenchel conjugate can have enough regularity as the following result indicates.
Let be a reflexive Banach space and let be a proper, lower semi-continuous function that is -convex in the sense of (2.1). Then , is Fréchet differentiable, and its gradient satisfies
which consequently implies
for any , where is the number conjugate to , i.e. .
See [35, Theorem 3.5.10 and Corollary 3.5.11]. ∎
Finally we conclude this section by providing a result which show that, when is -convex, then, for any , the distance from to can be controlled in terms of .
Let ba a reflexive Banach space and let be a proper, lower semi-continuous function that is -convex in the sense of (2.1). If and satisfy for some , then for any there holds
3. The method
We consider the system
consisting of equations, where, for each , is an operator between two reflexive Banach spaces and . Such systems arise in many practical applications including various tomography problems using multiple exterior measurements.
We will assume that (3.1) has a solution which consequently implies that
In practical applications, instead of we only have noisy data satisfying
with a small noise level . How to use to produce an approximate solution of (3.1) is an important question. In  we proposed a Landweber iteration of Kaczmarz type which makes use of every equation in (3.1) cyclically. In order to capture the feature of the sought solution, general convex functions have been used in  as penalty terms.
We will make the following assumption, where .
According to Assumption 3.1 (c), we can find a constant such that
for all which shows that is continuous on for each .
The formulation of the Landweber iteration of Kaczmarz type in  involves in each iteration step a minimization problem of the form
for any . The convergence result developed there requires to solving (3.6) exactly. The exact solution of (3.6) can be found for some special . However, this minimization problem in general can only be solved inexactly by iterative procedures. Furthermore, numerical simulations indicate that solving (3.6) very accurately in every step does not improve the final reconstruction result. Therefore, it is necessary to formulate a Landweber-Kaczmarz method with inexact inner solver in each iteration step and to develop the corresponding convergence theory.
Concerning the inexact resolution of (3.6), we make the following assumption.
For any given there is a procedure for solving (3.6) such that for any , the element satisfies
Moreover, for each , the mapping is continuous.
In subsection 3.5 we will discuss how to produce the inexact procedure by using concrete examples of including the total variation like convex penalty functions.
3.1. The method with noisy data
We are ready to formulate our Landweber-Kaczmarz method with inexact inner solvers. We will take and let denote the duality mapping over with gauge function . Given an integer , we set .
Algorithm 3.1 (Landweber-Kaczmarz method with noisy data).
Let , and be suitably chosen numbers, and let be a sequence of positive numbers satisfying .
Pick and such that .
Let and . Let and let be a small number. For we define and
We then update
Let be the first integer such that and use as an approximate solution.
In Algorithm 3.1, each is determined by completely without involving , and each is defined by the inexact procedure specified in Assumption 3.2 for solving (3.6) which is independent of , . This splitting character can make the implementation of Algorithm 3.1 efficiently. Furthermore, the definition of implies that
which shows that
We will use this fact in the forthcoming convergence analysis.
We first prove the following basic result which shows that Algorithm 3.1 is well-defined.
Let and be reflexive Banach spaces with being uniformly smooth, let and , satisfy Assumption 3.1, and let be a sequence of positive numbers satisfying
Let be a constant such that . If and are chosen such that
and if is chosen such that , where
then for Algorithm 3.1 there hold
for all ;
the method terminates after iteration steps;
for any solution of (3.1) in there holds
for all . Here we may take because .
Let be any solution of (3.1) in . We first show that, if for some , then
To see this, we consider
which can be written as
Since , we have from (2.3) that
By the definition of we also have
Since , we may use Lemma 2.3 to derive that
Plugging this estimate into (3.1) and using the definition of it follows that
we may use the condition , Assumption 3.1 (c), and the properties of to obtain
By the definition of we can see that
Combining the above two estimates with (3.1) we can obtain
We next consider the term . We claim that
where is the constant defined by (3.13). Indeed, this is trivial when . We only need to consider the case that . If , then
If , we may use the inequality for and to derive that
Thus, by using , we have
We therefore obtain (3.19).