# Robust THP Transceiver Designs for Multiuser MIMO Downlink with Imperfect CSIT

###### Abstract

In this paper, we present robust joint non-linear transceiver designs for multiuser multiple-input multiple-output (MIMO) downlink in the presence of imperfections in the channel state information at the transmitter (CSIT). The base station (BS) is equipped with multiple transmit antennas, and each user terminal is equipped with one or more receive antennas. The BS employs Tomlinson-Harashima precoding (THP) for inter-user interference pre-cancellation at the transmitter. We consider robust transceiver designs that jointly optimize the transmit THP filters and receive filter for two models of CSIT errors. The first model is a stochastic error (SE) model, where the CSIT error is Gaussian-distributed. This model is applicable when the CSIT error is dominated by channel estimation error. In this case, the proposed robust transceiver design seeks to minimize a stochastic function of the sum mean square error (SMSE) under a constraint on the total BS transmit power. We propose an iterative algorithm to solve this problem. The other model we consider is a norm-bounded error (NBE) model, where the CSIT error can be specified by an uncertainty set. This model is applicable when the CSIT error is dominated by quantization errors. In this case, we consider a worst-case design. For this model, we consider robust minimum SMSE, MSE-constrained, and MSE-balancing transceiver designs. We propose iterative algorithms to solve these problems, wherein each iteration involves a pair of semi-definite programs (SDP). Further, we consider an extension of the proposed algorithm to the case with per-antenna power constraints. We evaluate the robustness of the proposed algorithms to imperfections in CSIT through simulation, and show that the proposed robust designs outperform non-robust designs as well as robust linear transceiver designs reported in the recent literature.

Multiuser MIMO downlink, non-linear precoding, imperfect CSIT, Tomlinson-Harashima precoder, joint transceiver design.

## I Introduction

Multiuser multiple-input multiple-output (MIMO) wireless communication systems have attracted considerable interest due to their potential to offer the benefits of spatial diversity and increased capacity [1],[2]. Multiuser interference limits the performance of such multiuser systems. To realize the potential of such systems in practice, it is important to devise methods to reduce the multiuser interference. Transmit-side processing at the base station (BS) in the form of precoding has been studied widely as a means to reduce the multiuser interference [2]. Several studies on linear precoding and non-linear precoding (e.g., Tomlinson-Harashima precoder (THP)) have been reported in the literature [3],[4]. Joint design of both transmit precoder and receive filter can result in improved performance. Transceiver designs that jointly optimize precoder/receive filters for multiuser MIMO downlink with different performance criteria have been widely reported in the literature [5, 6, 7, 8, 9, 10, 11]. An important criterion that has been frequently used in such designs is the sum mean square error (SMSE) [6, 7, 8, 9]. Iterative algorithms that minimize SMSE with a constraint on total BS transmit power are reported in [6],[7]. These algorithms are not guaranteed to converge to the global minimum. Minimum SMSE transceiver designs based on uplink-downlink duality, which are guaranteed to converge to the global minimum, have been proposed in [8],[9]. Non-linear transceivers, though more complex, result in improved performance compared to linear transceivers. Studies on non-linear THP transceiver design have been reported in the literature. An iterative THP transceiver design minimizing weighted SMSE has been reported in [10]. References [8] and [11], which primarily consider linear transceivers, present THP transceiver optimizations also as extensions. In [12], a THP transceiver design minimizing total BS transmit power under SINR constraints is reported.

All the studies on transceiver designs mentioned above assume the availability of perfect channel state information at the transmitter (CSIT). However, in practice, the CSIT is usually imperfect due to different factors like estimation error, feedback delay, quantization, etc. The performance of precoding schemes is sensitive to such inaccuracies [13]. Hence, it is of interest to develop transceiver designs that are robust to errors in CSIT. Linear and non-linear transceiver designs that are robust to imperfect CSIT in multiuser multi-input single-output (MISO) downlink, where each user is equipped with only a single receive antenna, have been studied [14, 15, 16, 17, 18, 19]. Recently, robust linear transceiver designs for multiuser MIMO downlink (i.e., each user is equipped with more than one receive antenna) based on the minimization of the total BS transmit power under individual user MSE constraints and MSE-balancing have been reported in [20]. However, robust transceiver designs for non-linear THP in multiuser MIMO with imperfect CSIT, to our knowledge, have not been reported so far, and this forms the main focus of this paper.

In this paper, we consider robust THP transceiver designs for multiuser MIMO downlink in the presence of imperfect CSIT. We consider two widely used models for the CSIT error [21], and propose robust THP transceiver designs suitable for these models. First, we consider a stochastic error (SE) model for the CSIT error, which is applicable in TDD systems where the error is mainly due to inaccurate channel estimation (in TDD, the channel gains on uplink and downlink are highly correlated, and so the estimated channel gains at the transmitter can be used for precoding purposes). The error in this model is assumed to follow a Gaussian distribution. In this case, we adopt a statistical approach, where the robust transceiver design is based on minimizing the SMSE averaged over the CSIT error. To solve this problem, we propose an iterative algorithm, where each iteration involves solution of two sub-problems, one of which can be solved analytically and the other is formulated as a second order cone program (SOCP) that can be solved efficiently. Next, we consider a norm-bounded error (NBE) model for the CSIT error, where the error is specified in terms of uncertainty set of known size. This model is suitable for FDD systems where the errors are mainly due to quantization of the channel feedback information [17]. In this case, we adopt a min-max approach to the robust design, and propose an iterative algorithm which involves the solution of semi-definite programs (SDP). For the NBE model, we consider three design problems: ) robust minimum SMSE transceiver design ) robust MSE-constrained transceiver design, and ) robust MSE-balancing transceiver design. We also consider the extension of the robust designs to incorporate per-antenna power constraints. Simulation results show that the proposed algorithms are robust to imperfections in CSIT, and they perform better than non-robust designs as well as robust linear designs reported recently in the literature.

The rest of the paper is organized as follows. The system model and the CSIT error models are presented in Section II. The proposed robust THP transceiver design for SE model of CSIT error is presented in Section III. The proposed robust transceiver designs for NBE model of CSIT error are presented in Section IV. Simulation results and performance comparisons are presented in Section V. Conclusions are presented in Section VI.

## Ii System Model

We consider a multiuser MIMO downlink, where a BS communicates with
users on the downlink. The BS employs Tomlinson-Harashima precoding
for inter-user interference pre-cancellation (see the system model in
Fig. 1). The BS employs
transmit antennas and the th user is equipped with receive
antennas, . Let denote^{1}^{1}1We use the
following notation: Vectors are denoted by boldface lowercase
letters, and matrices are denoted by boldface uppercase letters. ,
, and , denote transpose, Hermitian, and
pseudo-inverse operations, respectively. denotes the
element on the th row and th column of the matrix .
vec operator stacks the columns of the input matrix into one
column-vector. denotes the Frobenius norm,
and denotes expectation operator.
implies is positive
semi-definite. the
data symbol vector for the th user, where ,
, is the number of data streams for the th user.
Stacking the data vectors for all the users, we get the global data
vector The output
of the th user’s modulo operator at the transmitter is denoted
by . Let
represent the precoding matrix for the th user. The global
precoding matrix .
The transmit vector is given by

(1) |

where The feedback filters are given by

(2) |

where , perform the interference pre-subtraction. We consider only inter-user interference pre-subtraction. When THP is used, both the transmitter and the receivers employ the modulo operator, . For a complex number , the modulo operator performs the following operation

(3) |

where , and depends on the constellation [22]. For a vector argument ,

(4) |

The vectors and are related as

(5) |

The th component of the transmit vector is transmitted from the th transmit antenna. Let denote the channel matrix of the th user. The overall channel matrix is given by

(6) |

The received signal vectors are given by

(7) |

The th user estimates its data vector as

(8) | |||||

where is the dimensional receive filter of the th user, and is the zero-mean noise vector with . Stacking the estimated vectors of all users, the global estimate vector can be written as

(9) |

where is a block diagonal matrix with on the diagonal, and . The global receive matrix has block diagonal structure as the receivers are non-cooperative. Neglecting the modulo loss, and assuming , we can write MSE between the symbol vector and the estimate at the th user as [10]

(10) | |||||

where

### Ii-a CSIT Error Models

We consider two models for the CSIT error. In both the models, the true channel matrix of the th user, , is represented as

(11) |

where is the CSIT of the th user, and is the CSIT error matrix. The overall channel matrix can be written as

(12) |

where , and . In a stochastic error (SE) model, is the channel estimation error matrix. The error matrix is assumed to be Gaussian distributed with zero mean and . This statistical model is suitable for systems with uplink-downlink reciprocity. We use this model in Sec. III. An alternate error model is a norm-bounded error (NBE) model, where

(13) |

or, equivalently, the true channel belongs to the uncertainty set given by

(14) |

where is the CSIT uncertainty size. This model is suitable for systems where quantization of CSIT is involved [17]. We use this model in Sec. IV.

## Iii Robust Transceiver Design with Stochastic CSIT Error

In this section, we propose a transceiver design that minimizes SMSE under a constraint on total BS transmit power and is robust in the presence of CSIT error, which is assumed to follow the SE model. This involves the joint design of the precoder , feedback filter , and receive filter . When , the CSIT error matrix, is a random matrix, the SMSE is a random variable. In such cases, where the objective function to be minimized is a random variable, we can consider the minimization of the expectation of the objective function. In the present problem, we adopt this approach. Further, the computation of the expectation of SMSE with respect to is simplified as follows Gaussian distribution. Following this approach, the robust transceiver design problem can be written as

(15) | |||||

subject to |

where is the limit on the total BS transmit power, and minimization over implies minimization over . Incorporating the imperfect CSIT, , in (10), the SMSE can be written as

smse | (16) | ||||

Averaging the smse over , we write the new objective function as

(17) | |||||

Using the objective function , the robust transceiver design problem can be written as

(18) | |||||

subject to |

From (17), we observe that is not jointly convex in , , and . However, it is convex in and for a fixed value of , and vice versa. So, we propose an iterative algorithm in order to solve the problem in (18), where each iteration involves the solution of a sub-problem which either has an analytic solution or can be formulated as a convex optimization program.

### Iii-a Robust Design of and Filters

Here, we consider the design of robust feedback and receive filters, and , that minimizes the smse averaged over . For a given and , as we can see from (17), the optimum feedback filter is given by

(19) |

Substituting the optimal given above in (17), the objective function can be written as

(20) | |||||

In order to compute the optimum receive filter, we differentiate (20) with respect to and set the result to zero. We get

(21) | |||||

From the above equation, we get

(22) | |||||

We observe that the expression for the robust receive filter in (22) is similar to the standard MMSE receive filter, but with an additional factor that account for the CSIT error. In case of perfect CSIT, and the expression in (22) reduces to the MMSE receive filters in [10], [12].

### Iii-B Robust Design of Filter

Having designed the feedback and receive filter matrices, and , for a given precoder matrix , we now present the design of the robust precoder matrix for given feedback and receive filter matrices. Towards this end, we express the robust transceiver design problem in (18) as

(23) | |||||

subject to |

where , , , , , and . Minimization over denotes minimization over . For given and , the problem given above is a convex optimization problem. The robust precoder design problem, given and , can be written as

(24) | |||||

subject to |

As the last term in (24) does not affect the optimum value of , we drop this term. Dropping this term and introducing the dummy variables , the problem in (24) can be formulated as the following convex optimization problem:

(25) | |||||

subject to | |||||

The constraints in the above optimization problem are rotated second order cone constraints [23]. Convex optimization problems like that in (25) can be efficiently solved using interior-point methods [24, 23].

### Iii-C Iterative Algorithm to Solve (15)

Here, we present the proposed iterative algorithm for the minimization of the SMSE averaged over under total BS transmit power constraint. In each iteration, the computations presented in subsections III-A and III-B are performed. In the th iteration, the value of , denoted by , is the solution to the following problem

(26) |

which is solved in the previous subsection. Having computed , is the solution to the following problem:

(27) |

and its solution is given in (22). Having computed and , is the solution to the following problem:

(28) |

and its solution is given in (19). As the objective function in (17) is monotonically decreasing after each iteration and is lower bounded, convergence is guaranteed. The iteration is terminated when the norm of the difference in the results of consecutive iterations are below a threshold or when the maximum number of iterations is reached. We note that the proposed algorithm is not guaranteed to converge to the global minimum.

## Iv Robust Transceiver Designs with Norm-Bounded CSIT Error

When the receivers quantize the channel estimate and send the CSI to the transmitter through a low-rate feedback channel, we can model the error in CSI at the transmitter by the NBE model [17]. In such cases, it is appropriate to consider the min-max design, where the worst-case value of the objective function is minimized. In this section, we address robust transceiver designs in the presence of a norm-bounded CSIT error. Specifically, we consider ) a robust SMSE transceiver design, ) a robust MSE-constrained transceiver design, and a robust MSE-balancing (min-max fairness) design.

### Iv-a Robust SMSE Transceiver Design

Here, we consider a min-max design, wherein the design seeks to minimize the worst case SMSE under a total BS transmit power constraint. This problem can be written as

(29) | |||||

subject to |

The above problem deals with the case where the true channel, unknown to the transmitter, may lie anywhere in the uncertainty region. In order to ensure, a priori, that MSE constraints are met for the actual channel, the precoder should be so designed that the constraints are met for all members of the uncertainty set. This, in effect, is a semi-infinite optimization problem [25], which in general is intractable. We show, in the following, that an appropriate transformation makes the problem in (29) tractable. We note that the problem in (29) can be written as

(30) | |||||

subject to | |||||

where . The first constraint in (30) is convex in and for a fixed value of and vice versa, but not jointly convex in , and . Hence, to design the transceiver, we propose an iterative algorithm, wherein the optimization is performed alternately over and .

#### Iv-A1 Robust Design of and Filters

For the design of the precoder matrix and the feedback filter for a fixed value of , the second term in the left hand side of the first constraint in (30) is not relevant, and hence we drop this term. Invoking the Schur Complement Lemma [26], and dropping the second term, we can write the constraint in (30) as the following linear matrix inequality (LMI):

(32) |

Hence, the robust precoder and feedback filter design problem, for a given , can be written as

(33) | |||||

subject to | |||||

where . From (32), the first constraint in (33) can be written as

(34) |

where

(35) |

, , and
Having reformulated the constraint as in (34), we can
invoke the following Lemma [27] to solve the problem in
(33).

Lemma 1: Given matrices , , with ,

(36) |

if and only if such that

(37) |

Applying Lemma 1, we can formulate the robust precoder design problem as the following convex optimization problem:

(38) | |||||

subject to | |||||

where

(39) |

#### Iv-A2 Robust Design of Filter Matrix

In the previous subsection, we considered the design of the and matrices for a fixed . Here, we consider the robust design for given and . This design problem can be written as

(40) | |||||

subject to | |||||

Applying the Schur Complement Lemma, we can represent the first constraint in (40) as

(41) |

The second inequality in the above problem, like in the precoder design problem, represents an infinite number of constraints. To make the problem in (40) tractable, we again invoke Lemma 1. Following the same procedure as in the precoder design, starting with (41), we can reformulate the robust receive filter design as the following convex optimization problem:

(42) | |||||

subject to |

where

(43) |

where .

#### Iv-A3 Iterative Algorithm to Solve (29)

In the previous subsections, we described the design of and for a given , and vice versa. Here, we present the proposed iterative algorithm for the minimization of the SMSE under a constraint on the total BS transmit power, when the CSIT error follows NBE model. The algorithm alternates between the optimizations of the precoder/feedback filter and receive filter described in the previous subsections. At the th iteration, the value of , denoted by , is the solution to problem (38), and hence satisfies the BS transmit power constraint. Having computed , is the solution to the problem in (42). So , where

(44) |

The monotonically decreasing nature of , together with the fact that is lower-bounded, implies that the proposed algorithm converges to a limit as . The iteration is terminated when the norm of the difference in the results of consecutive iterations are below a threshold or when the maximum number of iterations is reached. This algorithm is not guaranteed to converge to the global minimum.

#### Iv-A4 Transceiver Design with Per-Antenna Power Constraints

As each antenna at the BS usually has its own amplifier, it is important to consider transceiver design with constraints on power transmitted from each antenna. A precoder design for multiuser MISO downlink with per-antenna power constraint with perfect CSIT was considered in [28]. Here, we incorporate per-antenna power constraint in the proposed robust transceiver design. For this, only the precoder matrix design (38) has to be modified by including the constraints on power transmitted from each antenna as given below:

(45) | |||||

subject to | |||||

where . The receive filter can be computed using (42).

### Iv-B Robust MSE-Constrained Transceiver Design

Transceiver designs that satisfy QoS constraints are of interest. Such designs in the context of multiuser MISO downlink with perfect CSI have been reported in the literature [29],[30],[31]. Robust linear precoder designs for MISO downlink with SINR constraints are described in [32]. Here, we address the problem of robust THP transceiver design for multiuser MIMO with MSE constraints in the presence of CSI imperfections. THP designs are of interest because of their better performance compared to the linear designs.

When the CSIT is perfect, the transceiver design under MSE constraints can be written as

(46) | |||||

subject to |

where is the maximum allowed MSE at th user terminal. This problem can be written as the following optimization problem:

(47) | |||||

subject to | |||||

where is a slack variable. With the NBE model of imperfect CSI, the robust transceiver design with MSE constraints can be written as

(48) | |||||

subject to | |||||

In the above problem, the true channel, unknown to the transmitter, may lie anywhere in the uncertainty region. The transceiver should be so designed that the constraints are met for all members of the uncertainty set, . This again, in the present form, is a semi-infinite optimization problem. In the following, we present a transformation that makes the problem in (48) tractable.

The optimization problem in (48) is not jointly convex in
, , and . But, for fixed , it is
convex in and , and vice versa. So, in order to solve
this problem, we propose an alternating optimization
algorithm^{2}^{2}2For the case of single antenna users (i.e., MISO),
a solution based on non-alternating approach is presented in
[19]., wherein
each iteration solves two sub-problems. The first sub-problem given below,
involves the optimization over for fixed .

(49) | |||||

subject to | |||||

The second sub-problem involves optimization over for fixed , as given below

(50) | |||||

subject to | |||||

where are slack variables. The first sub-problem can be expressed as a semi-definite program (SDP), which is a convex optimization problem that can be solved efficiently [23]. Towards this end, we reformulate the problem in (49) as the following SDP:

(51) | |||||

subject to | |||||

where is a slack variable. In the reformulation given above, we have transformed the first constraint in (49) into an LMI using the Schur Complement Lemma [26]. We can show that the LMI in (51) is equivalent to

(52) |

where

, , , and .

Application of Lemma 1 to (52) and (51), as in Sec. IV-A, leads to the following SDP formulation of the first sub-problem:

(53) | |||||

subject to | |||||

Following a similar approach, it is easy to see that the second sub-problem can be formulated as the following convex optimization program:

(54) | |||||

subject to | |||||

The proposed robust MSE-constrained transceiver design algorithm alternates over both sub-problems. In the next subsection, we show that this algorithm converges to a limit.

#### Iv-B1 Convergence

At the th iteration, we compute and by solving the first sub-problem with fixed . We assume that this sub-problem is feasible, otherwise the iteration terminates. The solution of this sub-problem results in and such that , where

(55) |

Also, the transmit power . Solving the second sub-problem in the th iteration, we obtain such that

(56) |

Since the transmit power is lower-bounded and monotonically decreasing, we conclude that the sequence converges to a limit as the iteration proceeds.

### Iv-C Robust MSE-Balancing Transceiver Design

We next consider the problem of MSE-balancing under a constraint on the total BS transmit power in the presence of CSI imperfections. When the CSI is known perfectly, the problem of MSE-balancing can be written as

(57) | |||||

subject to |

This problem is related to the SINR-balancing problem due to the inverse relationship that exists between the MSE and SINR. The MSE-balancing problem in the context of MISO downlink with perfect CSI has been addressed in [30, 33]. Here, we consider the MSE-balancing problem in a multiuser MIMO downlink with THP in the presence of CSI errors. When the CSI is imperfect with NBE model, this problem can be written as the following convex optimization problem with infinite constraints:

(58) | |||||

subject to | |||||

An iterative algorithm, as in Sec. IV-B, which involves the solution of two sub-problems in each iteration can be adopted to solve the above problem. Transforming the first constraint into an LMI by Schur Complement Lemma, and then applying Lemma 1, we can see that the first sub-problem which involves optimization over and , for fixed , is equivalent to the following convex optimization problem:

(59) | |||||

subject to | |||||

The second sub-problem which involves optimization over , for fixed and can be reformulated as in (50). By similar arguments as in the MSE-constrained problem, we can see that this iterative algorithm converges to a limit.

## V Simulation Results

In this section, we present the performance of the proposed robust THP transceiver algorithms, evaluated through simulations. We compare the performance of the proposed robust designs with those of the non-robust transceiver designs as well as robust linear transceiver designs reported in the recent literature. The channel is assumed to undergo flat Rayleigh fading, i.e., the elements of the channel matrices , are assumed to be independent and identically distributed (i.i.d) complex Gaussian with zero mean and unit variance. The noise variables at each antenna of each user terminal are assumed to be zero-mean complex Gaussian. In all the simulations, all relevant matrices are initialized as unity matrices. The convergence threshold is set as .

First, we consider the performance of the robust transceiver design presented in Sec. III for the stochastic CSIT error model. We consider a system with the BS transmitting data streams each to users. In Fig. 2, we present the simulated SMSE performances of the proposed robust design and those of the non-robust design proposed in [10] for different numbers of transmit antennas at the BS and receive antennas at the user terminals. Specifically, we consider three configurations: , , , , and , . We use in all the three configurations. From Fig. 2, it can be observed that, in all the three configurations, the proposed robust design clearly outperforms the non-robust design in [10]. Comparing the results for and , we find that the difference between the non-robust design and the proposed robust design decreases when more transmit antennas are provided. A similar effect is observed for increase in number of receive antennas for fixed number of transmit antennas. It is also found that the difference between the performance of these algorithms increase as the SNR increases. This is observable in (17), where the second term shows the effect of the CSIT error variance amplified by the transmit power. In Fig. 3, we illustrate the SMSE performance as a function of different channel estimation error variances, , for similar system parameter settings as in Fig. 2. In Fig. 3 also, we observe that the proposed robust design performs better than the non-robust design in the presence of CSIT error; larger the estimation error variance, higher is the performance improvement due to robustification in the proposed algorithm.

Next, we present the performance of the robust transceiver designs proposed in Sec. IV for the norm-bound model of CSIT error. Figure 4 shows the SMSE performance of the proposed design in Sec. IV-A as a function of the CSIT uncertainty size, , for the following system settings: , , , , , , and . It is seen that the proposed design in Sec. IV-A is able to provide improved performance compared to the non-robust transceiver design in [10], and this improvement gets increasingly better for increasing values of the CSIT uncertainty size, . In Fig. 5, we illustrate the performance of the robust MSE-constrained design proposed in Sec. IV-B for the following set of system parameters: , , , , and . We plot the total BS transmit power, , required to achieve a certain maximum allowed MSE at the user terminals, . As expected, as the maximum allowed MSE is increased, the required total BS transmit power decreases. For comparison purposes, we have also shown the plots for the robust linear transceiver design presented in [20] for the same NBE model. It can be seen that the proposed THP transceiver design needs lesser total BS transmit power than the robust linear transceiver in [20] for a given maximum allowed MSE. The improvement in performance over robust linear transceiver is more when the maximum allowed MSE is small.

Further, in Fig. 6, we present the total BS transmit power required to meet MSE constraints at user terminals for different values of CSIT uncertainty size , for , , , , and maximum allowed MSEs . As can be seen from Fig. 6, the proposed robust THP transceiver design in Sec. IV-B meets the desired MSE constraints with much less BS transmit power compared to the robust linear transceiver in [20]. We note that infeasibility of robust transceiver design for certain realizations of channels is an issue in robust designs with MSE constraints. In Fig. 7, we show the performance of the proposed THP transceiver design in Sec. IV-B, in terms of the fraction of channel realizations for which the design is infeasible for , ,