Transceiver Design for Multiuser Multi
antenna Twoway Relay Cellular Systems
Abstract
In this paper, we design interference free transceivers for multiuser twoway relay systems, where a multiantenna base station (BS) simultaneously exchanges information with multiple singleantenna users via a multiantenna amplifyandforward relay station (RS). To offer a performance benchmark and provide useful insight into the transceiver structure, we employ alternating optimization to find optimal transceivers at the BS and RS that maximizes the bidirectional sum rate. We then propose a low complexity scheme, where the BS transceiver is the zeroforcing precoder and detector, and the RS transceiver is designed to balance the uplink and downlink sum rates. Simulation results demonstrate that the proposed scheme is superior to the existing zero forcing and signal alignment schemes, and the performance gap between the proposed scheme and the alternating optimization is minor.
Twoway relay, multiuser, multiantenna, transceiver, cellular systems.
I Introduction
Twoway relay (TWR) techniques have attracted considerable interest owing to its high spectral efficiency. Most of prior works study TWR systems with single user pair, where two users exchange information via a single relay station (RS) [1, 2, 3]. Various transmission schemes have been proposed for single antenna nodes [1] and multiantenna nodes [2, 3].
Recently, the design for TWR systems is extended to multiuser cases [4, 5, 6, 7, 8, 9, 10, 11], which can be roughly divided into two categories based on the system topologies, i.e., symmetric and asymmetric systems. In symmetric systems [4, 5, 6], multiple user pairs exchange information via a RS. In asymmetric systems, a base station (BS) exchanges messages with multiple users [7, 8, 11, 9, 10], which is a typical scenario of cellular networks.
In this paper, we study multiuser TWR cellular system, where a multiantenna BS communicates with multiple singleantenna users bidirectionally via a multiantenna amplifyandforward (AF) relay. Owing to the importance from practical perspective, there is a considerable amount of work on designing transceivers for such a system [8, 11, 9, 10]. However, its transceiver optimization is challenging due to the complicated interference among multiple users in the broadcast and multiaccess phases, and even its bidirectional sum capacity is still not available until now.
Allocating orthogonal time or frequency resources to the uplink and downlink signals of different users is an immediate way to eliminate the interference [7], with which existing singleuser TWR techniques can be directly applied. Since this is far from optimal, a further attempt is to introduce an interference free constraint, which is essentially the zeroforcing (ZF) principle. Though also suboptimal in a sense of sum rate, such a design can capture the inherent degrees of freedom of the system, which is an approximate characterization of the capacity at the high signaltonoise (SNR) level. Along this line, several ZFprinciple based transceivers have been proposed. Considering that the RS is equipped with multiple antennas, a natural solution is to apply ZF transceiver at the RS to separate all the signals from and to the BS and users [8]. This ZF scheme employs orthogonal spatial resources to differentiate different links, thereby the RS should be equipped with enough antennas. To remove all the interference, at least antennas are required at the RS for a system with antennas at the BS and single antenna users. When the RS is only with antennas, the multiple antennas at the BS also need to be exploited to ensure interference free transmission. In [9, 10, 11], the concept of signal alignment (SA) [12] is employed to reduce the number of interference experienced at the relay. The SA scheme exploits the selfinterference cancelation (SIC) [13] ability of TWR. Its basic idea is to project the uplink and downlink signals of each user onto the same spatial direction at the RS through proper BS precoding, such that the RS can separate superimposed signals. After receiving a superimposed signal forwarded by the RS, each user removes its transmitted uplink signal via SIC, and obtains its desired downlink signal.
Both the ZF and SA schemes are based on ZFprinciple. Nonetheless, they are not the only interference free solution^{1}^{1}1By using the terminology “interference free solutions”, we refer to the transmit strategies that can remove all interference. These solutions include the ZF beamforming and ZF detector, the SA scheme, as well as the transmit schemes using orthogonal frequency or time resources, which can null the interference thoroughly.. In fact, by analyzing the feasibility of interference free constraints for multiuser multiantenna TWR cellular systems, it is not hard to show that the SA scheme is the unique solution only for special antenna configurations, and the ZF scheme ensures interference free transmission only when the number of antennas at the RS is sufficiently large. Moreover, both of them are designed as low complexity schemes without taking into account the sum rate.
In this paper, we strive to find a low complexity interference free transceiver towards maximizing sum rate under general antenna settings. To provide a performance benchmark as well as useful insight into the transceiver structure, we employ a standard alternating optimization technique [14] to optimize the BS and RS transceivers aiming at maximizing bidirectional sum rate under interference free constraints. In order to develop a low complexity transceiver scheme, we fix the BS transceiver as the optimal BS precoder and detector in high power region found from the alternating optimization. Based on which we first optimize the RS transceiver to separately maximize the uplink and downlink sum rates and then balance the uplink and downlink sum rates to maximize the bidirectional sum rate. Simulation results show that the balanced scheme performs very close to the alternating optimization solution, and outperforms existing ZF and SA schemes under various scenarios.
The rest of the paper is organized as follows. Section II describes the system model. Section III introduces the alternating optimization solution. The balanced transceiver scheme is proposed in IV. Simulation results are given in section V, and conclusions are drawn in section VI. The major symbols used in the paper are summarized in Table I.
, ,  BS or RS antenna number or user number 

Channel matrix from the BS or from all users to the RS  
Channel vector from the th user to the RS  
Channel matrix from all users other than the th user to the RS.  
It is obtained from with the th column, , being removed.  
,  BS transmit or receive weighting matrix 
,  The th column of or 
RS weighting matrix  
Downlink signal vector transmitted by the BS  
Uplink signal vector transmitted by all users  
RS’s received signal vector in first phase  
BS’s or the th user’s received signal in second phase  
, ,  The transmit power of BS or RS or a single user 
Noise variance  
, ,  Uplink or downlink or bidirectional sum rate 
Identity matrix of size  
, ,  Transpose, conjugate transpose or conjugate of a matrix 
,  Norm or pseudo inverse of a matrix 
Orthogonal subspace of matrix  
if is a wide matrix  
if is a high matrix  
Diagonal matrix whose diagonal elements are the elements of vector  
Mean value of a random variable 
Ii System Model
We consider a multiuser multiantenna TWR system, which consists of a BS equipped with antennas, a RS equipped with antennas and singleantenna users. The BS and multiple users exchange downlink and uplink information via the RS, as shown in Fig. 1. The bidirectional transmission takes place in two phases.
At the first phase, both the BS and multiple users transmit to the RS. The received signal at the RS is given by
(1) 
where is the channel matrix from the BS to the RS, , is the channel vector from the th user to the RS, and are the downlink and uplink signal vectors to and from users and we assume , is the transmit power of each user, is the Gaussian noise vector at the RS with zero mean and covariance matrix , and is the precoder matrix at the BS, which satisfies the transmit power constraint as follows
(2) 
where is the maximal transmit power of the BS.
At the second phase, the RS precodes its received signals and then broadcasts them to the BS and users. The received signals at the BS and the th user are respectively given by
(3)  
(4) 
where is the weighting matrix at the RS, is the receive weighting matrix at the BS, and and are Gaussian noises at the BS and the th user, each with zero mean and variance .
The RS weighting matrix should satisfy the transmit power constraint , which can be rewritten as follows after substituting (1),
(5) 
where is the maximal transmit power of the RS^{2}^{2}2We do not consider power control at the BS and RS. The inequality power constraints are for simplifying the optimization..
Iii Transceiver Design Based on Alternating Optimization
Even after we introduce the interference free constraints, the problem of jointly optimizing BS and RS transceivers that maximizes the bidirectional sum rate of multiuser multiantenna TWR systems is still nonconvex and is very hard to deal with. In this section, we employ a standard tool, alternating optimization [14], to solve the optimization problem, which can serve as a performance benchmark for the interference free transceivers.
Substituting (1) into (3), the received signal at the BS can be rewritten as
(6) 
where the first term is the transmitted signal of the BS in the first phase which can be removed by SIC, the second term is the desired uplink signal, and the last two terms are the noise amplified by the RS and the noise at the BS receiver, respectively.
To eliminate the interference among the uplink signals, the following constraint should be satisfied,
(7) 
where is the th column of .
Substituting (1) into (4), the received signals at the th user can be rewritten as
(8) 
where the first term consists of the downlink signals for all users, the second term consists of the transmitted signals from users in the first phase, and the last two terms are noises.
To remove the interference, the BS and RS transceivers should satisfy the following constraints,
(9)  
(10) 
where is the th column of .
Considering the interuser interference (IUI) free constraints (7), (9) and (10) and the fact that the selfinterference can be canceled [13], the receive SNR of the th uplink and downlink signal can be respectively obtained as,
(11) 
Then the bidirectional sum rate of the TWR system is^{3}^{3}3The received nonwhite noise after amplifying and forwarding is treated as white noise as in existing literature. This is in fact the worst case of the problem, therefore the data rate obtained by can serve as a lower bound.,
(12) 
where and denote the uplink and downlink sum rate, and are the uplink and downlink data rates of the th user, and the prelog factor is due to the halfduplex constraint.
In the following, we optimize one of the three transceiver matrices by fixing the other two.
Iiia Optimization of Weighting Matrix of RS
Here we fix and , and optimize to maximize the bidirectional sum rate under the RS transmit power constraint and the IUIfree constraints by solving the following problem,
(13a)  
s.t.  (13b) 
The bidirectional sum rate is not a convex function of . To solve this nonconvex problem and find the maximum , we employ the concept of rate profile, which is introduced in [15] to characterize the boundary ratetuples of a capacity region. We introduce a vector to specify the rate profile, where and . Then by solving the following optimization problem,
(14a)  
s.t.  (14b)  
(5), (7), (9) and (10),  (14c) 
we will achieve a boundary point of the achievable rate region specified by each vector . After searching the optimal from all its possible values, we can find the optimal boundary point corresponding to the maximum sum rate. For multiuser case, it is too complicated to search all possible . To reduce the complexity, we use bisection algorithm [16] to search the optimal as in [8]. Although it is hard to rigorously prove that the achievable rate region boundary is a convex hull in terms of , simulation results show that bisection algorithm offers the same result as that of using bruteforce searching.
IiiB Optimization of Transmit Weighting Matrix of BS
In this subsection, we design given and . Since the transmit weighting matrix of the BS only affects downlink rate when and are fixed, we design it to maximize the downlink sum rate . The design of should consider the IUI free constraint (9) and the BS transmit power constraint (2). It is also associated with the RS transmit power constraint (5). Then the optimization problem can be formulated as
(15a)  
s.t.  (15b) 
This is also a nonconvex problem, which can be solved by the same method as that we used to solve problem (13). Define a vector , where and . The solution of (15) can be found from solving the following problem by searching the optimal ,
(16a)  
s.t.  (16b)  
(16c) 
Each of the BS and RS power constraints (2) and (5) imposes a constraint on the norm of a linear function of . The IUI free constraint (9) is a linear constraint on . According to [17], the rate tuple constraint (16b) can be converted to linear constraints on . Therefore the constraints in (16) form a secondordercone feasible region [17], and the size of the feasible region depends on . Consequently, we can solve (16) by searching the maximal that guarantees a nonempty feasible region. Bisection method is applied to search . We use the CVX tool[18] to check whether the feasible region is empty or not. If it is not empty, the CVX tool will return a value of in the feasible region. Finally, we will obtain both the maximum value of and the optimal .
IiiC Optimization of Receive Weighting Matrix of BS
Given and , only affects uplink sum rate. Among the three IUIfree constraints, is only associate with (7). Therefore, the optimization problem can be formulated as
(17)  
s.t.  (18) 
According to (III) and (III), the data rate of each uplink stream, , is only a function of . Therefore, this problem can be decoupled into subproblems. Since is a monotonic increasing function of , each subproblem can be formulated as
(19a)  
s.t.  (19b) 
Any feasible should satisfy , where is obtained from channel matrix with the th column being removed. Define as a matrix consisting of all the singular vectors of corresponding to its zero singular values. Then we have
(20) 
where is an arbitrary vector.
By substituting (20) and (IIIC), the optimization problem (19) becomes
(22) 
which is a generalized Rayleigh ratio problem. The optimal is the eigenvector of corresponding to its largest eigenvalue [19].
By now, we have solved the three problems (13), (15) and (17). When we find the alternating optimization solution, we need to assign initial values for the transceiver matrices, which should satisfy all the IUI free constraints and the transmit power constraints. The initial values are set according to the following procedure.
First, constraint (10) can be rewritten as a group of linear equations of as , where denotes Kronecker product, and is the vectorization of a matrix by stacking its columns. The general solution of this equation is given by
(23) 
where is the matrix by stacking all , , is the orthogonal subspace of a matrix, and is an arbitrary vector.
We pick one from the general solution. Then we substitute the chosen into (7) and (9), find general solutions of these two set of equations similar to (23), and pick one and one among the general solutions. Finally, we multiply and with proper scalars to satisfy the BS and RS power constraints.
After assigning the initial values, we alternately optimize one of the three transceiver matrices by fixing the other two. The sum rate must increase with each iteration, otherwise, the iteration is terminated. Due to this requirement, the alternating procedure will surely converge. Because of the nonconvex nature of the optimization problem, the converged solution is not guaranteed to be globally optimal, and depends on the initial values. Nevertheless, we can increase the probability to achieve the maximal bidirectional sum rate by repeating the alternating optimization procedure with multiple random initial values then picking the best solution.
Iv Balanced Transceivers
In this section, we design a low complexity transceiver toward achieving maximal bidirectional sum rate under the interference free constraints. To this end, we decouple the joint optimization of the BS and RS transceivers resorting to the asymptotic analysis in high power region. Specifically, we first find the BS precoder and detector from analyzing asymptotic results of the alternating optimization solution. Then we optimize the RS transceiver based on the given BS transceiver, also in high power region.
Iva BS Transceivers
IvA1 BS Precoder
To obtain a closed form solution, we consider an asymptotic region where the transmit power of RS goes to infinity. When , the RS transmit power constraint can be ignored, then the BS precoder optimization problem in (15) can be reformulated as
(24)  
s.t.  (25)  
(26) 
which can be viewed as linear precoder optimization for downlink multiuser multiantenna system with a channel matrix that maximizes the sum rate under interference free constraints and total transmit power constraint. According to [20], the optimal precoder is a ZF precoder with proper power allocation, i.e.,
(27) 
where is a diagonal power allocation matrix. For simplicity, we consider equal power allocation at the BS, i.e.,
(28) 
IvA2 BS Detector
To obtain a closed form detector, we consider another asymptotic region where the transmit power of the BS or users approaches infinity. When or , the received SNR at the RS in the first phase goes to infinity, then the RS forwarded noise can be neglected^{4}^{4}4This is not true for the case of deep fading where the channel coefficient is approximately zero, but such a case is of low probability.. In this case, in (22) is . By solving the problem (22) and applying (20), the optimal BS receiver vector can be obtained as , where spans the orthogonal subspace of [19]. Therefore, the optimal is the projection of onto the orthogonal subspace of , i.e., the optimal solution is a ZF receiver for the equivalent uplink channel , i.e.,
(29) 
IvB RS Transceiver
Now we find the solution of RS transceiver from (13) given the BS transceivers (27) and (29). The IUI free constraints (7) and (9) are satisfied owing to the usage of ZF transceivers at the BS, and thus can be removed. Note that we consider equal power allocation in the BS precoder, then the optimization problem of RS transceiver can be reformulated as
(30a)  
s.t.  (30b) 
To find a low complexity solution for this nonconvex problem, we decouple it into two subproblems, which respectively maximize the uplink and downlink sum rate. Then we combine these two solutions to maximize the bidirectional sum rate.
When the transmit power of each user goes to zero, i.e., ^{5}^{5}5This is not conflict with the optimality conditions of the ZF transceivers at the BS, which are and either or ,, the system uplink sum rate will approach to zero, then . From (III) and (III), the downlink sum rate does not depend on , therefore the constraint (29) in problem (30) can be removed. Moreover, in this case the RS received signal at the first phase . Then the RS power constraint (5) can be rewritten as . Consequently, the problem (30) reduces to the following problem that maximizes the downlink sum rate,
(31)  
s.t.  (10), (27), (28) and  
(32) 
Similarly, when the BS transmit power goes to zero, i.e., , the problem (30) reduces to the following problem that maximizes the uplink sum rate,
(33)  
s.t.  (10), (29) and  
(34) 
We will first solve these two subproblems, then combine the two solutions of to balance the uplink and downlink rates, so as to maximize the bidirectional sum rate.
IvB1 Design of From Subproblem (31)
We can show that the optimal solution of (31) has the following structure (see Appendix),
(35) 
where is a diagonal matrix and each column of has unit norm, i.e., .
The optimal structure of can be intuitively explained as follows. When , there is only downlink transmission, i.e., the RS receives signals from the BS and then forwards it to the users. In this case, represents the ZF precoder at the RS to broadcast signals to the users, is a power allocation matrix for different signal streams, and is the receive weighting matrix at the RS, which separates the downlink signals from the BS, see Fig. 2.
To obtain the power allocation matrix , we simply let the amplification coefficients at the RS for all streams to be identical. Denote each column of as . Since each downlink data stream is received by , amplified by , and forwarded by , we design to ensure that each has the same norm.
Our next task is to design the receive weighting matrix
. Upon substituting (35), the
optimization problem (31) can be rewritten
as
(36a)  
s.t.  (36b)  
(36c)  
(36d)  
(36e) 
This problem is nonconvex, thereby we turn to find its suboptimal solution. Since acts as the receiver at the RS in downlink transmission, we design it to maximize the “data rate” of BSRS transmission instead of the twophase downlink transmission^{6}^{6}6Since the RS does not decode message in AF protocol, in fact there is no “BSRS transmission data rate”. We use this terminology here for simplifying the optimization problem..
During the BSRS transmission, the halfduplex RS only receives signals from the BS. Therefore, we do not consider the RS transmit power constraint (36), which can be met later by adjusting . Then the BSRS transmission rate maximization problem is formulated as
(37a)  
s.t.  (36b), (36c) and (36d).  (37b) 
Remark 1: If , the objective function (36a) will be the same as (37a) except for the prelog factor , and the RS power constraint (36) can be omitted. This means that the two optimization problems are approximately equivalent when the RS has high transmit power.
Constraint (36c) shows that is a pseudo inverse of with power allocation. Define as the matrix with the th column being removed. Then using the principle of orthogonal projection[19], we obtain that
Substituting this expression and (36d) into (37a), then the problem (37) can be rewritten as
s.t.  (38) 
Constraints (36c) and (36d) are omitted since the objective function does not rely on now.
Solving problem (IVB1) is nontrivial because we need to jointly design all . To obtain a lowcomplexity solution, we employ alternating optimization [14] again. We first initialize . Then we alternately optimize each of the columns of . In each step, we optimize the th column by solving the problem (IVB1) with all other columns being fixed. After each step, we renew the matrix by replacing its th column by the optimized . The procedure stops when the value of objective function in (IVB1) does not increase any more. Simulations show that the procedure always converges after each of the columns has been optimized once.
In the above procedure, we need to solve the optimization problem (IVB1) with fixed . Note that the constraint , can be rewritten as . Any feasible must lie in the orthogonal subspace of . Therefore, we have
(39) 
where is an arbitrary vector. Then the optimization problem (IVB1) with fixed can be rewritten as follows by substituting (39),
s.t.  (40) 
The optimal value of is the left singular vector of corresponding to its largest singular value [19]. Then from (39), we can obtain the optimal .
Substituting the optimization result into (35), the RS weighting matrix designed for maximizing the downlink sum rate can be obtained as .
IvB2 Design of From Subproblem (33)
We can also show that the optimal RS weighting matrix that maximizes the uplink sum rate has the following structure,
(41) 
where and can be obtained similarly as in the last subsection. We do not present the detailed derivation for concision.
RS  BS  
major operations  complexity  major operations  complexity  
ZF scheme  pseudo inverse of a matrix  none  0  
SA scheme  pseudo inverse of a matrix  pseudo inverse of a matrix  
Balanced scheme 


pseudo inverse of matrices 
IvB3 Balancing and to Maximize Bidirectional Sum Rate
Consider that bidirectional sum rate , while and are respectively optimized for and . To improve , we propose the following RS weighting matrix,
(42) 
where the power adjusting factor , , is used for controlling the power proportion to and to balance the uplink and downlink sum rate, is used to meet the total RS transmit power constraint. In practical systems, after obtaining and , the RS can search for an optimal that maximizes the bidirectional sum rate.
Remark 2: In a TWR system with single user and singleantenna BS, turns out to be a maximalratio combination and maximalratio transmission (MRCMRT) weighting matrix. It is shown in [2] that the bidirectional sum rate gap between MRCMRT and the optimal scheme is no more than 0.2bps/Hz. Though in multiuser multiantenna TWR system, we cannot draw the same conclusion via rigorously analysis, the simulation results in section V will show that such a balanced solution performs closely to the alternating optimization solution.
Remark 3: By substituting and into (42), we have
(43) 
As shown in (39), each column of matrix lies in the orthogonal subspace of , i.e., . The th column of also lies in that orthogonal subspace, i.e., .
When , i.e., the number of RS antennas equals to the number of users, the matrix is a matrix. Therefore, the rank of its orthogonal subspace is one. Since both of and lie in the same rank1 subspace, they are linearly dependent, i.e., , where is a scalar. Therefore, we have , where . Substituting this expression into (43), we obtain
(44) 
where is a diagonal matrix.
Comparing (IVB3) with the RS transceiver in the SA scheme proposed in [11, 9, 10], we see that has the same form as that of the SA scheme. Substituting (IVB3) into (27) and (29), it is easy to show that the BS transceivers in our balanced solution also have the same forms as those in the SA scheme. This indicates that the SA scheme is a special case of the balanced solution when . In fact, in such a setting, it is not hard to show that the solution of interference free constraints (7), (9) and (10) is unique, which is exactly the SA scheme.
IvC Complexity Comparison
Here we compare the computational complexities of the balanced scheme and the existing ZF [8] and SA schemes [9, 10, 11].
In the ZF scheme, the major operation at the RS is to compute the pseudo inverse of a matrix. Since all the interference are eliminated by the RS, the BS needs to do nothing. In the SA scheme, the major operations at the RS and the BS are to compute the pseudo inverses of a matrix and a matrix, respectively.
In the balanced scheme, to obtain the RS transceiver (42), we need to compute , , and , and search for the optimal balancing factor . Specifically, we need to perform times of singular vector decomposition (SVD) to alternately design the columns of . According to (IVB1), each SVD is performed for a matrix. Only vector norm operation is required to compute the power allocation matrices and , for which the complexity can be neglected compared with those of the pseudo inverse and SVD. The complexity of finding the optimal can also be ignored, which only requires a scalar searching operation. To obtain the BS transceivers in our balanced scheme (27) and (29), we need to compute the pseudo inverses of two matrices.
A widely used method for computing pseudo inverse is using SVD, which results in a complexity of flops to compute the pseudo inverse of a matrix [21], where . The complexities of the transceiver schemes are present in Table II, which shows that the complexity of the balanced scheme is on the same order as those of the SA and ZF schemes.
V Simulation Results
In this section, we evaluate the performance of the proposed transceivers and compare them with existing schemes by simulations. We assume that all channels are independent and identically distributed Rayleigh fading channels, and all simulation results are obtained by averaging over 1000 MonteCarlo trails. For a fair comparison, we use equal power allocation at the BS and RS in all the transceiver schemes. We assume that the noise variance is identical at the BS, RS and each user. The transmit power of each user . The BS and RS transmit power are and , respectively. We define as the transmit SNR. Without otherwise specified, we set , , , , and dB.
Va Impact of the Adjusting factor
The sum rates of balanced scheme versus the power adjusting factor are shown in Fig. 3, where the upper subfigure shows the uplink and downlink sum rates and the lower subfigure shows the bidirectional sum rate. When , the RS weighting matrix , which aims to maximize the uplink sum rate. Therefore, the system achieves high uplink sum rate but low downlink sum rate in this case. By contrast, when , the system achieves high downlink rate but low uplink rate. By adjusting the value of , the uplink and downlink performance are balanced and higher bidirectional sum rate is achieved. The optimal under this case is 0.5.
VB Convergence of the Alternating Optimization Solution
To study the convergence of the alternating optimization algorithm, we respectively use the proposed balanced transceiver, the ZF and SA transceivers and multiple random weighting matrices as its initial value. When using random matrices as the initial values, we pick one from multiple results that converges to the highest sum rate.
Figure 4 shows the bidirectional sum rate versus the iteration number. The sum rate converges rapidly but the converged result depends on the initial values due to the nonconvexity nature of the optimization problem. Nonetheless, by using multiple random initial values, higher bidirectional sum rate can be achieved. We observe from extensive simulations that when the number of random initial values exceeds 20, the performance gain is marginal. Therefore, we can take the result with 20 random initial values as a nearoptimal result. It is shown that the performance of the balanced transceiver is very close to that of the nearoptimal result. In the following, we will use the balanced transceiver as the initial value for the alternating optimization.
VC Comparison among Different Transceivers
We compare the bidirectional sum rates of alternating optimization solution and the balanced transceiver with those of the ZF [8] and SA schemes [11, 9, 10]. We also compare with a minimummeansquareerror (MMSE) transceiver without the interference free constraints, where the MMSE BS transceiver and MMSE RS transceiver were alternately optimized [22].
Figure 5 shows the impact of the antenna number of the RS, where “AlOpt” denotes the alternating optimization solution. When there are two users, the ZF scheme needs at least 4 antennas at the RS to cancel all the interference, while the SA scheme only needs 2 antennas. From the simulation results, we see that when the sum rate of the ZF scheme reduces sharply due to the residual IUI, but the SA scheme performs much better. When , the ZF scheme becomes superior because it can remove all IUI but the SA scheme suffers from a power loss when aligning the downlink signals with the uplink signals. The sum rate of the balanced transceiver is close to that of the alternating optimization solution, both are higher than the existing ZF and SA schemes for any antenna number at the RS.
Figure 6 shows the impact of the user number on the performance of different transceivers. We set , and . Round robin scheduler is applied, where the scheduled user number is from 1 to 4. It shows that the performance of the ZF scheme degrades severely when because the fourantenna RS can not cancel all IUI. With the SA scheme, the proposed balanced scheme and the alternating optimization solution, the system achieves the highest bidirectional sum rate when three users are scheduled, where both the balanced scheme and alternating optimization result have about 2bps/Hz sum rate gain over the SA scheme. When , we see that the performance of the balanced transceiver and the SA scheme are exactly the same. This agrees well with our earlier analysis in Remark 3.
In Fig. 7 we compare the sum rate of the interference free transceiver schemes with the MMSE transceiver [22]. We can see that our balanced scheme provides higher sum rate than the existing ZF and SA schemes in a wide range of transmit SNR. The MMSE scheme is slightly superior to our balanced scheme in low SNR region, but is inferior to the proposed scheme in high SNR region. This is because the MMSE solution in [22] is obtained via alternating optimization, which is not guaranteed to be globally optimal. In high SNR region, the system is interferencelimited, therefore the proposed scheme outperforms the MMSE solution by removing all the interference.
In Fig. 8, we provide the sum rate under each single channel realization to understand the behavior of the IUI free transceivers. We see that the ZF and SA schemes perform differently for a given channel. The ZF scheme requires the RS to separate all the signals transmitted by the users and BS, and performs well only when the channel vectors from the users and the BS are mutually orthogonal. Contrarily, the SA scheme needs to align the signals transmitted by the BS onto the same directions of the signals transmitted by the users, and thus performs well only when the channel vectors from the users and those from the BS have the same direction. Our balanced scheme can adaptively adjust transmission strategy depending on the channel condition to ensure IUI free without the requirements for channel “orthogonalization” or “alignment”. Therefore, its sum rate is always higher than those of ZF and SA schemes.
In Fig. 9, we compare the outage probabilities of the IUI free transceivers with MonteCarlo trails, where the system is in outage if its bidirectional sum rate drops below a given threshold, which is set as 2bps/Hz. We see that our balanced scheme achieves much lower outage probability than both the ZF and SA schemes. Moreover, since the sum rate of the balanced scheme is always “riding on the peak” of the ZF and SA schemes as shown in Fig. 8, the balanced scheme achieves higher diversity gain, and its outage probability decreases much faster than those of the ZF and SA schemes as the SNR increases.
Vi Conclusion
In this paper, we have designed transceiver for multiuser multiantenna twoway relay systems. We first employed alternating optimization to find the BS and RS transceivers that maximizes the bidirectional sum rate under interference free constraints. We proceeded to propose a low complexity balanced transceiver scheme. By analyzing the solution of the alternating optimization in high transmit power region, we find that zeroforcing BS transceivers are asymptotically optimal. Given the BS transceivers, we designed the RS transceivers by respectively maximizing the uplink and downlink rate, which are then combined with a power adjustment factor to maximize the bidirectional sum rate. Existing signal alignment scheme was shown as a special case of the balance scheme where the relay antenna number equals to the user number. Simulation results showed that the performance gap between the balanced scheme and the alternating optimization solution is minor. In general system settings, the bidirectional sum rate of the balanced transceiver is higher than the existing signal alignment and zeroforcing schemes.
Proof of the Optimal Structure of in (35)
Define as a matrix consisting of the singular vectors of , and as a matrix consisting of the singular vectors of the orthogonal subspace of . Then is a unitary matrix, and can be expressed as
(45) 
where , are two arbitrary matrices.
Since