Data-Rate Driven Transmission Strategy for Deep Learning Based Communication Systems

Data-Rate Driven Transmission Strategy for Deep Learning Based Communication Systems

Abstract

Deep learning (DL) based autoencoder is a promising architecture to implement end-to-end communication systems. In this paper, we focus on the fundamental problems of DL-based communication systems, including high rate transmission and performance analysis. To address the limited data rate issue, we first consider the error rate constraint and design a transmission algorithm to adaptively select the transmission vectors to maximize the data rate for various channel scenarios. Furthermore, a novel generalized data representation (GDR) scheme is proposed to improve the data rate of DL-based communication systems. Then, we analyze the effect of signal-to-noise ratio (SNR) and mean squared error performance of the proposed DL-based communication systems. Finally, numerical results show that the proposed adaptive transmission and GDR schemes achieve higher data rate and have lower training complexity than the conventional one-hot vector scheme. Both the new schemes and the conventional scheme have comparable block error rate (BLER) performance. According to both theoretical analysis and simulated results, it is suggested that low or wide-range training SNR is beneficial to attain good BLER performance for practical transmission with various channel scenarios.

Autoencoder, communication systems, data rate, deep learning, transmission strategy.

I Introduction

To ensure high demand for various communication applications and services, the next-generation network must be able to deliver enhanced mobile broadband, ultra-reliable and low-latency communications (URLLC), and massive Internet of Things (IoT) ecosystems [1, 2, 3, 4]. The primary concern is to satisfy the exponential rise in the number of user equipments and the traffic capacity of future communication systems. Hence, several promising technologies have been proposed, and they include massive multi-input and multi-output (MIMO) transmissions, millimeter wave (mmWave) communications, ultra-dense networks (UDNs) [5, 6, 7, 8, 9]. However, there exist a number of limitations for conventional communication systems, such as unavailable channel state information in complex transmission scenario, high complexity to process big data, and sub-optimal performance caused by conventional block structure. For this reason, with the significant development of deep learning (DL) [10, 11, 12], researchers are attempting to apply the machine learning (ML), especially DL technologies, to communication system design for new benefits [13, 14, 15, 16] that cannot be obtained using the conventional approaches.

As a promising technique, deep learning applies a useful and insightful way to implement communication systems using deep neural networks (NNs). Different from the conventional communication system that consists of multiple independent blocks (e.g., source/channel coding, modulation, channel estimation, equalization), the DL-based communication system can jointly optimize transmitter and receiver for end-to-end performance without block structure [17, 18]. DL-based system design is promising for future communications with the following reasons:

  1. A DL-based communication system is optimized for end-to-end performance by using deep NNs, which is fundamentally different from the block-structure conventional communication systems with suboptimal performance.

  2. A DL-based communication system can be optimized for a practical system over any type of channel without requiring a mathematically tractable model, including the channel models that take into account of different transmission scenarios and non-linearities.

  3. DL algorithms provide faster processing speed than conventional communication algorithms, since the execution of NNs can be highly parallelized on concurrent architectures and can be implemented with low-precision data types [19].

Attracted by these advantages, there have been a number of studies on DL-based communication and signal processing using state-of-the-art tools and hardware [20, 21, 22, 23, 17, 24, 25, 18, 26, 27, 28, 29, 30, 31, 32, 33, 34]. The DL method is used to deal with certain challenges in existing communication systems. For example, the DL-based belief propagation algorithm was originally used to improve the performances of channel decoding, where low complexity and near optimal decoder performance were obtained [20, 21, 22]. Around the same time, autoencoder was developed to address the problem of learning an efficient physical layer [23]. In DL theory, an autoencoder describes a deep NN in order to find a low-dimensional representation of its input at certain intermediate layer that allows reconstruction at the output with minimal error [35, Ch. 14]. The DL-based communication system can be represented and implemented by an autoencoder that is trained using the dataset off line. Then, the trained autoencoder can be directly applied to practical systems on line. A DL-based communication system interpreted as an autoencoder performs an end-to-end reconstruction task that jointly optimizes transmitter and receiver as well as learns signal encoding [23, 17, 24, 32]. Considering the advantages of the autoencoder, a complete communication system represented as an autoencoder was proposed to address the challenges of frame synchronization [18, 26], where a competitive performance can be achieved even without extensive hyperparameter tuning. More recently, DL-based algorithm has been used to solve the channel state information feedback and channel estimation problems in massive MIMO systems, and it outperforms the state-of-the-art compressive sensing based algorithms [27, 28, 29].

For future communication systems, there is a huge demand for data rate due to increased number of communication devices and equipment types, and improved quality of services (QoS). Consequently, high data-rate schemes should be developed in DL-based communication systems for future wireless networks. However, in [17, 15, 18, 24, 25, 26, 22, 32], one-hot vector, being the only data representation, has a low data rate in DL-based communication systems. The reason is that an one-hot vector consists of s in all entries with the exception of a single [36], e.g., , and there are only possible transmitted messages that lead to limited data rate, which is a barrier for developing DL-based communication systems in the future. Besides, the autoencoder with one-hot vector is trained with a fixed vector size , which has low flexibility for various communication systems. On the other hand, the conventional autoencoder is trained under a fixed signal-to-noise ratio (SNR) with unrealistic expectation to operate well for a wide range of SNR values in practical transmission scenarios. It was reported that training the autoencoder at different SNR values will lead to various autoencoder performances [17], but there is yet a detailed study on the effect on such a system. Therefore, our objective is to design a new transmission scheme and replace the conventional one-hot vector scheme in order to achieve high data rate and flexibility. As well, we will investigate the effect of training SNR on the performance of DL-based communication systems.

In this paper, an adaptive transmission algorithm is first designed for different communication scenarios to maximize the data rate in DL-based communication systems with a QoS constraint. Then, we propose a generalized data representation (GDR) scheme to improve the data rate of DL-based communication systems. Finally, we analyze the effect of SNR and mean squared error (MSE) performance in DL-based communication systems. Comparable block error rate (BLER) performance can be achieved by the proposed transmission schemes with low complexity and high data rate, when compared with the conventional DL-based communication system1.

The major contributions of this paper are summarized as follows:

  1. In DL-based communication systems, we formulate the data rate problem of the conventional one-hot vector scheme. To address this issue, we design an adaptive transmission algorithm for different channel scenarios, where the QoS constraint is considered. In the proposed algorithm, the optimal transmission vectors are adaptively selected for different SNR conditions, where the goal is to maximize the data rate with the constraint on MSE performance. It is shown that the proposed adaptive transmission algorithm has large BLER performance gain compared with the conventional one-hot vector scheme with the same data rate.

  2. Furthermore, a generalized data representation scheme is proposed to replace the conventional one-hot vector scheme. The proposed scheme can improve the data rate in DL-based communication systems. As expected, high data rate is obtained by our GDR scheme with comparable BLER performance and low complexity. To the best of the authors’ knowledge, this is the first time that the GDR scheme is proposed and its effectiveness is also verified.

  3. We investigate the effect of SNR on the system performances in DL-based communication systems. Simulation results show that the high training SNR can improve the convergence performance in training, but it can also degrade the BLER performance in practical transmission. As a tradeoff, we introduce a wide-range training SNR strategy, which shows great performance balance in convergence and BLER performance. Furthermore, it is proved that training the autoencoder at low SNR can achieve BLER and MSE performance gains when the trained autoencoder is applied to high SNR scenario. These results provide a reliable guidance to select the suitable training SNR for the optimal system performance.

The rest of this paper is organized as follows. In Section II, we describe the system model of the DL-based communication system. Section III presents an adaptive transmission algorithm. Section IV proposes the generalized data representation scheme for DL-based communication systems. Section V investigates the effect of SNR and analyzes the MSE performance of the autoencoder. In Section VI, we show the numerical results of the proposed schemes and system performances. Section VII concludes this paper.

We use the following notations throughout this paper: is a matrix; is a vector; is a scalar; is a set; is the Frobenius norm of matrix ; is the th row of matrix ; is the th element of vector a; is the zero vector; is the identity matrix; is used to denote expectation; is the floor operation; denotes the number of combinations when choosing out of .

Ii Deep Learning-Based Communication Systems

In this section, we describe the DL-based autoencoder for end-to-end communication systems and then provide the research motivations of this paper.

Ii-a Autoencoder for End-to-End Communication Systems

Fig. 1: A DL-based communication system represented as an autoencoder with its NN structure [17].
Name Function
Activation
functions
Linear
ReLU max
Softmax
Loss
functions
MSE
Categorical cross-entropy
TABLE I: Activation Functions and Loss Functions

We consider a DL-based communication system represented as an autoencoder that consists of transmitter, channel, and receiver as shown in Fig. 1, where its NN structure is shown below. The autoencoder describes a deep NN that applies unsupervised learning in order to reconstruct the input at the output [35, Ch. 14]. At the transmitter, the message is first transformed to a vector after the vector expression processing, where . For example, if the message is transmitted, the corresponding vector expression is a one-hot vector in a conventional DL-based communication system [17]. Then, the multiple dense layers, including rectified linear unit (ReLU) layer and linear layer, apply the transformation to produce the transmitted signal for discrete channel uses [18]. The commonly used activation functions are shown in TABLE I. Finally, the normalization layer ensures the power constraint of the transmitted signal as ().

The Rayleigh fading channel is implemented by a noise layer with its output being the received signal that can be given by

(1)

where is the fading coefficient with a Rayleigh distribution, denotes the additive white Gaussian noise (AWGN) vector with a fixed variance , and where is the data rate, is the energy per bit, and denotes the noise power spectral density. Notably, there is no complex operation in the existing NN architectures, and the complex number is represented by two real numbers [17]. Here, we assume that the channel coefficients are real-valued. Furthermore, the autoencoder-represented communication system is available for any type of channel with/without a mathematically tractable model [17, 18].

At the receiver, the received signal is passed through the ReLU layer to realize the transformation . The last layer of the receiver has a softmax activation as shown in TABLE I, which is a generalization of the logistic function that compresses an -dimensional vector of arbitrary real values to an -dimensional probability vector , where each element lies in the range (0, 1], and all the elements add up to 1 [35]. For the autoencoder scheme with conventional one-hot vector, the estimated message is derived from the index of the element having the highest probability in . Here, the BLER of DL-based communication systems is defined as

(2)

The autoencoder based communication system can be trained on a large training dataset off line, while the iterative training process depends on the value of loss function in each iteration. The most common loss functions are MSE and categorical cross-entropy as shown in TABLE I, and these loss functions are determined by the vector expression and the probability vector . The training parameters of the autoencoder are produced to minimize the loss function. Furthermore, the trained autoencoder with the fixed parameters is applied to various practical communication scenarios on line.

Ii-B Motivations

One-hot vector is the conventional data representation with only one non-zero element. Thus, the data rate of the conventional DL-based communication system with one-hot vector is limited to

(3)

Over the last few years, the demand for high data rates has experienced unprecedented growth in communication systems [1, 2]. Therefore, providing a high data rate is essential for DL-based communication systems in future communications.

To improve the data rate, two different research problems can be formulated as follows:

  1. Adaptive transmission scheme. For the conventional one-hot vector scheme, the DL-based autoencoder is trained over a fixed-size transmission vector with dimension at fixed SNR scenario, which leads to two main limitations. On the one hand, the trained autoencoder for a certain value of cannot work in the scenarios with different values of . On the other hand, the performance of DL-based communication system will not be optimal when the trained autoencoder is applied to different SNR scenarios. For these reasons, there is a need for an adaptive transmission strategy for the autoencoder to improve the applicability and optimize the system performances, such as maximizing the data rate and satisfying the QoS constraint [37, 38].

  2. Generalized data representation scheme. From the definition of the data rate , it is obvious that, for the same channel environment, the data rate is proportional to the number of bits being conveyed. However, the size of transmission vector cannot be infinite due to the high complexity associated with deep NNs. Therefore, a new data representation scheme is required to meet the high data rate demands in future communication systems.

Considering the two aspects above, data-rate driven transmission strategies are required for DL-based communication systems to be employed in the future networks.

As for system performances, the autoencoder that is trained using a fixed SNR off line is expected to have a robust performance for a wide SNR region on line. In [17], it was found that an unaccommodated training SNR will result in the performance degradation of DL-based communication systems, but there is little theoretical analysis. Consequently, the effect of the training SNR needs to be investigated and a reliable criterion needs to be developed for selecting training SNR. Furthermore, it is necessary to provide more theoretical performance analysis for DL-based communication systems as the fundamental principles for future study.

Iii Adaptive Transmission Strategy

Considering the high data rate and general applicability, adaptive transmission technology can be employed in the DL-based communication system. In this section, the adaptive transmission algorithm is presented to maximize the data rate with the MSE constraint for different channel scenarios.

Fig. 2: Adaptive transmission scheme applied to the DL-based communication system on line.

In Fig. 2, the adaptive transmission algorithm is considered to adaptively design the on-line transmission strategy for DL-based autoencoder represented communication systems. Before the on-line transmission, the autoencoder has been trained with one-hot vectors over a fixed training SNR () off line, while should be suitably large2, such as . As a result, the trained autoencoder used in Fig. 2 is produced. First, for the adaptive transmission algorithm in Fig. 2, all one-hot vectors in are transmitted through the trained autoencoder over the practical channel with a practical SNR value (); Second, the receiver calculates the MSE of each one-hot vector and sends the MSE of the th vector () back to the transmitter; Then, the transmitter sorts the MSE results and selects one-hot vectors from defined as , where is the label of the selected one-hot vector. The selection goal is to maximize the data rate and satisfy the MSE requirement as

(4)

where is less than or equal to , and is a preset MSE threshold. Finally, the selected one-hot vectors are used for the autoencoder on line over the current channel with .

The main steps of the adaptive transmission algorithm are summarized as follows:

 

Algorithm 1 Steps of the Adaptive Transmission Algorithm

 

  1. Train the autoencoder with all possible one-hot vectors off line.

  2. Each one-hot vector is transmitted through the trained autoencoder over the practical channel.

  3. Calculate the practical MSE of each vector and select according to (4).

  4. Transmit the message by using the selected vectors over the current channel on line.

 

Iv Generalized Data Representation Scheme

In this section, we propose a generalized data representation scheme to improve the data rate for DL-based communication systems.

Instead of the conventional one-hot vector, bit vector is first considered to improve the data rate for DL-based communication systems. An -order bit vector is defined as

(5)

where indicates the number of non-zero elements. The bit vector provides possible messages for the transmission. In general, the number of possible symbols in the constellation diagram is a power of 2. For this reason, we only select out of possible symbols for communication.

Furthermore, for the autoencoder shown in Fig. 1, the vector expression at the transmitter is the practical probability distribution, and the probability vector at the receiver is the corresponding estimated probability distribution. The training goal of the autoencoder is optimizing to reconstruct through minimizing the loss function.

Thus, combining the form of bit vector in (5), we propose a generalized data representation as a probability distribution

(6)

where the estimated message can be derived from the indices of elements with the highest probabilities in . The conventional one-hot vector is a special case of the proposed GDR scheme when .

The data rate of the DL-based communication system can be improved by employing the proposed GDR as

(7)

When , the data rate of the conventional one-hot vector scheme in (3) is obtained. The data rate increases with , while value is suitably chosen and remains fixed. The performance gain of the proposed GDR scheme will significantly increase when the vector size increases.

The channel capacity of the proposed GDR scheme in the DL-based communication system is derived as

It is obvious that the capacity can be improved by using the proposed GDR in the DL-based communication system.

V Performance Analysis of the Autoencoder

In this section, we provide a theoretical analysis of MSE performance for DL-based communication systems.

V-a MSE Performance Analysis

In Fig. 1, the output of the ReLU layer at receiver can be written as

(9)

where , and denote the trainable parameters of the ReLU layer, and they are defined as

(10)

respectively, where represents the symmetric interaction term between unit and unit in Fig. 1, and is the bias term. Thus, from (1) and (9), the th element of is given by

(11)

where is the th row of .

Next, a probability vector is derived from the softmax function at the receiver, and its th element can be written as

(12)

From (11)-(12), in the off-line training processing, different will lead to different trainable parameters and , which will have effect on in (11). As a result, , the probability of the th element is directly affected by the training SNR. Also, in the on-line practical transmission, the trainable parameters and are constant since the autoencoder has been trained. When the autoencoder is applied to a different practical SNR scenario, it will lead to a different estimated probability vector as well. The effect of SNR will also be analyzed through simulations.

In Appendix A, it is shown that, based on (12), the probability vector at the receiver in Fig. 1 can be approximated as

(13)

where is a diagonal matrix that is equivalent to the effect of softmax activation layer.

At the receiver, the output of the ReLU layer consists of zero and non-zero elements as shown in (11). In this paper, we aim to analyze the effect of SNR on MSE performance. While the zero elements cannot reflect the characteristic of MSE, the non-zero output of the ReLU layer is considered and can be derived from (11) as

(14)

if

(15)

Thus, the probability vector under the assumption of (15) can be expressed as

(16)

where is the equivalent matrix of softmax activation layer in the non-zero case as (15).

Here, the average MSE of the DL-based communication system in the case of (15) can be given from (14) and (16) as

(17)

where is the variance of the Rayleigh fading coefficient , and step is due to . After the autoencoder is trained over , the transformation matrices of the autoencoder are constant, where is the noise variance at the training scenario. When the trained autoencoder is applied to the practical communication scenario with , the noise variance of the current practical channel scenario is . For the non-zero case, it can be observed from (17) that, when , the practical MSE performance will be better than that of the training scenario; when , the converse is true. It indicates that the trained autoencoder can attain better system performance when it is applied to higher SNR scenario. For the zero case in (11), the variance of noise has no effect on the MSE performance. The MSE performance of the DL-based communication system will also be verified through simulations.

V-B Wide-Range Training SNR Strategy

In conventional DL-based communication systems, the autoencoder is trained over a fixed SNR off line, which leads to limited generalization performance while facing with various communication scenarios. Here, we propose a wide-range training SNR strategy by employing multiple training SNRs, and it will improve the diversity of training dataset aiming to obtain a great generalization performance. For example, the wide-range training SNR can be set to dB for off-line training. Also, the system performance gain of the proposed wide-range training SNR strategy will be shown by simulation results.

Vi Numerical Results

Parameter Value
Optimizer Adam [39]
Loss function MSE
Epoch 150
Batch size 45
Trained samples 2
Test samples 1
TABLE II: Parameters for the autoencoder setup

In this section, we evaluate the numerical results of the proposed adaptive transmission algorithm, the GDR scheme, and the system performances in the DL-based communication system via simulations on the TensorFlow framework. In all simulations, the autoencoder is trained over the stochastic AWGN channel model with channel uses without exhaustive hyperparameter tuning. Here, we use the same set of parameters for the autoencoder setup as described in TABLE II. Notably, it has been proved that the DL-based communication system with one-hot vector can obtain competitive system performance compared to the conventional communication system in [25, 18]; therefore, we will not compare with the conventional communication system.

Vector
size
Multiple
dense layers
Normaliz-
ation layer
ReLU
layer
Softmax
layer
Total
Simulated
parameters
Theoretical
parameters
TABLE III: Training parameters of autoencoder

TABLE III presents the simulated and theoretical number of training parameters in autoencoder, where different size of the data representation is employed. From TABLE III, it is clear that the simulated number of trainable parameters increases with from to , not only for the total number but also for the number of each layer except for the normalization layer. The simulated results agree with the theoretical number of parameters as shown in the last row of TABLE III. The increasing training parameters lead to an increasing complexity for training. For the conventional one-hot vector, the data rate can be improved by increasing as shown in (3) at the cost of high complexity. While, the data rate of the proposed GDR scheme can be improved by controlling the number of non-zero elements with a small as shown in (7), which leads to a low complexity.

Vi-a Performance of the Proposed Adaptive Transmission Scheme

In this subsection, we show the simulated BLER and MSE performance of the proposed adaptive transmission scheme in the DL-based communication system. Here, the autoencoder is trained when the training SNR is dB.

Fig. 3: Simulated BLER for the autoencoder with conventional one-hot vector and proposed adaptive transmission schemes, while the MSE thresholds are , and .

Figure 3 depicts the simulated BLER performance of the DL-based autoencoder that employs the proposed adaptive transmission scheme and the conventional one-hot vector scheme, where the MSE thresholds are , , and . First, it can be seen from Fig. 3 that, the BLER of the conventional one-hot vector scheme increases when is varied from to , since the smaller requests less trainable parameters as shown in TABLE III. With the same training dataset, the less trainable parameters contribute to better training accuracy. Second, for the proposed adaptive transmission scheme, the BLER increases when the MSE threshold is increased from to in Fig. 3. The reason is that, to maximize the data rate, the lower MSE threshold (means the tighter bound) requires smaller to satisfy the MSE constraint, which results in lower BLER. Then, for each MSE threshold, the number of selected vectors adaptively increases from to with the increasing practical SNR at the -axis. For example, while , the adaptively changes with SNR as . For this reason, higher practical SNR makes it easy to obtain the MSE requirement and leads to larger to maximize the data rate. At last, when the data rate is the same in Fig. 3, the BLER performance of the adaptive transmission scheme is better than that of the one-hot vector scheme. The reason for the performance gain is that the proposed adaptive transmission algorithm can select the optimal vectors that meet the MSE requirement as shown in (4).

Fig. 4: Data rate performance for the autoencoder with conventional and adaptive transmission schemes, while the MSE thresholds are , and .

Figure 4 illustrates the data rate performance of the autoencoder that employs the conventional one-hot vector scheme and the proposed adaptive transmission algorithm with the MSE thresholds being , and . From Fig. 4, we see that the data rates of the conventional one-hot vector scheme are constant for all practical values of SNR, which cannot obtain the maximum data rate with the great BLER performance at the same time. However, in Fig. 4, the data rate of the proposed adaptive transmission scheme increases with SNR due to the increasing as shown in Fig. 3, which is consistent with (4). By comparing Fig. 3 with Fig. 4, the proposed adaptive transmission algorithm can obtain the better BLER performance than that of the conventional one-hot vector scheme when operating at the same data rate.

Fig. 5: Simulated MSE for the autoencoder employing the adaptive transmission scheme with the MSE thresholds being , and .

Figure 5 presents the simulated MSE performance for the practical communication system that employs the proposed adaptive transmission algorithm with the MSE thresholds being , and . It is seen from Fig. 5 that the simulated MSE of the proposed adaptive transmission scheme increases with the MSE threshold. Furthermore, the simulated MSE of the proposed scheme decreases while the practical SNR increases, which is consistent with the result in (17). As expected, when the simulated MSE reaches the corresponding MSE threshold, the number of the selected vectors is almost which is the maximum value, and the maximum data rate is obtained.

Vi-B Performance of the Proposed GDR Scheme

This subsection shows the BLER performance and the channel capacity of the proposed GDR scheme in the DL-based communication system, where the training SNR is dB.

Fig. 6: Simulated BLER for the autoencoder with different data representations, while the trained SNR is 5 dB.

Figure 6 shows the simulated BLER performance of the DL-based communication system that employs the proposed GDR and conventional one-hot vector schemes for comparison. There are a number of observations can be seen from Fig. 6. First, with the same data rate (bits/channel use) including the proposed schemes with , with , and the conventional scheme with , the proposed GDR schemes have better BLER performance with lower complexity than that of the conventional one-hot vector scheme. Obviously, the BLER decreases with the vector size for the same reason as that in Fig. 3. Second, with the same vector size , the proposed GDR schemes ( and ) obtain comparable BLER performances when compared to the conventional one-hot vector scheme (). It indicates that, with the same vector size, the number of non-zero elements has little effect on the BLER performance. With the similar BLER, the data rates of the GDR schemes are (bits/channel use) while and (bits/channel use) while . In both cases, the data rate is greater than that of the conventional one-hot vector scheme as (bits/channel use). Third, the simulated BLER is less than when the SNR is greater than 2 dB in Fig. 6, which demonstrates that the autoencoder obtains a high accuracy by sufficient training over dB.

Fig. 7: Channel capacity for the autoencoder with different data representations.

Figure 7 illustrates the channel capacity of the DL-based communication system employing different data representations. It can be seen from Fig. 7 that, with , the channel capacity increases when the order increases from to , which is consistent with the result in (IV). This shows that the proposed GDR scheme can obtain a remarkable channel capacity improvement. Notably, the performance gain of the proposed GDR scheme is increased when the vector size increases. As shown in Fig. 7, the GDR scheme employing with has a great performance gain when compared with the conventional scheme employing with . Besides, the channel capacity of proposed GDR schemes ( with and with ) is the same as that of the conventional one-hot vector scheme (, ) in Fig. 7. To obtain the same capacity with GDR scheme, the conventional one-hot vector scheme needs to improve the vector size , which leads to a BLER performance degradation as shown in Fig. 6.

Vi-C Performance Comparison of Different Training SNR

Fig. 8: Simulated loss function performance of autoencoder in training process, while different fixed SNRs and wide-range SNR are employed.

In this subsection, we investigate the effect of training SNR on system performance for the DL-based communication system, including the loss function performance in training process, and the simulated BLER as well as MSE performances in practical transmission process. Here, the data representation parameters are and .

Figure 8 shows the simulated loss function performance in training processing, while the autoencoder is trained over different SNRs and wide-range SNR. The wide-range SNR is set as dB, which includes all the fixed SNRs except for dB. Here, an epoch is the process that training dataset totally passes through the autoencoder once. In Fig. 8, with the fixed training SNR increasing from dB to dB, the loss value decreases and the convergence of loss function gets better, which indicates that the good channel environment contributes to improve the training performance. However, with dB, the loss value does not converge within epoches. Furthermore, it can be seen from Fig. 8 that, the loss value of the autoencoder training with wide-range SNR is similar to that of the autoencoder training with dB. The simulated results suggest that the training SNR has significant effect on the training performance of the autoencoder.

Fig. 9: Simulated BLER for the DL-based communication system employing the trained autoencoder with different fixed training SNRs and wide-range training SNR.

Figure 9 depicts the simulated BLER performance of the practical DL-based communication system employing the trained autoencoder with different fixed training SNRs and wide-range training SNR. In Fig. 9, the BLER decreases with the fixed training SNR ranging form dB to dB. The reason is that, with the lower training SNR (that is to say the worse channel environment), the autoencoder needs to learn more features to reconstruct the input at the output, which leads to a robust autoencoder and better BLER performance. However, the training SNR has a lower bound for the autoencoder. As shown in Fig. 9, while the training SNR is dB, the BLER is approximately 0.6, which demonstrates that the autoencoder trained over this channel environment cannot learn the features anymore. It is consistent with the non-convergence performance of the loss function with dB in Fig. 8. Besides, Fig. 9 shows that the BLER performance of the wide-range training SNR scheme is similar to that of dB scheme, which is almost the best performance except for the dB scheme. It shows that training with wide-range SNR can improve the generalization performance of the autoencoder. Combining Fig. 8 with Fig. 9, it can be found that, with the higher training SNR, the autoencoder obtains better convergence performance in training but worse BLER performance in testing. The simulated results indicate that the training SNR will directly affect the system performance, which agrees with the analysis in Subsection V-A.

Fig. 10: Simulated MSE for the DL-based communication system employing the trained autoencoder with different fixed training SNRs and wide-range training SNR.

Figure 10 illustrates the simulated MSE performance of the practical DL-based communication system employing different trained autoencoders, while the training SNRs include different fixed SNRs and wide-range SNR. In Fig. 10, it is clear that the MSE decreases when the practical SNR at the -axis increases. It indicates that the MSE performance improves when the trained autoencoder is applied to a higher SNR scenario, which is consistent with the analysis in (17). Furthermore, the simulated MSE performance in Fig. 10 is similar to the BLER performance as shown in Fig. 9 for the same reasons.

Vii Conclusion

In this letter, we proposed new transmission schemes to address the problem of limited data rate and investigated the system performances of the DL-based communication system. We designed an adaptive transmission scheme for different channel scenarios to maximize the data rate with the error rate constraint. Furthermore, the GDR scheme was proposed, and it can obtain higher data rate and comparable BLER performance with a lower complexity when compared with the conventional one-hot vector scheme. Besides, the effect of training SNR and MSE performance were analyzed and verified by simulations. We discovered that high training SNR can lead to good convergence in training process but worse BLER performance for practical transmission. We also introduced a wide-range training SNR strategy to address the tradeoff between convergence and error rate. On the other hand, it is shown that the autoencoder trained over a low SNR can attain better BLER and MSE performances when operating in the high SNR region. As a result, it is concluded that training the autoencoder at low SNR will lead to good system performance.

Appendix A Derivation of (13)

Let

(18)

According to (12), eq. (18) can be formulated as

(19)

and it can be obtained that

(20)

Next, can be approximated according to the Taylor’s theorem as

(21)

where is a sufficiently large integer.

Finally, combining (20) and (21), the elements of matrix can be derived as

(22)

Thus, eq. (22) shows that the probability vector at the receiver can be approximated as (13).

Footnotes

  1. Notably, throughout this paper, the conventional DL-based communication system refers to the autoencoder based communication system that employs the one-hot vector.
  2. If is too large, the training complexity is prohibitive since the autoencoder must see every message once at least [17].

References

  1. F. Boccardi, R. W. Heath, A. Lozano, T. L. Marzetta, and P. Popovski, “Five disruptive technology directions for 5G,” IEEE Commun. Mag., vol. 52, no. 2, pp. 74–80, Feb. 2014.
  2. M. Agiwal, A. Roy, and N. Saxena, “Next generation 5G wireless networks: A comprehensive survey,” IEEE Commun. Surveys Tut., vol. 18, no. 3, pp. 1617–1655, 3rd Quart. 2016.
  3. P. Popovski, “Ultra-reliable communication in 5G wireless systems,” in Proc. 1st Int. Conf. 5G Ubiq. Connect., Nov. 2014, pp. 1–6.
  4. Z. Dawy, W. Saad, A. Ghosh, J. G. Andrews, and E. Yaacoub, “Toward massive machine type cellular communications,” IEEE Wireless Commun., vol. 24, no. 1, pp. 120–128, Feb. 2017.
  5. J. Hoydis, S. ten Brink, and M. Debbah, “Massive MIMO in the UL/DL of cellular networks: How many antennas do we need?” IEEE J. Sel. Areas Commun., vol. 31, no. 2, pp. 160–171, Feb. 2013.
  6. E. G. Larsson, O. Edfors, F. Tufvesson, and T. L. Marzetta, “Massive MIMO for next generation wireless systems,” IEEE Commun. Mag., vol. 52, no. 2, pp. 186–195, Feb. 2014.
  7. Z. Pi and F. Khan, “An introduction to millimeter-wave mobile broadband systems,” IEEE Commun. Mag., vol. 49, no. 6, pp. 101–107, June 2011.
  8. X. Ge, S. Tu, G. Mao, C. Wang, and T. Han, “5G ultra-dense cellular networks,” IEEE Wireless Commun., vol. 23, no. 1, pp. 72–79, Feb. 2016.
  9. H. Zhang, Y. Dong, J. Cheng, M. J. Hossain, and V. C. M. Leung, “Fronthauling for 5G LTE-U ultra dense cloud small cell networks,” IEEE Wireless Commun., vol. 23, no. 6, pp. 48–53, Dec. 2016.
  10. X. Chen and X. Lin, “Big data deep learning: Challenges and perspectives,” IEEE Access, vol. 2, pp. 514–525, May 2014.
  11. J. Schmidhuber, “Deep learning in neural networks: An overview,” Neural Networks, vol. 61, pp. 85–117, 2015.
  12. Q. Mao, F. Hu, and Q. Hao, “Deep learning for intelligent wireless networks: A comprehensive survey,” IEEE Commun. Surveys Tut., vol. 20, no. 4, pp. 2595–2621, 4th Quart. 2018.
  13. D. Yu and L. Deng, “Deep learning and its applications to signal and information processing, exploratory DSP,” IEEE Signal Process. Mag., vol. 28, no. 1, pp. 145–154, Jan. 2011.
  14. C. Jiang, H. Zhang, Y. Ren, Z. Han, K. Chen, and L. Hanzo, “Machine learning paradigms for next-generation wireless networks,” IEEE Wireless Commun., vol. 24, no. 2, pp. 98–105, Apr. 2017.
  15. T. Wang, C. K. Wen, H. Wang, F. Gao, T. Jiang, and S. Jin, “Deep learning for wireless physical layer: Opportunities and challenges,” China Commun., vol. 14, no. 11, pp. 92–111, Nov. 2017.
  16. X. You, C. Zhang, X. Tan, S. Jin, and H. Wu, “AI for 5G: Research directions and paradigms,” Science China Information Sciences, Sep. 2018.
  17. T. O’Shea and J. Hoydis, “An introduction to deep learning for the physical layer,” IEEE Trans. Cogn. Commun. Netw., vol. 3, no. 4, pp. 563–575, Dec. 2017.
  18. S. Dörner, S. Cammerer, J. Hoydis, and S. t. Brink, “Deep learning based communication over the air,” IEEE J. Sel. Topics Signal Process., vol. 12, no. 1, pp. 132–143, Feb. 2018.
  19. V. Vanhoucke, A. Senior, and M. Z. Mao, “Improving the speed of neural networks on CPUs,” in Deep Learn. Unsupervised Feature Learn. NIPS Workshop, 2011.
  20. E. Nachmani, Y. Be’ery, and D. Burshtein, “Learning to decode linear codes using deep learning,” in IEEE Annu. Allerton Conf. Commun. Control Comput. (Allerton), Sep. 2016, pp. 341–346.
  21. E. Nachmani, E. Marciano, L. Lugosch, W. J. Gross, D. Burshtein, and Y. Be��ery, “Deep learning methods for improved decoding of linear codes,” IEEE J. Sel. Topics Signal Process., vol. 12, no. 1, pp. 119–131, Feb. 2018.
  22. T. Gruber, S. Cammerer, J. Hoydis, and S. t. Brink, “On deep learning-based channel decoding,” in IEEE Annu. Conf. Inf. Sci. Syst. (CISS), Mar. 2017, pp. 1–6.
  23. T. O’Shea, K. Karra, and T. Clancy, “Learning to communicate: Channel auto-encoders, domain specific regularizers, and attention,” in IEEE Int. Symp. Signal Process. Inf. Technol., Dec. 2016, pp. 1–6.
  24. T. Erpek, T. J. O’Shea, and T. C. Clancy, “Learning a physical layer scheme for the MIMO interference channel,” in IEEE Int. Conf. Commun (ICC), May 2018, pp. 1–5.
  25. T. J. O��Shea, T. Roy, and T. C. Clancy, “Over-the-air deep learning based radio signal classification,” IEEE J. Sel. Topics Signal Process., vol. 12, no. 1, pp. 168–179, Feb. 2018.
  26. A. Felix, S. Cammerer, S. Dörner, J. Hoydis, and S. t. Brink, “OFDM-autoencoder for end-to-end learning of communications systems,” arXiv preprint arXiv:1803.05815, 2018.
  27. C. K. Wen, W. T. Shih, and S. Jin, “Deep learning for massive MIMO CSI feedback,” IEEE Wireless Commun. Lett., to be published, DOI 10.1109/LWC.2018.2832128.
  28. H. He, C. Wen, S. Jin, and G. Y. Li, “Deep learning-based channel estimation for beamspace mmwave massive MIMO systems,” IEEE Wireless Commun. Lett., vol. 7, no. 5, pp. 852–855, Oct. 2018.
  29. T. Wang, C. Wen, S. Jin, and G. Y. Li, “Deep learning-based CSI feedback approach for time-varying massive MIMO channels,” IEEE Wireless Commun. Lett., to be published, DOI 10.1109/LWC.2018.2874264.
  30. N. Samuel, T. Diskin, and A. Wiesel, “Deep MIMO detection,” arXiv preprint arXiv: 1706.01151, 2017.
  31. M. Kim, N. Kim, W. Lee, and D. Cho, “Deep learning-aided SCMA,” IEEE Commun. Lett., vol. 22, no. 4, pp. 720–723, Apr. 2018.
  32. S. Xue, Y. Ma, N. Yi, and R. Tafazolli, “Unsupervised deep learning for MU-SIMO joint transmitter and noncoherent receiver design,” IEEE Wireless Commun. Lett., to be published, DOI 10.1109/LWC.2018.2865563.
  33. K. Kim, J. Lee, and J. Choi, “Deep learning based pilot allocation scheme (DL-PAS) for 5G massive MIMO system,” IEEE Commun. Lett., vol. 22, no. 4, pp. 828–831, Apr. 2018.
  34. H. Ye, G. Y. Li, and B. Juang, “Power of deep learning for channel estimation and signal detection in OFDM systems,” IEEE Wireless Commun. Lett., vol. 7, no. 1, pp. 114–117, Feb. 2018.
  35. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning.   Cambridge, MA, USA: MIT Press, 2016.
  36. A. D. Stefano, O. Mirabella, G. D. Cataldo, and G. Palumbo, “On the use of neural networks for Hamming coding,” in IEEE Int. Symp. Circuits and Systems, vol. 3, Jun. 1991, pp. 1601–1604.
  37. L. Wu, Z. Zhang, and H. Liu, “Adaptive modulation with finite rate feedback for QR decomposition-successive interference cancellationbased multiple-in multiple-out systems,” IET Commun., vol. 7, no. 5, pp. 456–462, Mar. 2013.
  38. X. Chen, L. Wu, Z. Zhang, J. Dang, and J. Wang, “Adaptive modulation and filter configuration in universal filtered multi-carrier systems,” IEEE Trans. Wireless Commun., vol. 17, no. 3, pp. 1869–1881, Mar. 2018.
  39. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv: 1412.6980, 2014.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
330594
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description