A Predictive OnDemand Placement of UAV Base Stations Using Echo State Network
Abstract
The unmanned aerial vehicles base stations (UAVBSs) have great potential in being widely used in many dynamic application scenarios. In those scenarios, the movements of served user equipments (UEs) are inevitable, so the UAVBSs needs to be repositioned dynamically for providing seamless services. In this paper, we propose a system framework consisting of UEs clustering, UAVBS placement, UEs trajectories prediction, and UAVBS reposition matching scheme, to serve the UEs seamlessly as well as minimize the energy cost of UAVBSs’ reposition trajectories. An Echo State Network (ESN) based algorithm for predicting the future trajectories of UEs and a KuhnMunkresbased algorithm for finding the energyefficient reposition trajectories of UAVBSs is designed, respectively. We conduct a simulation using a real open dataset for performance validation. The simulation results indicate that the proposed framework achieves high prediction accuracy and provides the energyefficient matching scheme.
I Introduction
With the rapid development of cellular communication techniques, people become dependent on mobile devices, such as cell phones and tablets in daily life. Most communication services for these devices are provided by ground base stations (BSs). However, in some remote or hotspot areas where ground BSs are unable to cover, the quality of communications service is usually very poor. To solve this problem, the use of unmanned aerial vehicles (UAVs) as BSs has been a popular research topic in recent years. UAVBSs can deliver reliable, costeffective, and ondemand cellular communication service especially to dynamic scenarios [9].
Recent study [3] on UAVBSs mainly focuses on finding the optimal placement of the UAVBSs while serving the user equipments (UEs) in the target area. However, in most papers [1] [5] [7], few take dynamic scenarios into consideration. In real scenarios, the UAVBSs are expected to dynamically reposition in response to the dynamic movement of UEs. In [8], the approach considered dynamic scenarios, but the reposition action is performed after the movement of UEs, which may cause communication service outage. In order to serve the UEs seamlessly, a prediction on the future positions of UEs can be performed before the reposition action of UAVBSs in the next time slot. This prediction can be implemented by applying an Echo State Network (ESN), a type of recurrent neural network.
The main contribution of this paper is to propose a system framework for dynamic placement of multiple UAVBSs, aiming to serve the UEs seamlessly and minimize the energy cost of UAVBSs’ trajectories. The merits of this work are described from the following three aspects:

We design an ESNbased algorithm for predicting the future trajectories of UEs, aiming at serving the UEs seamlessly. With the precise predicted locations of UEs, the system can search for the potential locations of UAVBSs in the next time slot for the seamless services.

We develop a KuhnMunkresbased weighted bipartite matching algorithm to find the energyefficient trajectories of multiple UAVBSs when reposition from current positions to the predicted positions in the next time slot. This algorithm is aiming at minimizing the energy cost in the reposition action phase.

We adopt the GeoLife dataset [11] as the input information with some preprocessing steps for our simulations. The simulation results indicate that the proposed algorithm can achieve high accuracy in terms of root mean square error (RMSE).
The remaining parts of this paper are organized as follows. In Section II, the related work is introduced. In Section III, we discuss the ESNbased prediction algorithm, the KMbased matching algorithm, and the proposed algorithm step by step. In Section IV, the simulation results are discussed in detail. We make the conclusion remarks in Section V.
Ii Related Work
For UAV control, in [8], the authors studied on finding the optimal altitude for a single UAVBS with the maximum coverage and minimum required transmit power. The authors in [1] showed that the UAVBS placement in the horizontal dimension can be modeled as a circle placement problem and a smallest enclosing circle problem, and they proposed an optimal 3D placement algorithm for maximizing the number of served users with the minimum transmit power. In [5], a method using the heuristic algorithm for finding the positions of UAVBSs in an area with different user densities was proposed. This algorithm can estimate the minimum number of UAVBSs and their placement, while satisfying coverage and capacity constraints. In [7], the placement problem was modeled as a knapsacklike problem, and a densityaware placement algorithm to maximize the number of covered users with the constraint of the minimum required data rates per user was proposed.
For learning, the authors in [2] proposed a Qlearning based method for finding the optimal trajectory of an UAVBS to serve multiple users. In their method, the UAVBS acts as an autonomous agent to learn the trajectory that maximizes the sum rate of the transmission. A dataset from a project called GeoLife is introduced in [11]. This dataset consists of 182 users’ locations and GPS trajectories in a period of over five years. In [10], the authors proposed an Markov Chain predictive model for predicting the next location of an individual based on its recent locations and mobility behavior over a period of time. In their simulation part, the GeoLife dataset was used to evaluate the proposed algorithm.
Iii Proposed System Framework
We consider the dynamic application scenario depicted in Fig. 1. The UAVBSs are expected to reposition to the potential positions in advance of the movement of the UEs. In Section IIIA, we design an ESNbased algorithm to predict the future trajectories of UEs for finding the potential positions of UAVBSs. In Section IIIB, we also take the energy cost of the reposition action of UAVBSs into account. A KMbased matching algorithm is proposed to find the energyefficient trajectories. In Section IIIC, the proposed algorithm consisting of clustering, placement, prediction, and matching is introduced step by step.
Iiia ESNbased Prediction Algorithm
To obtain the predicted positions of UEs in the next time slot, an algorithm based on ESN is proposed. We choose ESN due to its low computation time and energy cost. ESN forms a hidden layer of the network by randomly deploying a large numbers of neurons, known as the reservoir pool. The ESN model has the following characteristics:

containing a large number of neurons;

the connection between neurons is generated randomly;

the links between neurons are sparsity.
The structure of the ESN model is drawn in Fig. 2, where represents the input weight matrix, is the reservoir weight matrix and is the output weight matrix.
In the input layer, we define the input vector as , and its dimensions is . In the reservoir pool, the typical update equation is written as [4]
(1) 
where is a vector of internal units in the reservoir pool. is the connection weight matrices between input layer and reservoir pool, and is the recurrent weight matrices.
The reservoir pool is linearly connected to the output layer, which can be defined as
(2) 
where is the output vector and is the connection weight matrices between reservoir pool and output layer.
We use root mean square error (RMSE) to evaluate the quality of this model, the expected value is and the actual result is . RMSE is defined as
(3) 
where , is the number of predictions, and are the weights. In the considered dataset, some UEs move unexpectedly (Uturn, rightangled turn), and the positions of these unexpected movements are assigned with the smaller weights.
IiiB KMbased Matching Algorithm
Without considering environmental factors, if the summation of all UAVBSs’ moving distance from the current positions to the positions in the next time slot is smaller, the less energy will be consumed. Such an EnergyEfficient Displacement Optimization (EEDO) problem can be formulated as Definition 1. The energyefficient placement optimization problem also can be reduced from a wellknown problem, Minimum Weighted Perfect Bipartite Matching, as shown in Fig. 3.
Definition 1 (Eedo)
At time , the current position of a UAV is denoted as , and the potential position of a deployed UAV in the next time slot is denoted as , is the number of UAVs, and is the energy cost for a UAV to fly from position to position . The total energy cost of the UAVBSs’ displacements can be optimized by
(P1)  
s.t.  
In this work, we assume the number of UAVBSs is , and the heights of the multiple UAVBSs are the same and equal to a constant. At time , the number of current positions and the number of predicted positions of UAVBSs are both equal to . The coordinate of the th current position is defined as and the coordinate of the th predicated position is defined as , where is a predefined time period slot between the predictions, and and are the latitude and longitude of the th current position, respectively. and are the latitude and longitude of the th predicated position, respectively. The vertex labeling of and are denoted as and , respectively. The weight between and is defined as , which can be calculated by the distance between and .
We aim at minimizing the energy cost, which is equivalent to finding the smallest sum of total weights. A bipartite matching method based on KuhnMunkres (KM) algorithm is used. According to KM algorithm, and will be
(4) 
Since the number of UAVBSs is equal to the number of predicated positions, a weighted perfect matching can be found using the Hungarian algorithm [6].
IiiC Proposed Algorithm
In this part, the proposed system framework for dynamic placement of multiple UAVBSs is introduced. Assume is the number of UEs and is the number of UAVBSs. is the coordinate of the th UE’s current position where .

In the initial time slot, cluster the UEs by using the means algorithm, and divide UEs into clusters.

Use the densityaware placement algorithm [7] to find the local optimal positions of each cluster to deploy UAVBSs in the initial time slot. Let the UAVBSs fly to those positions.

Collect the realtime latitude and longitude information of each UE’s position every seconds for seconds.

Use the collected information in 3) as the input UEs’ trajectories data for the ESNbased prediction algorithm to predict the UE’s positions in the next time slot: .

Use as input, calculate the optimal positions of UAVBSs in the next time slot by using the means algorithm and densityaware placement algorithm again.

Use the KMbased matching algorithm to find the energyefficient trajectories of UAVBSs when moving from current positions to the predicted positions. Let the UAVBSs fly to those positions.

Repeat from Step 3) to Step 6) after a given time slot .
In general, the system performance on allocated data rates will be better if is smaller. However, the total energy cost of UAVBSs will dramatically increase if becomes too small. The pseudocodes of the proposed algorithm are shown in Algorithm LABEL:alg:proposed_algo.
algocf[!t] \end@float
Iv Simulation Results
In this section, we discuss the performance of our proposed algorithms. In Section IVA, the Euclidean distance between the predicted trajectory and actual trajectory of the UE is used to evaluate the ESNbased prediction algorithm. In Section IVB, the KMbased matching algorithm is compared to random matching methods in term of the sum of total distance between the current positions and the predicted positions.
Iva Performance of Prediction
In the first part, the dataset from the GeoLife project provided by Microsoft Research Asia is used to evaluate our proposed ESNbased prediction algorithm. We select the data in the area of Beijing. 75% of the data is used as train set and the rest 25% is used as the test set. In order to adapt the data to the real UAVBSs application scenario, we set each UE’s trajectory data time interval to 3 seconds by interpolating the data and finetuning the value of data time.
The actual trajectory and predicted trajectory of the UE are both depicted with Google Map and shown in Fig. 4. The actual trajectory of the UE is reddotted line, and the predicted trajectory of UE is bluedashed line. From this figure, we can find that the predicted trajectory is very close to the actual trajectory.
We use the actual trajectory data of the UE within 15 minutes as input and predict the trajectory of the UE in the next 5 minutes time slot with the proposed ESNbased prediction model. In Fig. 5 and Fig. 6, the longitude and latitude of predicted trajectory (marked as a bluedashed line) and actual trajectory (marked as a redsolid line) of the UE are shown respectively. The predicted trajectory and actual trajectory are depicted with 100 predicted positions and 100 actual positions, respectively. The time interval between the predicted positions is set to 3 seconds, which equals to the time interval between the actual positions.
It is obvious that for both longitude and latitude, the predicted value is close to the actual value. Due to the randomness and uncertainty of movement of users, the curve of predicted value is almost smooth, while the curve of actual value fluctuates a little bit. As there is a vertical turn of the actual trajectory in Fig. 6, we find the reason after reviewing the actual data, which is the UE changed its direction at that time. We also observe that the curve of predicted longitude and latitude in both Fig. 5 and Fig. 6 are straight. It means that the proposed ESN model cannot predict the trend of the position changes very well. The reason is that there is no sufficient data of nonstraight trajectories in GeoLife dataset.
We also evaluate the performance of the ESN model with different sizes of the reservoir pool. We set the size to 500, 1000, 2000, 3000 and 5000, and calculate the distance between the predicted trajectory and actual trajectory of the UE, respectively. According to the result in Fig. 7, it indicates that when is set to 5000, the predicted value is closest to the actual value. The RMSE is also calculated and shown in Fig. 8. We find that the RMSE value of each model is no more than 0.030, which indicates that the ESN model has a high prediction accuracy. In addition, when is no more than 5000, the predicted accuracy increases with the increment of , which is corresponding to the results in Fig. 7.
IvB Performance of Matching
We set the number of UAVBSs to 3. TABLE I shows the current latitude and longitude of UAVBSs, and the predicted latitude and longitude of UAVBSs in the next 5 minutes time slot. We calculate the distance between each current position and each predicted position when taking the curvature of the Earth into account. The results are shown in TABLE II. For instance, if a possible matching scheme is obtained, the UAVBS whose current position is needs to fly to the position in the next time slot. So the distance of the reposition trajectory for this UAVBS is 267 meters. The rest distance can be deduced in the same way, and the total distance of this matching scheme will be meters.
We implement the KMbased matching algorithm in Python 3 and use the values in TABLE II as the input information. The output reposition matching scheme is shown in TABLE III.
Latitude  Longitude  

Current position  
Current position  
Current position  
Predicted position  
Predicted position  
Predicted position 
Predicted position  Predicted position  Predicted position  

Current position  m  m  m 
Current position  m  m  m 
Current position  m  m  m 
Predicted position  Predicted position  Predicted position  

Current position  
Current position  
Current position 
^{1}^{1}footnotemark: 1
: Mismatched : Matched
Each possible reposition matching schemes and its corresponding sum of distance are listed in TABLE IV. It is obvious that the reposition matching scheme is the same as the one obtained by the KMbased matching algorithm and it has the minimal sum of the distance. In other words, our KMbased matching algorithm can find the energyefficient trajectories of multiple UAVBSs when reposition from the current positions to the predicted positions in the next time slot.
Reposition matching scheme  Sum of distance (in meters) 

(minimal)  
V Conclusion
In this paper, the dynamic placement problem of UAVBSs is studied. We propose an ESNbased algorithm to predict the future trajectories of the UEs. The predicted trajectories can be used to find the potential positions to reposition the UAVBSs in the next time slot. Additionally, we consider the energy cost of reposition. A KMbased algorithm is designed to solve the minimum weighted perfect bipartite matching problem and find the minimum energy cost reposition scheme. The simulation results show that the ESNbased algorithm has high accuracy on predicting the next 5 minutes trajectories of the UEs based on previous 15 minutes actual trajectories data, and the matching scheme obtained by the KMbased algorithm satisfies the energyefficient requirement of UAVBSs reposition trajectories. In the future, we are going to develop a Generative Adversarial Network (GAN) based framework for generating sufficient trajectory data and thus improve the performance of prediction.
Acknowledgment
This research is supported by Ministry of Science and Technology under the Grant MOST 1082634F009006 through Pervasive Artificial Intelligence Research (PAIR) Labs, Taiwan.
References
 [1] (2017Aug.) 3d placement of an unmanned aerial vehicle base station (uavbs) for energyefficient maximal coverage. IEEE Wireless Communications Letters 6 (4), pp. 434–437. External Links: Document, ISSN 21622337 Cited by: §I, §II.
 [2] (2018Jun.) Trajectory optimization for autonomous flying base station via reinforcement learning. In IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Vol. , Kalamata, Greece. External Links: Document, ISSN 19483252 Cited by: §II.
 [3] (2016Dec.) Dynamic base station repositioning to improve performance of drone small cells. In IEEE Global Communications Conference Workshops (GLOBECOM Wkshps), San Diego, CA. External Links: Document Cited by: §I.
 [4] (2001Jan.) The “echo state” approach to analysing and training recurrent neural networkswith an erratum note. Bonn, Germany: German National Research Center for Information Technology GMD Technical Report 148 (34), pp. . Cited by: §IIIA.
 [5] (2016Sep.) On the number and 3d placement of drone base stations in wireless cellular networks. In IEEE 84th Vehicular Technology Conference (VTCFall), Vol. , Montreal, QC, Canada. External Links: Document Cited by: §I, §II.
 [6] (1955) The hungarian method for the assignment problem. Naval Research Logistics Quarterly 2 (1‐2), pp. 83–97. External Links: Document Cited by: §IIIB.
 [7] (to be published) Ondemand densityaware uav base station 3d placement for arbitrarily distributed users with guaranteed data rates. IEEE Wireless Communications Letters. External Links: Document, ISSN 21622337 Cited by: §I, §II, item 2.
 [8] (2015Dec.) Drone small cells in the clouds: design, deployment and performance analysis. In IEEE Global Communications Conference (GLOBECOM), Washington, DC. Cited by: §I, §II.
 [9] (to be published) A tutorial on uavs for wireless networks: applications, challenges, and open problems. IEEE Communications Surveys and Tutorials. Cited by: §I.
 [10] (2018Jul.) Predicting human mobility using sina weibo checkin data. In International Conference on Audio, Language and Image Processing (ICALIP), Vol. , Shanghai, China. External Links: ISSN Cited by: §II.
 [11] (2010Jun.) GeoLife: A collaborative social networking service among user, location and trajectory. IEEE Data(base) Engineering Bulletin 33 (2), pp. 32–39. Cited by: 3rd item, §II.