# Optimal Measurement Policy for Predicting UAV Network Topology

## Abstract

In recent years, there has been a growing interest in using networks of Unmanned Aerial Vehicles (UAV) that collectively perform complex tasks for diverse applications. An important challenge in realizing UAV networks is the need for a communication platform that accommodates rapid network topology changes. For instance, a timely prediction of network topology changes can reduce communication link loss rate by setting up links with prolonged connectivity.

In this work, we develop an optimal tracking policy for each UAV to perceive its surrounding network configuration in order to facilitate more efficient communication protocols. More specifically, we develop an algorithm based on particle swarm optimization and Kalman filtering with intermittent observations to find a set of optimal tracking policies for each UAV under time-varying channel qualities and constrained tracking resources such that the overall network estimation error is minimized.

## 1Introduction:

Utilizing a swarm of autonomous UAVs to perform complicated tasks in military and commercial applications has recently gained a lot of momentum and is expected to continue growing in the coming years [1]. For instance, the U.S. Navy has recently launched a prototype for the LOw-Cost Unmanned aerial vehicle Swarming Technology (LOCUST) project that implements the required technology for UAV swarm attacks [2]. The UAV market value was estimated to be USD 13.22 Billion in 2016 and is projected to reach USD 28.27 Billion by 2022 [3].

Swarms of fast-flying UAVs form dynamic networks with rapidly changing topologies, where conventional communication protocols that make decisions at different layers solely based on current network configuration and ignore topology evolution often perform poorly [4]. For instance, a conventional relay selection algorithm, which selects intermediate relay nodes merely based on the current system configuration falls behind the abrupt topology changes and hence fails in providing prolonged network connectivity for member UAVs [6]. As such, developing algorithms that enable the UAVs to effectively predict their surrounding nodes’ motion trajectories can significantly improve the performance of topology-aware communication and control algorithms.

An example for the utility of predicting network topology in optimizing communication protocols is illustrated in Figure 1.

The motion trajectory of 3 UAVs () are depicted in figure 2. UAV1 has limited communications range that is shown by three circles centered at locations ( and ) for three time points . In order to retain connectivity with at least one neighboring UAV, intends to choose the best relay node. At time , and are both within the ’s accessible range. Under a conventional relay selection method, chooses since it is closer in distance and requires less transmission power (or yields lower delay). At time , moves out of ’s accessible range and hence needs to hand over to another relay node, which requires additional signaling. Also, if node has already been assigned with a relaying task for another UAV, then falls out of the network and looses its connection to the base station. However, if was capable of predicting motion trajectory of neighbor UAVs and recognizing a better alignment between ’s and its own trajectories, it would have chosen at time , which never goes out of the accessible range. Consequently, would have retained the connectivity until the end of its mission [7].

In this paper, we use the popular method of Dubin’s curves to model the motion of UAVs and use the time and measurement update equations developed for Kalman filtering with intermittent observations to predict the locations of UAVs, when the tracking measurements are sporadically available. In order to obtain the next network prediction results and retain maximal connectivity [8], we propose an optimal measurement policy for UAVs such that the prediction error of the surrounding UAV locations are minimized under a constrained measurement resource and an interrupted observation model. The idea is to use particle swarm optimization to develop a resource management policy for an individual UAV to allocate its measurement resources optimally among a set of neighbor UAVs such that the expected error covariance is minimized. The proposed method provides a practical solution for a theoretically intractable problem and has the advantage of adapting to time-varying measurement conditions.

## 2Topology Prediction

Recently, several methods including data-driven methods [10], piecewise segment methods [12], hidden Markov models [13], Levy and Levy flight process [14], manifold learning [15], and Gaussian mixture models [16] are proposed to model mobile user motions (e.g. vehicles and pedestrians) in wireless networks. However, these models are not well suited for freely flying UAVs that do not follow man-made or natural path profiles (e.g. roads and streets). Here, we use the Dubin’s curve method, which is general and specifies the motion trajectory of a moving object based on the exerted forces in 3 dimensions as follows: [17].

where we have:

Here, dt is the time step of the discretized system, is the state vector of UAV at time , representing the location () and velocities (), captures the impact of applied forces on the accelerations, is the measurements provided by the tracking system. We assume that and are zero mean Gaussian random vectors with covariances and which respectively represent the model turbulence and observations noise. The measurements success is modeled as a sequence of Bernoulli distributed random variables () [18]. It is known that an optimal estimation of the state vectors (in MMSE sense) is obtained using Kalman filtering with the following steps:

where the measurement update equations are performed only for and we set for [18]. Under some mild convergence conditions on the A, B, C, Q matrices, (i.e. observability of (), controllability of (), stabilizability of (), and bounded system and measurement noise covariances ), the sequential error covariance matrices starting from any initial value converge to a unique limit, which is the solution of the following Modified Discrete-time Algebraic *Ricatti* equations (MARE) [18]:

The solution of (Equation 3) for intermittent observation is not known in general, but its statistical properties are studied under different assumptions [19]. In particular, it is known that the convergence of is ensured if the observation availability occurs with probability , where is a critical value with known upper and lower bounds [24].

Here, we assume that the input vector, is known. Note that the role of unknown input and the time update equations in general is more important for intermittent observations, since we rely more on the time-update equations. Intuitively, in a fully observable system, the measurement update equations enhance our estimates and partially compensate the lack of information about the unknown inputs, as expected. Fig. ? demonstrates the impact of unknown input for a 1D motion. However, we can use recently proposed techniques to tackle the case of unknown input vectors. For instance, with a simple conversion technique, the unknown input can be absorbed into the state vector to transform the system to an equivalent system with known inputs [27]. If the statistics of the unknown input is fully known, then the optimal state estimator can be represented by the two-stage Kalman filter [28]. The simulation results for the case of unknown input and and is shown in Fig. ?.

## 3Optimal Tracking Policy

In this section, we assume a scenario, where a UAV intends to track surrounding UAVs. The parameters for target UAV includes the motion control noise covariance , navigation noise covariance , and the packet transmission success rate . The observer UAV is assumed to be equipped with tracking instruments, where . The objective is to design an algorithm to optimally assigns the tracking resources among the surrounding UAVs, during a measurement cycle composed of consecutive time slots, such that a desired evaluation metric is minimized. We develop the algorithm in two sequential steps for a measurement cycle. We first consider a probabilistic model, where at each time slot, UAV is tracked with probability , where . The objective of this step is to find the *optimal measurement probability vector* , such that a desired performance criterion is met. Apparently, due to resource constraint, we have . We consider the following two objective functions.

One may choose to minimize the worst-case squared error by solving the following optimization problem:

where is the critical value for the channel success probability to have bounded expected error covariance, characterized in [18]. The feasible set of this problem is defined by its constraint functions and the solution for this problem exists only if we have sufficient tracking resources (i.e. ). In particular, we use a water filling approach to assign sufficient tracking attempt probability for each UAV such that its effective measurement rate exceeds the corresponding critical value until all constraints are satisfied. However, finding the optimal way to associate the remaining tracking resources to optimize the objective function is not tractable. An alternative objective function is to minimize the overall aggregated estimation error. More formally, one may intend to solve the following optimization problem:

This is a stochastic optimization problem. It is practically hard to solve due to the randomness of , therefore we solve the following deterministic by almost equivalent problem:

Note that this problem is deterministic due to the expectation operator. However, and do not admit closed-from equations. Furthermore, estimating and in practice requires a large number of consecutive measurement samples, which is not feasible in time-varying conditions. Therefore, finding an analytical solution for this optimization problem is not practically feasible. Therefore, we propose to use the following approximate optimization algorithm based on Particle Swarm Optimization (PSO) to solve this problem.

The main idea here is that we generate a set of particles, each representing the system with a set of hypothetical parameters , which are randomly generated. Also, we assume a random measurement policy vector to each particle . Then, for each UAV , we update the Kalman filtering equations and estimate the state vector and the estimation covariance matrix . Then, for each particle, we use MARE to generate the current hypothetical error covariance matrix based on the previous error covariance matrix . Over time, the hypothetical error covariance matrix of a particle which has more accurate presumed parameters is expected to be closer to the estimate error covariance matrix . As such, we move each particle in a direction which is the linear combination of three directions including: i) their motion direction in the previous iteration, ii) their motion towards a point in their local history that yields the best match (i.e. local best), and iii) towards the particle with the best match (global best). At each iteration, the parameters of each particle are updated based on the obtained particle velocity. After a number of iterations, the particles are expected to converge to a particle with the best value for observation probability vector , which in turn yields the best prediction results. A formal algorithmic description of the proposed algorithm is included in Algorithm 1.

Here, and are tuning parameters to balance between the motion velocities of each particle towards its local minimum and the global minimum. Also, and , respectively, store the best particle ID (global optimum) and the time index of the local optimum for particle .

Finally, we note that finding the optimal observation probability vector does not complete the problem. It only determines the rate under which each UAV should be subject to tracking. However, actual observation policy comprises fully determining a binary matrix , where represents a measurement attempt to track UAV at time . In order to determine the actual measurement attempt matrix , in this case, we recall the following constraints:

We conjecture that one guideline to minimize the accumulated estimation error () for each UAV, is to use the most alternating measurement pattern such that measurement resources are evenly distributed over time. For instance, if , the optimal measurement pattern is . This fact is confirmed by simulation results and we are working to provide an analytical proof.

## 4Simulation Results

In this section, simulation results are provided to verify the performance of the proposed PSO-based algorithm. Figure 2 presents the average estimation error in terms of Mean Squared Error for using three methods. The *uniform* distribution is corresponding to the case of evenly assigning tracking resources among UAVs at each time slot (i.e. with Prob. for ). PSO denotes the average MSE obtained by probabilities determined by the proposed PSO-based algorithm (see Algorithm 1). The MSE for random particles are also shown for the sake of comparison completeness. It can be seen that the proposed PSO algorithm outperforms the uniform and randomly selected measurement policies. The fluctuations of MSE are corresponding to alternating measurement success and fail events (). Figure 3 also represents the measurement probability evolution by the PSO algorithm confirming that after about 40 iterations the probabilities converge to their optimal values.

Finally, different measurement patterns are compared in Figure 4, which verifies our conjecture about the optimality of the most evenly distributed patterns. In other words, it is desired that the “1”s of each row of the actual measurement attempt matrix are evenly distributed. The intuitive justification behind this fact is that consecutive time slots without measurements cause dramatic uncontrolled rises in estimation error, whereas as a series of consecutive measurements does not provide the same amount of improvement. As such, the most alternating pattern provides the lowest accumulated estimation error on average.

## 5Conclusions

The proposed methodology provides a numerical solution for an otherwise intractable optimization problem of assigning limited measurement resources among a large number of targets. Our approach suggests a low-complexity algorithm to optimally assign tracking resources by a UAV to discover and predict its surrounding UAVs. The proposed method is general and applicable to similar problems that use Kalman filtering as their optimization methods.

### References

- =2plus 43minus 4
*Unmanned Aircraft Systems roadmap 2005-2030*.1em plus 0.5em minus 0.4emOffice of the Secretary of Defense. [Online]. Available: https://fas.org/irp/program/collect/uav-roadmap2005.pdf =0pt - =2plus 43minus 4 D. Smalley. Locust: Autonomous, swarming uavs fly into the future. [Online]. Available: http://www.onr.navy.mil/Media-Center/Press-Releases/2015/LOCUST-low-cost-UAV-swarm-ONR.aspx =0pt
- =2plus 43minus 4 M. and Markets. (2016) Unmanned aerial vehicle (uav) market, by application. [Online]. Available: http://www.marketsandmarkets.com/Market-Reports/unmanned-aerial-vehicles-uav-market-662.html =0pt
- A. Valehi and A. Razi, “Maximizing energy efficiency of cognitive wireless sensor networks with constrained age of information,”
*IEEE Transactions on Cognitive Communications and Networking*, vol. PP, no. 99, pp. 1–1, 2017. - A. Razi and A. Abedi, “Adaptive bi-modal decoder for binary source estimation with two observers,” in
*2012 46th Annual Conference on Information Sciences and Systems (CISS)*, March 2012, pp. 1–5. - =2plus 43minus 4 I. Bekmezci, O. K. Sahingoz, and c. Temel, “Flying ad-hoc networks (fanets),”
*Ad Hoc Netw.*, vol. 11, no. 3, pp. 1254–1270, May 2013. [Online]. Available: http://dx.doi.org/10.1016/j.adhoc.2012.12.004 =0pt - A. Sugranes and A. Razi, “Predictive routing for dynamic uav networks,” in
*Wireless for Space and Extreme Environments (WiSEE)*, Oct 2017. - H. Zhang and J. C. Hou, “Maintaining sensing coverage and connectivity in large sensor networks.”
*Ad Hoc & Sensor Wireless Networks*, no. 1-2. - A. Ghosh and S. K. Das, “Coverage and connectivity issues in wireless sensor networks: A survey,”
*Pervasive and Mobile Computing*, vol. 4, no. 3, pp. 303–334, 2008. - J. G. Lee, J. Han, and X. Li, “A unifying framework of mining trajectory patterns of various temporal tightness,”
*IEEE Transactions on Knowledge and Data Engineering*, vol. 27, no. 6, pp. 1478–1490, June 2015. - =2plus 43minus 4 O. Ossama, H. M. Mokhtar, and M. E. El-Sharkawi, “An extended k-means technique for clustering moving objects,”
*Egyptian Informatics Journal*, vol. 12, no. 1, pp. 45 – 51, 2011. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S1110866511000089 =0pt - P. Y. Choi and M. Hebert, “Learning and predicting moving object trajectory: a piecewise trajectory segment approach,” Robotics Institute, Pittsburgh, PA, Tech. Rep. CMU-RI-TR-06-42, August 2006.
- M. Bennewitz, W. Burgard, G. Cielniak, and S. Thrun, “Learning motion patterns of people for compliant robot motion,”
*Internationl Journal of Robotics Research*, vol. 24, pp. 31–48, 2005. - M. C. Gonzalez, C. A. Hidalgo, and A.-L. Barabasi, “Understanding individual human mobility patterns,”
*Nature*, vol. 453, no. 7196, pp. 779–782, June 2008. - G. Lee, R. Mallipeddi, and M. Lee, “Identification of moving vehicle trajectory using manifold learning,”
*Springer Journal of Neural Information Processing*, vol. 7666, pp. 188–195, 2012. - G. Aoude, J. Joseph, N. Roy, and J. How, “Mobile agent trajectory prediction using bayesian nonparametric reachability trees,” in
*InfotechAerospace Conferences*, 2011. - Y. Lin and S. Saripalli, “Path planning using 3d dubins curve for unmanned aerial vehicles,” in
*Unmanned Aircraft Systems (ICUAS), 2014 International Conference on*, May 2014, pp. 296–304. - B. Sinopoli, L. Schenato, M. Franceschetti, K. Poolla, M. I. Jordan, and S. S. Sastry, “Kalman filtering with intermittent observations,” in
*Decision and Control, 2003. Proceedings. 42nd IEEE Conference on*, vol. 1, Dec 2003, pp. 701–708 Vol.1. - J. Moon and T. Başar, “Estimation over lossy networks: A dynamic game approach,” in
*52nd IEEE Conference on Decision and Control*, Dec 2013, pp. 2412–2417. - S. Kluge, K. Reif, and M. Brokate, “Stochastic stability of the extended kalman filter with intermittent observations,”
*IEEE Transactions on Automatic Control*, vol. 55, no. 2, pp. 514–518, Feb 2010. - S. Y. Park and A. Sahai, “Intermittent kalman filtering: Eigenvalue cycles and nonuniform sampling,” in
*Proceedings of the 2011 American Control Conference*, June 2011, pp. 3692–3697. - S. Kar and J. M. F. Moura, “Moderate deviations of a random riccati equation,”
*IEEE Transactions on Automatic Control*, vol. 57, no. 9, pp. 2250–2265, Sept 2012. - J. Wu, G. Shi, and K. H. Johansson, “Probabilistic convergence of kalman filtering with nonstationary intermittent observations,” in
*53rd IEEE Conference on Decision and Control*, Dec 2014, pp. 3783–3788. - X. Shen, D. Rus, and M. H. Ang, “Bounds for kalman filtering with intermittent observations,” in
*Control Conference (ECC), 2015 European*, July 2015, pp. 2842–2846. - J. Rohde, J. E. Stellet, H. Mielenz, and J. M. Zöllner, “Localization accuracy estimation with application to perception design,” in
*2016 IEEE International Conference on Robotics and Automation (ICRA)*, May 2016, pp. 4777–4783. - B. Sinopoli, L. Schenato, M. Franceschetti, K. Poolla, M. I. Jordan, and S. S. Sastry, “Kalman filtering with intermittent observations,”
*IEEE Transactions on Automatic Control*, vol. 49, no. 9, pp. 1453–1464, Sept 2004. - B. Boulkroune, M. Darouach, and M. Zasadzinski, “Moving horizon estimation for discrete time linear systems with unknown inputs,” in
*Control Conference (ECC), 2007 European*, July 2007, pp. 2875–2878. - =2plus 43minus 4 C.-S. Hsieh, “Optimal filtering for systems with unknown inputs via the descriptor kalman filtering method,”
*Automatica*, vol. 47, no. 10, pp. 2313–2318, Oct. 2011. [Online]. Available: http://dx.doi.org/10.1016/j.automatica.2011.08.012 =0pt