A Collaborative Aerial-Ground Robotic System for Fast Exploration
1 Motivation, Problem Statement, Related Work
Exploration of unknown environments using autonomous robots has been considered as a fundamental problem in robotics applications such as search and rescue Shen2017Collaborative (), industrial inspection and 3D modelling. For exploration, the basic requirement for robots is to scan unknown space or detect free space as fast as possible. UGV (unmanned ground vehicle) yamauchi1997frontier () and UAV (unmanned aerial vehicle) cieslewski2017frontier () both have been employed for such a task with differences primarily in: (1) UGVs are more payload-capable. A ground vehicle can carry heavy, long-range laser scanners which are inapplicable for weight-constrained UAVs; (2) UAVs have superiors mobility and agility. UAV can fly above obstacles and cover areas that are inaccessible to UGVs, like obstacle’s top surfaces. Consequently, a UGV often enjoys a larger sensor-coverage, yet cluttered and view-blocking environments could hamper its performance; on the other hand, a UAV may deliver inferior exploration efficiency due to its short-range sensor, but enjoys unblocked downward-looking view. Therefore, UGV favors open areas while UAV prefers cluttered places.
In this paper, considering the environmental preferences, we propose an autonomous collaborative framework which utilizes their complimentary characteristics to achieve higher efficiency and robustness in exploration applications.
For robotics exploration, yamauchi1997frontier () first proposed the concept of frontier, which is defined as unknown grid-map cells adjacent to free ones and thus represents accessible new information. Harmonic function, the solution to , is used to plan path to frontiers silva2002harmonic (). This method generates a scalar field in free-space based on its surrounding boundary conditions (occupied cells and frontier cells) and obtains the path using gradient-descent. For air-ground exploration, butzke2015airground () uses the UAV as an back-up instead of an independent explorer. It is only deployed when UGV encounters high, invisible areas. delmerico2016exploration () is also proposed based on the same spirit that one vehicle helps another, failing to exploit both vehicles’ full potential. Compared to these works, the collaborative system proposed in this paper fully utilizes advantages of different vehicles and thus results in a more efficient exploration. We summarize our contribution as: 1. An efficient exploration framework that combines UAV and UGV’s advantages. 2. A more efficient computation method of harmonic function for robotic exploration tasks. 3. Integration of the proposed collaborative exploration framework with the state estimation, sensor fusion and trajectory optimization. Extensive field experiments are presented to validate the efficiency and robustness of the proposed method.
2 Technical Approach
We first formulate the proposed exploration-with-preference problem using frontier. Since original definition of frontier in yamauchi1997frontier () does not apply to aerial robots as they hover above obstacles, we denote such definition as frontier A, and define frontier B as unknown cells adjacent to occupied cells. Frontier B represents new information solely accessible to UAV, which often appears on obstacles’ top surfaces. In short, both frontier A and frontier B are visible to UAV, but only frontier A is visible to UGV. Inheriting the intuitive preferences stated in Sec. 1, UGV pursues longer frontier A lines, while UAV prefers frontier B over frontier A.
The exploration planners are designed based on these intuitions. For the UGV’s planner, we employ harmonic function with two boundary conditions: occupied cells are assigned zero value, and frontier cells take on the negative value of its frontier A lines’ length. The path obtained in this method has several desirable features: (1) Being repelled from obstacles, thus the sensor’s range is more utilized (2) Being inclined to lead to longer and nearer frontier lines. Fig. 4 gives an example of harmonic function field and its gradient field.
We adopt the SOR method connolly1993harmonic () to iteratively compute harmonic function value until convergence, and in each iteration, cells’ function value are updated sequentially. Such update-sequence is conventionally the map-array’s sequence (row-by-row), but we propose an update-sequence that spreads out from new boundary conditions as shown in Fig. 1(e). This sequence is able to spread their influence faster, and the computation can converge within less iterations.
For the UGV, We first generate an obstacle-free travel coridor along the gradient descend path to utilize free space, and optimize the energy consumption of the path within this corridor to piecewise polynomial trajectories using Bernstein polynomial basis, as described in our previous work fei2018icra ().
To achieve rapid exploration for UAV, we employ motion primitive method likhachev2009planning () to generate flight-path candidates and model it as a linear quadratic minimum-time problem verriest1991linear (). The extensive and far-fetching path-tree is acquired by iteratively enforcing a discretized set of accelerations, and the path candidate with the highest information - cost ratio is executed. Information gain is simply frontier A/B’s coverage with the preferred frontier B having a larger weight. Once the local path-tree cannot acquire enough information gain, a distant yet lucrative frontier goal is generated. With the path waypoints created, a minimum jerk trajectory generator mellinger2011minimum () is implemented to minimize the consumed energy.
Both aerial and ground platforms used in experiments are custom-built, as shown in Fig. 2. Our ground platform is an omni-directional vehicle with four Mecanum wheels installed. A Velodyne VLP-16 LiDar is mounted to serve both localization with IMU and exploration purposes. Exploration planner, state estimation loam (), and control modules all run on an on-board Intel NUC with i7-5500U processor and a STM32F407 MCU board. The aerial platform is a self-developed quadrotor. For state estimation, it carries an IMU and a forward-looking camera with a 235 fish-eye lens. A downward-looking stereo camera is also installed for exploration. The Computations including state estimation vins () and exploration planning run on an Intel i7-5500U processor and a Nvidia TX2. A geometric controller lee2010geometric () is used to track the generated trajectory, and the attitude control is left to the DJI A3 autopilot on the quadrotor.
The real world experiment site is an area, with various obstacles inside, as shown in Fig. 3.The quadrotor and the ground vehicle both start from the right side of Fig 3(b) to perform exploration. The downward looking camera is configured to have a FOV. The UAV is flying at a constant height of 2m with maximum speed 1.4m/s, while the UGV is configured to have a laser scanned of 6m range and travels at a maximum speed of 0.5m/s.
We also perform simulations on distinctive maps. All environments are 20x20 meters in size and have 0.1 meter’s resolution. Exploration starts with both robots placed at a corner. UGV travels at a maximum speed of 1m/s, while the UAV has an higher maximum speed of 1.4 m/s. The simulated UAV is equipped with a down-ward looking depth camera with a field of view (FoV) of [60, 60] in vertical and horizontal directions, and the UGV carries a laser scanner with 6 meters’ range. The UAV hovers at a constant height of 2 meters, resulting in a 4.6 m’s sensor coverage.
Fig. 4 shows a visualization-capture, as an illustration of the framework. To examine the framework, we compare the performance (progress against time) of UAV-UGV team, a single UAV, and a single UGV. Fig. 4 and Fig. 4 show exploration performance in simulated and real-world studies respectively. We also test the sequences in Sec. 2, and the conventional-sequence required 16% more iterations than the proposed spreading-sequence in average.
|Environment Type||UGV||UAV (s)||Collaborative||UAV Avg.||UGV Avg.|
|Fence and Clusters||281||458||248||28.5||23.9|
|Real Experiment Site||94||191||59||73.9||45.0|
1. denotes the time duration from start of the exploration to exploration progress of 95%, since the inner part of the obstacles cannot be fully scanned by UGV.
2. denotes the computation time for trajectory generation.
3.Note that the simulation and the real experiment platforms are different. The trajectory generation time section of the real experiment is just for reference.
As shown in Fig. 4 and Fig. 4, the collaborative team can have significantly higher exploration efficiency than individual vehicles. The time for exploration of the system is significantly reduced, meanwhile trajectory length of each vehicle is significantly reduced. According to the result shown in Table 1, the collaborative system can have at least 13% higher exploration speed compared with single UGV, at least 84% higher exploration speed compared with single UAV and the advantage can reach up to 86% and 223% respectively. The computation time of the trajectories can all be restricted in 80ms, meaning that the method can be adopted for real time online navigation.
5 Experimental Insights
Control is always the most basic issue in robotics experiment. The control of a quadrotor system can be affected by multiple factors. The error from the actuators can significantly deteriorate the performance, affecting other parts of the system. On our aerial platform, the original DJI 2312E motor with maximum thrust 850g/ rotor was substituted with DJI 2305 Snail motor with maximum thrust 1.32kg/ rotor. Reserving larger margin on actuators can reduce the uncertainty in control segment, enhancing the control performance and it can further increase the reliability in field application.
In this experiment, the re-planning strategy is one of the most crucial issues to be considered. Time allocation is still a significant problem in path planning and it can produce serious effect on the consistency of the generated trajectories. In our system, to avoid the inconsistency caused by improper time allocation, we took trajectory length, time, initial and end state into consideration to produce an effective re-planning strategy.
-  J. Butzkey, A. Dornbushy, and M. Likhachevy. 3-d exploration with an air-ground robotic system. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3241–3248, 2015.
-  T. Cieslewski, E. Kaufmann, and D. Scaramuzza. Rapid exploration with multi-rotors: A frontier selection method for high speed flight. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2135–2142, 2017.
-  C Connolly. On the application of harmonic functions to robotics. Journal of Robotic Systems, 10, 1993.
-  J. Delmerico, E. Mueggler, J. Nitsch, and D. Scaramuzza. Active autonomous aerial exploration for ground robot path planning. IEEE Robotics and Automation Letters, 2(2):664–671, April 2017.
-  Edson Prestes e Silva, Paulo M. Engel, Marcelo Trevisan, and Marco A.P. Idiart. Exploration method using harmonic functions. Robotics and Autonomous Systems, 40(1):25 – 42, 2002.
-  Fei Gao, William Wu, Yi Lin, and Shaojie Shen. Online safe trajectory generation for quadrotors using fast marching method and bernstein basis polynomial. In Proc. of the IEEE International Conference on Robotics and Automation (ICRA), 2018.
-  Taeyoung Lee, Melvin Leoky, and N Harris McClamroch. Geometric tracking control of a quadrotor uav on se (3). In Decision and Control (CDC), 2010 49th IEEE Conference on, pages 5420–5425. IEEE, 2010.
-  Maxim Likhachev and Dave Ferguson. Planning long dynamically feasible maneuvers for autonomous vehicles. The International Journal of Robotics Research, 28(8):933–945, 2009.
-  Daniel Mellinger and Vijay Kumar. Minimum snap trajectory generation and control for quadrotors. In International Conference on Robotics and Automation (ICRA), pages 2520–2525. IEEE, 2011.
-  Tong Qin, Peiliang Li, and Shaojie Shen. Vins-mono: A robust and versatile monocular visual-inertial state estimator. arXiv preprint arXiv:1708.03852, 2017.
-  Changsheng Shen, Yuanzhao Zhang, Zimo Li, Fei Gao, and Shaojie Shen. Collaborative air-ground target searching in complex environments. In IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR), pages 230–237, 2017.
-  EI Verriest and FL Lewis. On the linear quadratic minimum-time problem. IEEE transactions on automatic control, 36(7):859–863, 1991.
-  B. Yamauchi. A frontier-based approach for autonomous exploration. In Computational Intelligence in Robotics and Automation, 1997. CIRA’97., Proceedings., 1997 IEEE International Symposium on, pages 146–151, 1997.
-  J. Zhang and S. Singh. Loam : Lidar odometry and mapping in real-time. In Robotics: Science and Systems Conference, 2014.