Optimization driven kinematic control of constrained
collaborative mobile agents with high mobility
Abstract
Industrial robots, in particular serial industrial manipulators, have enabled a lot of recent research in large scale robotic systems such as used in construction robotics, robotics in architecture. However, industrial manipulators have very low payload to weight ratio, are too generic systems with inflexible hardware and software for task adaptability. High mobility large scale robotic systems often involve even heavier mobile bases to move around these industrial manipulators, thus even further lowering the payload to weight ratio of such systems. Moreover, such system architecture is inflexible and eventually reaches its limits when higher mobility is demanded such as higher overall reach with same payload. This paper presents a concept of constrained collaborative mobile agents where the actuated mobile agents are constrained by a passive kinematic structure whose topology can be inexpensively configured according to different functions, task and mobility requirements. The type and number of mobile agents, the choice of actuation scheme are other important characteristic of this system which can be altered to improve system performance. A novel optimization framework for modeling and kinematic control of such systems is presented which is flexible to the abovementioned system elements and characteristics. Finally, two prototypes are presented which are used to demonstrate the optimization driven kinematic control of the system with different topologies.
Collaborative robots, Multirobot systems, optimization driven control
I Introduction
From structured warehouses to unstructured environments such as construction sites, agricultural fields, research into large scale robotics systems is fuelled by demands for increased mobility, task adaptability while able to perform various functions and tasks in such environments. In particular for construction robotics or robotics in architecture, digital fabrication techniques increasingly make use of mobile robotic systems [1]. These mobile robotic systems typically include a fully actuated industrial manipulator mounted on a single mobile base which can be wheeled or tracked [2, 3]. The disadvantages of such robotic systems include very low payload to weight ratio, inflexibility in the hardware and software for task adaptability. When certain tasks require higher mobility requirements such as large overall reach, such systems become infeasible because first there might not be existing industrial solutions and second a system meeting such requirements would be prohibitively heavy to operate, for example on floors in a construction site.
As such mobile robotic systems comprising of fully actuated industrial manipulators on a single mobile agent are not scalable and adaptable in hardware to changing task and corresponding mobility requirements. Attempts have been made to solve this problem of scalability and adaptability by completely different hardware system architecture comprising of multiple mobile agents manipulating a single endeffector. For example, multiple flying agents were used for building an architecture scale installation in the work [4]. Another approach consists in using wheeled mobile agents manipulating an object which is suspended through several cables which are in turn attached to the mobile bases [5, 6]. Commonality in this hardware system architecture is use of multiple mobile agents as the only actuators in the system. Flight based mobile agents are suitable for discrete material handling, logistics but they can not replace industrial robots where endeffector manipulation requires rigidity, stiffness and capability of much higher payloads. Cable driven object manipulation through actuated mobile agents have the capability of providing the rigidity, stiffness and higher payloads. However to fulfil these requirements the topology of the cables encircles the object being manipulated. So though these cable driven systems can be useful for logistics and material handling, they also can not replace the industrial manipulators for variety of in situ digital fabrication tasks such as drilling, robotic brickwork [7], robotic formwork [8] where unencumbered workspace around the endeffector is desired.
Hardware system architecture, where a passive kinematic chain with rigid links is connected to multiple mobile agents acting as actuators, would allow to build scalable and task adaptable systems with high mobility which have the potential to replace conventional large scale mobile robotic systems for a variety of tasks. Owing to high rigidity of the links, flexible topology and arrangement of the kinematic chain around the endeffector, such systems would allow for open workspace to carry out traditional in situ digital fabrication tasks not possible with cable driven systems. The work of [9, 10, 11] present specific systems with wheeled mobile agents which are connected to a common endeffector through a passive kinematic chain. The topology of the passive chain is limited to DOF (degree of freedom) consisting of a single link with two passive joints, where as the task adaptability might require and necessitate consideration of other possible topologies. Choice of the actuation scheme is also conservative in these papers where number of actuators exactly match the DOF of the endeffector. This misses a big potential advantage of such systems where overactuation, with use of multiple mobile agents with varying actuated DOF, can be exploited to enhance their performance. Kinematics of these specific systems was solved individually finding closed form solutions, however inverse and especially forward kinematics problem for such systems with parallel architecture more often does not have a closed form solution. As such a unified optimization framework for doing the kinematic control of such systems where topology and actuation scheme can be altered for task adaptability and system performance is missing.
Our long term goal is to leverage the advantages brought by combining robotic mobility and manipulation capabilities. To this end, in this paper we show that driven by appropriate control systems, very simple robots equipped with very simple manipulators can be quite dexterous. The mobile manipulation systems we study in this work have much greater workspace and reach than stationary robots, they are very lightweight, and they can easily be reconfigured to fit the needs of different tasks. In this paper, we make following contributions:

A general concept for constrained collaborative mobile agents (CCMA), in which an endeffector is connected via a passive parallel kinematic chain to multiple mobile agents

Varying topology of the kinematic chain and actuation schemes of the mobile agents for task adaptability

A novel unified optimization framework for modeling and kinematic control of such systems

Prototypes demonstrating the optimization driven kinematic control of the CCMA system with different topologies in simulation and experiments
In section II, we discuss the CCMA concept in detail. The modeling and optimization framework for these systems is described in section III. Prototypes are presented in section IV and their kinematic control utilizing the optimization framework is demonstrated. Finally, we discuss the challenges faced in the current work and the directions for the future work in section V.
Ii Constrained collaborative mobile agents
CCMA as a general concept (Fig. 1) consists of a number of mobile bases, each having  DOF which move collaboratively in order to manipulate one or several endeffectors. The mobile bases can vary from standard wheeled mobile robots to quadruped mobile robots. One of the prototypes presented in the paper (more detail in section IV) is shown in the Fig. 2, which has omnidirectional bases constrained by a passive kinematic chain carrying an endeffector. The CCMA system, more specifically its endeffector is manipulated solely by the movement of the mobile bases where each DOF of the mobile base can be counted as one actuator. So for example in the Fig. 2, we have a total of DOF in actuation for manipulating a DOF endeffector. By adding more mobile bases, number of actuators in the CCMA system can be changed. Also topology in the Fig. 2 is just one of possible choices.
Thus we have a system which has many more actuators than the DOF of the endeffector which is to be controlled and at the same time, we can choose several different topologies for the passive kinematic chain. While the varying topology of the passive kinematic chain provides a lot more flexibility in the hardware which can be inexpensively configured, a general optimization framework is needed and presented in this work (more detail in section III) for flexibility in the software to easily swap different topologies, number of mobile bases and different actuation schemes. By different actuation scheme, we mean that we may decide to use only certain DOF of the mobile bases for actuation. Also having mobile bases as actuators mean that the resulting CCMA system can have very large translational workspace especially along the ground plane ( plane) and large orientation workspace along the vertical axis . Thus the CCMA system has the potential to be scalable to large scale systems with high mobility. This is in contrast to the parallel robots [12], where a similar passive kinematic chain with an endeffector is actuated with motors at the fixed base. For parallel robots, usually the number of actuators matches exactly the number of DOF of the endeffector and they have very limited workspace both in translation and orientation.
Iii Modeling and optimization framework
This section presents a general modeling and optimization framework for the kinematic control of the CCMA system which is independent of the topology of the passive kinematic chain and can be easily configured for different actuation schemes. The CCMA system can be represented as an assembly of rigid bodies from mobile bases to the endeffector which are connected to each other by different type of joints such as revolute, prismatic etc. For the mathematical modeling, we utilize the constraint based formulation presented in the paper [13]. Let be the number of the rigid bodies. Every rigid body has DOF, thus the state of the CCMA system has elements, corresponding to the position and orientation in 3D, of every rigid body in it, relative to the world reference frame (,,,). Different passive joint connections such as revolute and prismatic impose constraints , as explained in the paper [13].
For the CCMA system in this paper, we count each mobile base as one rigid body and it has a special connection to the world reference frame (,,,) depending upon its type. For example, an omnidirectional wheeled mobile robot will have a planar joint connection with the world, where it is free to translate along and axes and rotate around axis. We introduce a special type of connection which connects the mobile base to the world. This mobile connection imposes constraints and allows freedom depending on the type of the mobile agent. The state of the CCMA system consists as well the full state of the rigid body representing the mobile agent. The scalar constraints owing to the mobile joint connection can be simply formulated as , where is the component of the rigid body state of the mobile agent which can’t be controlled. Let also introduce the control vector , which consists of the part of the state of the rigid body, representing a mobile agent, which can be controlled. In order to control the CCMA system, we introduce actuated mobile base connections which output actuation constraints formulated as . For example, for an omnidirectional wheeled agent, index in will correspond to the z coordinate of the rigid body representing the mobile base, and the two orientations about the and the axis. The control vector , in this specific case, will consist of the translation along and axes and rotation about the axis. As multiple mobile agents are considered in this paper, full control vector may not be used as number of actuator far exceeds the DOF of the endeffector which needs to be controlled. Therefore a subset of the control vector may be considered which is what we refer to as different actuation schemes. For different actuation schemes, additional constraints must be added to account for control variables not used for actuation. The constraints including the ones imposed by the mobile , and passive connections are then collected in a vector of constraints . Minimizing the below cost function gives a feasible state of the CCMA system which satisfy all the constraints.
This allows us to simulate the CCMA system and solve the forward problem which is to find the endeffector state, , when the control vector or is given. From now on, we just use the full control vector in further discussion but corresponding to different actuation scheme can be interchangeably used.
In order to solve the control problem which is to find for desired endeffector states , we need to solve the optimization problem below and calculate the derivatives of the objective function O with respect to the control variables in order to use a gradient based function minimizer such as BFGS or Newtons.
However, the expression for is not generally analytically available and use of finite differences to compute it will require to minimize the constraint energy function k times where k is the size of the control vector to high degree of accuracy. This is computationally too demanding for a real time simulation and control tool. We solve this problem by doing the sensitivity analysis over the gradient of the constraint energy function rather than over the constraint vector itself as done in the paper [13]. Because constraints cannot be assumed to be satisfied during the intermediate iterations of the optimization of the objective function . On the other hand, the gradient of the constraint energy function is equal to zero when a feasible state of assembly is achieved and the optimization converges. To express it more clearly for every control vector , you can find a suitable such that gradient of the constraint energy function is zero.
Doing the sensitivity analysis over this gradient leads to the following equation:
is the hessian of the constraint energy function , analytical expressions for which we calculate. in general can be calculated using finite differences as the state is assumed to be constant and the constraint energy function does not need to be minimized. Thus we can calculate and therefore the gradient of the objective function . This gradient can then be used to update the control vector in the current step of the optimization problem as follows:
To calculate the step size using exact Newton method, we had need anlytical expression for the derivative of the hessian of with respect to state , which we don’t have. Therefore, we approximate the step size using the BFGS quasiNewton method.
The sensitivity analysis is based on the assumption that the gradient of , , is zero, which leads to the calculation of the gradient and of control vector . Since the CCMA system consists of closed loop kinematic chains, might still be zero when is not zero. This would mean that CCMA system does not assemble properly as total constraint energy is not zero. This is not acceptable for control as for corresponding , we expect to be zero as well. We solve this problem by reformulating the objective function and corresponding gradient by adding the E and , respectively, as follows, where :
With this new formulation of the objective function and corresponding gradient, convergence of the optimization was much improved and the total constraint energy also goes to zero when we calculate control variables for desired endeffector states .
Iv Application demonstrators
In order to demonstrate the control of CCMA system with the optimization framework developed in the previous section, we simulated and fabricated a DOF and a DOF prototype, as shown in the Fig. 3 and 4. The Fig. 5 shows the different topologies used for generating DOF and DOF prototypes.
In order to determine the position and orientation of the mobile bases with respect to the world reference frame, a motion capture system comprising of OptiTrack Prime cameras, as shown in the Fig. 6 was used.
Fig. 7 shows the omnidirectional base used for proof of concept demonstrators in the paper. Each mobile base has optical markers which can be tracked with the motion capture system. Motion capture system then sends the rigid body position and orientation in the world reference frame to the computer running the optimization. In order to wirelessly transmit the control signals to the mobile bases for the control of the CCMA demonstrators, we use wireless transreceivers XBees from Digi International. Each mobile base is equipped on its back with an XBee. The computer running the optimization sends the control commands to a microcontroller via serial communication. This microcontroller has a transreceiver XBee which acts as the coordinator and broadcasts the respective control signals to the respective XBees in each mobile base. The optimization framework first calculates the states of the rigid bodies representing the mobile bases. The position and orientation of the mobile bases are then converted to the corresponding wheel speeds based on the difference between set state and the tracked state by the mocap system, using standard mobile robot kinematics for swedish wheels based omnidirectional mobile robot [14].
Also while it is possible to calculate directly the final control vector for desired in simulation, for hardware experiments we did steps of small size both in endeffector position and orientation until we reach the desired . This leads to smoother motion of the CCMA system especially during endeffector rotations. Please refer to the accompanying video for results in simulation for the DOF and DOF CCMA systems and experimental results demonstrating the different kinds of motion for the DOF CCMA system.
V Conclusion and Future Work
In this work, we have introduced the general concept of constrained collaborative mobile agents which have the potential to be scalable and adaptable according to the different task requirements. We have developed a novel optimization framework using sensitivity analysis which allows for much greater flexibility in software to test different topologies of passive kinematic chain, different type of mobile agents and different actuation schemes. With results in simulation and proof of concept prototypes, we have demonstrated the efficacy of the developed optimization tool for the kinematic control of such systems.
Due to use of mobile bases, the robots presented in paper can be quite dexterous and have large workspace. However, certain workspaces such as translation workspace in z direction can be limited by the design parameters such as link lengths in the passive kinematic chain. As a future work, we aim to do the design optimization of the CCMA system for certain prescribed workspaces. Since the CCMA system effectively includes closed loop kinematic chains, treatment of singularities especially parallel singularities would be necessary in the optimization framework for the control to detect and avoid regions of instability. The multiple actuation schemes, overactuation and increase in number of mobile bases would be exploited to provide potential solutions in the future work. Our simulation and optimization framework is a real time control capable tool. However, in order to enable closed loop feedback control over wireless networks for these untethered mobile systems, different sensing/localization systems and different communication protocols need to be explored to maintain the real time control capability with a high bandwidth.
As these systems are scaled, they are bound to exhibit a lot more flexibility and compliance which is not necessarily a bad outcome, as the compliance can be good for safety reasons, but its effects have to be studied and possibly included in the simulation framework for accurate prediction of the endeffector states. The play in the joints during the fabrication process can have adverse effects on the mobility of these systems as it introduces unmodelled compliances and might make the system uncontrollable. This is what we observed for the DOF prototype that due to play in the joints combined with the compliance of the plastic pipes, the fabricated system did not behave as in simulation. Therefore these issues related to proper fabrication to maintain the strict geometric arrangement of the axes and modeling of the compliance in the passive kinematic chain due to link deformation would be studied in the future work. In this work, we presented robots actuated with omnidirectional mobile bases. In the future work, we aim to work with different mobile bases such as standard wheel mobile bases or quadruped robots which would lead to very different kinds of CCMA systems.
Vi Acknowledgment
References
 [1] R. Loveridge and T. Coray, “Robots on construction sites: The potential and challenges of onsite digital fabrication,” Science Robotics, vol. 2, no. 5, p. eaan3674, Apr. 2017. [Online]. Available: http://robotics.sciencemag.org/content/2/5/eaan3674
 [2] M. Giftthaler, T. Sandy, K. Dörfler, I. Brooks, M. Buckingham, G. Rey, M. Kohler, F. Gramazio, and J. Buchli, “Mobile Robotic Fabrication at 1:1 scale: the In situ Fabricator,” Construction Robotics, vol. 1, no. 14, pp. 3–14, Dec. 2017, arXiv: 1701.03573. [Online]. Available: http://arxiv.org/abs/1701.03573
 [3] S. J. Keating, J. C. Leland, L. Cai, and N. Oxman, “Toward sitespecific and selfsufficient robotic fabrication on architectural scales,” Science Robotics, vol. 2, no. 5, p. eaam8986, Apr. 2017. [Online]. Available: http://robotics.sciencemag.org/content/2/5/eaam8986
 [4] F. Augugliaro, S. Lupashin, M. Hamer, C. Male, M. Hehn, M. W. Mueller, J. S. Willmann, F. Gramazio, M. Kohler, and R. D’Andrea, “The Flight Assembled Architecture installation: Cooperative construction with flying machines,” IEEE Control Systems, vol. 34, no. 4, pp. 46–64, Aug. 2014.
 [5] T. Rasheed, P. Long, D. MarquezGamez, and S. Caro, “Kinematic Modeling and Twist Feasibility of Mobile CableDriven Parallel Robots,” in Advances in Robot Kinematics 2018, ser. Springer Proceedings in Advanced Robotics. Springer, Cham, July 2018, pp. 410–418. [Online]. Available: https://link.springer.com/chapter/10.1007/9783319931883_47
 [6] B. Zi, J. Lin, and S. Qian, “Localization, obstacle avoidance planning and control of a cooperative cable parallel robot for multiple mobile cranes,” Robotics and ComputerIntegrated Manufacturing, vol. 34, pp. 105–123, Aug. 2015. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0736584514001033
 [7] K. Dörfler, T. Sandy, M. Giftthaler, F. Gramazio, M. Kohler, and J. Buchli, “Mobile Robotic Brickwork,” in Robotic Fabrication in Architecture, Art and Design 2016. Springer, Cham, 2016, pp. 204–217. [Online]. Available: https://link.springer.com/chapter/10.1007/9783319263786_15
 [8] N. Kumar, N. Hack, K. Doerfler, A. N. Walzer, G. J. Rey, F. Gramazio, M. D. Kohler, and J. Buchli, “Design, development and experimental assessment of a robotic endeffector for nonstandard concrete applications,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), May 2017, pp. 1707–1713.
 [9] Z. Wan, Y. Hu, J. Lin, and J. Zhang, “Design of the control system for a 6DOF Mobile Parallel Robot with 3 subchains,” in 2010 IEEE International Conference on Mechatronics and Automation, Aug. 2010, pp. 446–451.
 [10] Y. Hu, Z. Wan, J. Yao, and J. Zhang, “Singularity and kinematics analysis for a class of PPUU mobile parallel robots,” in 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO), Dec. 2009, pp. 812–817.
 [11] Y. Hu, J. Zhang, Y. Chen, and J. Yao, “Type synthesis and kinematic analysis for a class of mobile parallel robots,” in 2009 International Conference on Mechatronics and Automation, Aug. 2009, pp. 3619–3624.
 [12] J. P. Merlet, Parallel Robots, 2nd ed., ser. Solid Mechanics and Its Applications. Springer Netherlands, 2006. [Online]. Available: //www.springer.com/de/book/9781402041327
 [13] S. Coros, B. Thomaszewski, G. Noris, S. Sueda, M. Forberg, R. W. Sumner, W. Matusik, and B. Bickel, “Computational Design of Mechanical Characters,” ACM Trans. Graph., vol. 32, no. 4, pp. 83:1–83:12, July 2013. [Online]. Available: http://doi.acm.org/10.1145/2461912.2461953
 [14] R. Siegwart and I. R. Nourbakhsh, Introduction to Autonomous Mobile Robots. Scituate, MA, USA: Bradford Company, 2004.