Ant Colony Optimization Algorithm for Industrial Robot Programming in a Digital Twin

Advanced manufacturing that is adaptable to constantly changing product designs often requires dynamic changes on the factory floor to enable manufacture. The integration of robotic manufacture with machine learning approaches offers the possibility to enable such dynamic changes on the factory floor. While ensuring safety and the possibility of losses of components and waste of material are against their usage. Furthermore, developments in design of virtual environments makes it possible to perform simulations in a virtual environment, to enable human-in-the-loop production of parts correctly the first time like never before. Such powerful simulation and control software provides the means to design a digital twin of manufacturing environment in which trials are completed at almost at no cost. In this paper, ant colony optimization is used to program an industrial robot to avoid obstacles and find its way to pick and place objects during an assembly task in an environment containing obstacles that must be avoided. The optimization is completed in a digital twin environment first and movements transferred to the real robot after human inspection. It is shown that the proposed methodology can find the optimal solution, in addition to avoiding collisions, for an assembly task with minimum human intervention.


INTRODUCTION
Usage of artificial intelligence approaches in a manufacturing environment is quite challenging because of inevitable iterations required to be performed before success.In a manufacturing environment, such iterations are highly expensive and may cause serious damages which make them impossible to implement.A digital twin is a virtual representation of real world which is designed for various purposes such as product design, smart manufacturing, usage monitoring and maintenance, repair and overhaul [1] of which smart manufacturing applications fell into the scope of this paper.The complete production line from raw material to market can be designed and optimized in a factory floor [1], [2].Such virtual environment make trial and errors required by machine learning and artificial intelligence approaches possible.An exact and realistic digital twin is desirable to make sure that the iteratively found solution is applicable to the system right from the first time.
Many applications require robots to move through obstacles to reach a desired position.Path planning is therefore commonly used in navigation for autonomous cars, UAVs, mobile robots, and satellites to enable collision free motion [3] [4].Similarly, in manufacturing tasks autonomous path planning of industrial robots is required due to product changes.Particularly in assembly tasks where changing the product design often necessitates changes in robot movement to accommodate new variations in production methods, currently done using manual reprogramming.
Path planning algorithms can be broadly classified as conventional and heuristic approaches [5].Conventional approaches which are old algorithms require manual programming at some places and are computationally expensive as compared to heuristic approaches [6].Examples of conventional approaches [5] are BUG algorithms, roadmap, cell decomposition, potential fields, and mathematical programming.Such approaches may fall in local minima more frequently than heuristic optimization algorithms.Heuristic path optimization approaches are autonomous path planning methods that have recently been developed.A few of the commonly used heuristic methods [5] are neural network [7], [8] and [9], genetic algorithms, particle swarm optimization [10], ant colony optimization (ACO) [11] and [12], stigmergy, wavelet theory, fuzzy logic [12] and [13], and Tabu Search.Heuristic approaches are preferred over conventional methods as they are better in dealing with complicated problems and local minimum solutions.Ant colony optimization was used for path planning in this study as it is easy to employ and adapt to a new problem [14].
In case of industrial robots, it is highly desirable to make robot autonomously find an optimal collision free path.This became more important as in new industrial systems manufacturing customizable products emerge flexible factory line.Soft computing approaches have been previously applied to such applications.For instance, continuous Genetic Algorithms is used to find the Collision-Free Cartesian Path for robotic manipulators [15].Optimal positioning of surgical robot and path planning of its end-effector in a crowded operating room is another challenging task to which genetic algorithm is applied [16].A 3D working model including surgical object is designed and the optimization is performed in such an environment.
In this paper, the ACO is used in a digital twin environment to find the shortest collision free path in 3D space for an industrial robot undergoing a pick and place operation within an environment containing obstacles that Proceedings of the 25 th International Conference on Automation & Computing, Lancaster University, Lancaster UK, 5-7 September 2019 must be avoided.To reduce risks and number of experiments done with the robot, in this case a UR5, a digital twin of the environment is designed in Gazebo software.The obstacles designed in the digital twin have been created to represent the existence of obstacles in a production environment.The digital twin model has been created to detect possible collisions and are used to penalize the cost function when obtaining a collision free solution.Possible robot movements in 3D space are then encoded in terms of an array that represent translation and rotation of the end factor in the virtual environment.Inverse kinematics is then used to make robot joints move to the required position, where collisions will result in displacement of obstacles in the digital twin.By comparing the initial and final position of obstacles, a possible collision is detected automatically without carrying out manual calculations.As multiple calls of the cost function are required during the optimization process, the position of the robot and other components in the digital twin are required to reset to their initial conditions after each cost function execution.An optimal path was generated in virtual environment by integrating ACO algorithms to find a collision free route to fulfill the pick and place task.The resulting digital twin therefore provides a means to test obtained movement in a visual interface by a human expert to guarantee implementability and safety.In the simulation section, it is shown that the proposed method is a successful optimization algorithm to find the optimal path in the presence of obstacles.This paper is organized as follows.The general ACO technique is described in Section II.Then the overall methodology and description of steps taken to perform the optimization in a digital twin are given in Section III, simulation results are presented in Section IV, and concluding marks are provided in Section V.

II. ANT COLONY OPTIMIZATION
ACO is a nature inspired optimization algorithm.It is based on ant's behavior to use swarm intelligence to find food [17].ACO is a heuristic path planning method used to find optimal path.In this method, ants deposit pheromones on the path while travel that are sensed by other ants.Figure 1 shows a group of ants finding an optimal path from nest to food.The changes paths taken can be easily accommodated in ACO as pheromones evaporate with time.Pheromones deposition speed is faster than the evaporation rate in the paths visited more frequently by ants due to that path's short length.ACO is easy to implement, adaptable to new problems and can achieve both local and global minimum solutions [14].This means that whenever a new feature is added to the path pheromone levels can be locally initialized by just changing the pheromone values near the object added or can be globally initialized by setting all the pheromones back to original level.ACO equations were used to employ a group of ants and update pheromone levels to find desired solution.Equations used (equation 1-4) are shown below [17].The probability of ant k located at node i to go to node j depends on heuristic information and pheromone update in Equation (1).
Heuristic information depends on movements given to reach target and position number of ways ant can move.Pheromones remaining due to evaporation are determined as: where, represents pheromones deposited on the path from nodes i to node j, ρ is pheromone evaporation rate, and is selected as 0 ≤ ρ ≤ 1. Pheromone levels after a new pheromone is deposited then become: where k represents ant doing path planning and is associated with cost or reward ant k gets for choosing this path.The algorithm evolves for a few iterations to find the optimal path with highest concentration of pheromones.

A. Cost function to be optimized
The digital twin designed in this paper is a replica of real industrial process which includes industrial components and machinery.The main goal is to automatically design the path for a robot end effector to move from its initial condition to a designated target position and orientation, without having any collision with obstacles in the environment.Collision detection is done automatically in the digital twin using specially written software in Python.Collision between the robot and components make the objects move from their initial position.By comparing initial position of components and their final position, a flag is returned as a consequence and can then be used to penalize the cost function.Collisions during movement are captured using software with the cost function defined as:

 
.   where , , , , and represent the desired values of position and orientation of robot, P represent the penalty term and C which is assigned a discrete value representing the absolute displacement value of any of investigated components.The parameters , , , , and are the position and orientation of robot before the last movement to the destination.Displacement values larger than software tolerance are considered to be collision.Two values are assigned for small displacements and large displacements in components.Following formula summarizes this logic.
where and represent penalties given for the case when the sum of absolute values of displacements are larger than and and is the tolerance of software.

B. Encoding movements
To optimize movements, they need to be encoded into an evolvable array.The ACO then evolves this array to find the optimal solution minimizing the cost function.The python code is then integrated with a gazebo model and the end effector of the robot moves in 3D space using an array containing integer values from the set 0,1, … ,11 .To move the robot, joint angles corresponding to movements are required.Positions of the end effector were used to find rotation matrix (equation 8-11) then used to find joint angles of robot using inverse kinematics equations.The interpretation of movements is presented in Table I. .

Kinematic and inverse kinematics of UR5
The values obtained from the movement array need to be translated in terms of robot movements.To enable this a transformation matrix that represents position and orientation of robot needs to be obtained [18] [19].The transformation matrix for frame 6 with respect to frame 0 is as: , , , , , 0 1 0 0 0 1


where , and are unit vectors defining the axis of frame 6 in relation to frame 0 and: in which the rotational matrices are obtained as follows: Inverse kinematics is then used to find joint angles of robot to move to position given by ACO.The angles of robot are then obtained as [18] [19]:

D. Collision detection
To evaluate the cost function, it is required to detect collisions and penalize the optimization routine accordingly.The position of components are published from Gazebo 7.0, which can be used to detect collision by comparing final positions with initial positions of the components.This is automatically done within the software as the position of components are reported as rostopic which can be used to detect possible collisions.Figure 2 illustrates a possible collision generated by robot moving according to a solution generated by ACO.As can be seen from the figure, Gazebo completely simulates such collisions and considers physical interaction between components including friction and gravitational force.

SIMULATION RESULTS
The results obtained for 3D path planning of industrial robots in digital twin using ACO is presented in this section.The cost function and the movements are as defined in Section III.Number of movements are considered to be equal to 20, number of ant agents considered is equal to 10 and the optimization algorithm is iterated for 100 times.
Figure 4 compares the real world and the digital twin designed for that.Figure 5 illustrates the method used for this simulation.The ACO is implemented under python software that communicates with ROS using rostopics.The results of simulations in terms of final distance between the end effector and desired position as well as possible displacement in obstacles are reported back in terms of corresponding rostopics that are then used in Gazebo to evaluate movements.The whole algorithm is iterated for 100 iterations before its convergence.The   IV. CONLUSIONS To implement product changes rapidly on the factory floor, manufacturing elements are required to learn to perform given tasks autonomously with minimum human intervention.Machine learning typically requires trial and error approaches that in most cases cannot be performed directly on the factory floor due to factors such as health and safety and to reduce waste.The utilization of a digital twin of a robotic manufacturing system provides the mean for performing multiple learning iterations at almost no cost.In such ACO as a successful, nature inspired, method to deal with path planning problems is utilized here in the planning of a pick and place task.The task is encoded for ACO and a cost function is considered to plan a path to the desired position without any collisions with environmental objects.Simulation results demonstrate that ACO can successfully perform the optimization in the digital twin using the proposed procedure.

Figure 1 .
Figure 1.Ants finding optimal path to food

Figure 2 .
Figure 2. Collision due to movement generated by an ACO solution in digital twin E. Methodology Figure 3 demonstrates the steps taken to perform automatic path planning for UR5 in digital twin using ACO.The green box in this figure represent the steps related to the evaluation of the cost function.

Figure 3 .
Figure 3. Steps taken for 3D path planning in simulation environment

Figure 4 .
Figure 4.The digital twin of environment including UR5 robot vrrsus real robot

Figure 5 .
Figure 5. Communication between Python and ROS movements are implemented in digital twin using /trajectory_controller/command rostopic in terms of robot joint angles.The positions of obstacles are reported back using /gazebo/modelstates rostopic.The rostopic /gazebo/reset_world is used to reset the position of objects after each evaluation of the cost function.

Figure 6
Figure 6 illustrates the evolution of cost function versus iterations.As can be seen from this figure, ACO has effectively been successfully in reducing the value of cost function.As can be seen from the figure ACO has successfully reduced the value of cost function from 1.37 to 0.31 which is 77% of improvement.The result of doing optimization in terms of motion time-lapse of robot is presented in Fig. 7.This figure shows that the final motion obtained is collision free and end effector successfully hits its target destination.

Figure 6 .Figure 7 .
Figure 6.The evolution of cost function versus iterations