Enhancing Learning Capabilities of Movement Primitives under Distributed Probabilistic Framework for Assembly Tasks

This paper presents a novel distributed probabilistic framework based on movement primitives for flexible robots assembly implementation. Since modern advanced industrial cell usually deals with various tasks that are not fixed via-point trajectories but highly reconfigurable application templates, the industrial robots used in these applications must be capable of adapting and learning new skills on-demand, without programming experts. Therefore, we propose a probabilistic framework that could accommodate various learning abilities trained with different movement-primitive datasets, separately. Thanks to the fusion theory of the Bayesian Committee Machine, this framework could infer new adapting trajectories with weighted contributions of every trained datasets. To verify the feasibility of our proposed imitation learning framework, state-of-the-art movement learning framework Task-parameterized GMM is compared from several crucial aspects, such as generalization capability, accuracy and robustness. Moreover. this framework is further tested on the YUMI collaborative robot with a rivet picking assembly scenario. Potential applications can be extended to more complicated industrial assembly manufacturing or service robotic applications.


I. Introduction
In modern advanced manufacturing, the industrial robots are widely used in assembly tasks, such as peg-inhole [1], bolt screwing [2] and pick-and-place [3]. For flexible manufacturing applications, industrial robots are required to perform several tasks with various end-effectors regarding different assembly environments. Therefore robots must generate various trajectories according to different targets identified by machine vision, respectively. Assistant measurement devices, i.e., machine vision system and metrology [4], could provide a feasible target for the robot controllers. Nevertheless, they can only be applied in the certain region of interest, which more or less limits the generalisation of retrieving novel trajectories.
Generally, the core idea of assembly is to generate ordered operations consisted of a set of movement primitives, which bring individual components together to produce a novel product. Similarly, an excellent operator does have the prime skills in terms of performing assembly tasks, which promotes a feasible scenario for robots to learn from human demonstration.
The remainder of the paper is organised as follows: Section II presents the related works. Section III outlines the overall distributed probabilistic framework, including human demonstration encoding and multiple movement primitives learning. Section IV presents the comparison between the task-parameterised GMM and our proposed learning framework, along with an assembly task using ABB YUMI robot in order to verify the application feasibility. The conclusion is reported in Section V.

II. Related Works
In the context of learning from demonstration, several algorithms, i.e., probabilistic movement primitives (ProMP) [5] and dynamic movement primitives (DMP) [6], have been proposed to generate desired trajectories regarding different modulations. Both ProMP and DMP introduce various weight coefficients and factors to describe basis functions and govern explicit dynamic equations, separately. As a time-driven algorithm, the weight parameters of the basis function are learned towards an optimal function value without addressing highdimensional inputs.
In order to address high-dimensional issues and alleviate specified trajectory equations, Gaussian Mixture Model (GMM) [7] is applied to model several Gaussian distributions of demonstrations probabilistically using the EM algorithm. Combining with Gaussian Mixture Regression (GMR) [8], the novel predicted trajectories are derived from a weighted conditional Gaussian distribution. However, the capability of generating trajectories is limited by the similarity (Euclidean distance in the covariance function) [9] of the demonstration and the desired input. A similar kernel-based framework, such as movement primitives with multi-output Gaussian Process [10] and Kernelised Movement Primitives [11] could be seen as the variations of GMM/GMR, which take advantage of the kernel function to retrieve more flexible trajectories.
Although robots are usually supposed to generate feasible trajectories in a wide range of various circumstances, human demonstrations could only provide limited sets of learning instances. Therefore, in addition to abovementioned learning from demonstration (LfD) algorithms, several modified versions have been proposed to add more advanced properties in order to enhance the capability of generating adapting trajectories. Based on ProMP, a probabilistic human-robot interaction methodology is proposed in [12] collaboration with an operator. Moreover, the spring-damper dynamic behaviour regarding impedance control is discussed in [13] which is derived from DMP. A task-parameterised formulation extended from GMM is presented in [14], which essentially models movement behaviours with a set of task parameters, and therefore improving generalisation capability.

III. Distributed Probabilistic Framework
We start Subsection III-A by introducing a Gaussian process regression model that maps the Cartesian task parameters to GMM parameters. Also, the proposed framework for multiple movement-primitive datasets is carefully addressed in Subsection III-B.

A. Task-parameterised model
The individual primitive model is encoded with GMM clustering and GMR regression [7]. A graphic explanation of encoding the human demonstrations is given in Fig. 1. Therefore, the learned individual primitive model based on several human demonstrations can be represented by GMM parameters Θ = {π k , µ k , Σ k } K k=1 . Inspired by [14], if a connection between the GMM parameters and the task-specific feature is established, an individual primitive model could generate more extensions.
In order to encode the relationship between the task parameter Q and the GMM parameters Θ, we consider a regression model based on Gaussian process with the Gaussian white noise ω and the variance Σ ω . The regression model can be fully specified by mean function m f (·) and semi-positive covariance function k f (·, ·). Moreover, the kernel covariance is defined as with the length-scales Λ = diag (l 2 1 , ..., l 2 n ), the signal variance σ 2 f , and the noise variance σ 2 ω , which are defined as the GP hyper-parameters θ = {l i , σ f , σ ω }.
Given desired task parameters Q d , the new GMM parameters derived from conditional probability of Gaussian distribution are defined as where  In the initialisation step, seven Gaussian distribution models are initialised with K-means. After 48 steps training with EM algorithm, the expectation of GMM converges to predefined interval. Therefore, the trajectory is hence retrieved using GMR as shown above.
In [14], the covariance k f (Q d , Q d ) of conditional probability is neglected and only the mean m f (Q d ) is used to retrieve novel trajectories. As the hyper-parameters are not optimised, the covariance may have a negative value. In the proposed framework, The covariance is seen as the crucial information that is utilised to indicate the confidence interval in the data fusion. The hyper-parameters of the Gaussian process model should be optimised and therefore, the covariance could have a meaningful value which indicates a positive connection among different GMM parameters.
After choosing a flat p(θ), the posterior distribution is only proportional to the marginal likelihood To optimise the vector of hyper-parameters θ, we follow the recommendation from [15]. Particularly, the logmarginal likelihood can be given as Therefore, the hyper-parameter is set by maximising the marginal likelihood. Consequently, we define the partial derivatives of the marginal likelihood w.r.t the hyper-a b c d e f g h i j k l Fig. 2. Twelve movement-primitive datasets and corresponding retrieved trajectories. Each dataset generated randomly has four movement primitives. The task frame and origin frame contain the information of position and orientation shown in green and pink, respectively.
In the above equation, the two terms usually refer to the data-fit term and the model complexity. The gradient technique aims to seek the tradeoff between the data-fit and model complexity.

B. Distributed Learning
The obtained covariance k f (Q d , Q d ) of Gaussian distribution shows the confidence interval of the predictions, which could be seen as the robustness of Gaussian process regression. In this paper, the covariance of prediction is used as a data fusion indicator.
Owing to independence assumption, the marginal likelihood could be factorised into several individual terms where each factor term p k depends on the k-th individual GP regression model as discussed in Subsection III-A.
The following information details how to combine M individual primitive models to form an overall prediction with the Bayesian Committee Machine (BCM) [16]. As we can see, the BCM explicitly combines the GP prior p(f ) when making prediction.
Given M individual primitive models, the predictive distribution can be generally defined by with p(f * ) the prior over functions and D (k) , k = 1, ..., M the M dataset. Under BCM conditional independence assumption, the predictive is rewritten as Therefore, given a input x * , the posterior predictive distribution is defined as Then the mean and the precision are separately, with σ −2 * * the prior covariance of p(f * ).
IV. Evaluation In order to verify the feasibility of the proposed probabilistic framework, several experiments are implemented in this section. In Subsection IV-A, the Task-parameterized GMM is compared from several aspects, such as, generalization capability, accuracy and robustness. Furthermore, the assembly task rivets picking is given in Subsection IV-B to further demonstrate the real-world application feasibility.

A. Comparison with Task-parameterized GMM
The proposed distributed probabilistic framework aims to mutually combine and simultaneously accommodate various movement primitives in a overall scenario, and therefore augments the generalization capability and makes great use of every single movement primitive. In this subsection, we would like to explore more functions both from Task-parameterised GMM and our proposed framework in terms of generalization capability, accuracy and robustness.
Generalization capability: For exploring the generalization capability, twelve movement primitive datasets are generated randomly as shown in Fig. 2. Each dataset accommodate four movement primitives with three GMM components. Moreover, the origin frame and task frame are recorded in pink and green separately for further analysis. It is worth pointing out that basically each   3. The retrieved trajectories of the movement-primitive datasets. The generalization capability is tested with two different task frames given in green. The initial frame is shown in pink and three GMM components corresponding to each movement primitive are presented in blue, yellow and purple. In addition, each retrieved trajectory could be seen as the prediction of the Task-parameterized GMM.
dataset could be seen as a Task-parameterized GMM model.
(a) (b) Fig. 4. The retrieved trajectories of our proposed distributed probabilistic framework. For a group of twelve movement primitive datasets given in Fig. 3(a) and Fig. 3(b), our proposed framework could accommodate all the primitive datasets together and predict novel trajectories regarding the desired task frame.
Two different task frames are presented for testing the generalisation capability, as shown in Fig. 3(a) and Fig.  3(b). Particularly, for a desired task frame in green, each dataset retrieves its own predicted trajectories. Moreover, three GMM components are displayed with the mean in black dot and the covariance in blue, yellow and purple ellipses.
As shown in the Fig. 3, although all the movement  5. The confidential interval and weights corresponding to each movement primitive dataset in Fig. 3. The weights adn the confidential interval are derived from Equ. 13 and Equ. 14, respectively.
datasets give its own predictions, some of these predictions do not match the desired task frame in terms of position and orientation. This is because a single movement dataset has a limited generalisation capability. If the desired task frame is too far from the task frames of the data sample, the Task-parameterized will have a poor predicted trajectory. In addition, our proposed probabilistic framework could bear poor prediction derived from several datasets, and meanwhile output with a satisfying results as presented in Fig. 4. Accuracy and Robustness: In order to provide a more comprehensive analysis, the weights and prediction intervals of each dataset are presented in Fig. 5   Equ. 13 and Equ. 14, separately. Moreover, the prediction accuracies of each primitive dataset and our proposed distributed framework are compared in Fig. 7. As shown in Fig. 5, the confidential interval in red bar, shows the prediction range of each movement dataset. If the confidential interval is large, then the corresponding movement dataset would lose its confidence in predicting novel trajectories. On the contrary, if the confidential interval is narrow, then the movement dataset has more faith in its own prediction.

derived from
As shown in Fig. 7(a), the prediction error of each primitive dataset is nearly proportional to the confidential intervals in Fig. 5(a). The similar situations can also be observed in the other three group simulations, i.e., Fig.  7(b) and Fig. 5(b). This is why we use the information of the confidential intervals of each primitive datasets are used to quantitatively explain the weights applied in Equ. 13. In addition, the proposed distributed framework shown in green, gives a better prediction accuracy compared with the accuracy from each primitive datasets shown in yellow according to the two prediction errors given in Fig.7(a), and Fig. 7(b). In order to obtain a better prediction, sometimes we could choose high-weight predictions from several datasets [17], [18], which will make our distributed framework more effective.

B. Assembly Tasks
After addressing all the key issues of our proposed distributed framework in Subsection IV-A, in this subsection, the feasibility with real world experiments should be verified. As presented in Fig. 8, we test our proposed framework with ABB YUMI robot. The YUMI is a two-arm collaborative robot with industrial camera mounted on the wrist of the right arm and the payload is 0.5kg for each arm. To amplify the function of the YUMI and compensate the picking accuracy, two grippers are equipped with two arms, respectively. Rivet picking: The experiment implemented in this subsection is picking rivets from the rivet block as shown in Fig. 9. We collect twelve groups of human demonstrations as given in Fig. 6, along with the trajectories in Fig.  10(a). As shown in Fig. 10(a), the collected demonstrations have some inaccuracies. Particularly, the demonstrations are not smooth enough and some of them may not be successfully inserted into the holes of rivet block. Each of the primitive-dataset group is trained with three GMM using EM algorithm as presented in Fig.  10(b). Moreover, it is observed in the Fig. 10(b) that if the training dataset is decentralised, the GMM ellipsoid is large and the Gaussian process model has a wide distribution and verse versa.
Besides, we construct a Gaussian process regression between the task frames of twelve groups primitive datasets and corresponding GMM model parameters. Under our proposed distributed framework, the twelve novel trajectories are inferred in Fig. 10(c), along with the desired Fig. 10. The learning process of our proposed distributed probabilistic framework. The training trajectories are collected in Fig.  10(a). Then, the trained GMM components of each dataset using EM algorithm is given in Fig. 10(b). The retrieved trajectories inferred by twelve movement primitive datasets are shown in Fig. 10(c) and the desired frames are given in green markers. Finally, the confidential interval of each three consecutive trajectories according to GMR are given in Fig. 10(d), Fig. 10(e), Fig. 10(f) and Fig. 10(g).
task frames in square black and green makers and the origin frames in square black and yellow makers.
To reveal further details, each three consecutive predicted trajectories are separated in four figures as represented in Fig.10(d), Fig. 10(e), Fig. 10(f) and Fig. 10(g). Additionally, the confidential intervals derived from GMR are plotted in green. The prediction errors are given in Fig. 11. As we can see in the figure, all the prediction errors are below 0.35mm. Most of the prediction errors are lower than 0.2mm, which is the reference assembly precision of the aerospace manufacturing. However, we still notice that the prediction error of the ninth hole is larger than 0.2mm and the second is even higher than 0.3mm. This is mainly caused from the accuracy of the human demonstrations. If all the human demonstrations are far from the desired target, the prediction will have a poor retrieved trajectories. We would like to point out that the above precision or prediction errors are enough accurate for picking applications, such as picking rivets. Theoretically, the proposed distributed framework can manage to keep the prediction errors below 0.2mm with more accurate demonstrations.

V. Conclusion
In this paper, we propose a novel distributed probabilistic framework, which combines various movement primitives together in a weight-based scenario. In order to automatically optimise the GMM models, the regression model based on the Gaussian process are trained with Evidence Maximisation. Given a desired task frame, the retrieved trajectories are predicted using Bayesian Committee Machine. The assembly tasks experiments show the application feasibility of the our proposed framework. Future work will focus on the movement primitives library as well as the enhancement of the precision.