The Goods and Bads in Dyadic Co-Manipulation: Identifying Conﬂict-Driven Interaction Behaviours in Human-Human Collaboration

—One of the challenges in collaborative human-robot object transfer is the robot’s ability to infer about the interaction state and adapt to it in real time. During joint object transfer humans communicate about the interaction states through multiple modalities and adapt to one another’s actions such that the interaction is successful. Knowledge of the current interaction state (i.e. harmonious, conﬂicting or passive interaction) can help us adjust our behaviour to carry out the task successfully. This study investigates the effectiveness of physical Human-Human Interaction (pHHI) forces for predicting interaction states during ongoing object co-manipulation. We use a sliding-window method for extracting features and perform online classiﬁcation to infer the interaction states. Our dataset consists of haptic data from 40 subjects who are partnered to form 20 dyads. The dyads performed collaborative object transfer tasks in a haptics-enabled virtual environment to move an object to predeﬁned goal conﬁgurations in different harmonious and conﬂicting scenarios. We evaluate our approach using multi-class Support Vector Machine classiﬁer (SVMc) and Gaussian Process classiﬁer (GPc) and achieve 80% accuracy for classifying general interaction types.


I. INTRODUCTION
Physical human-human interaction (pHHI) is complex; it involves good interpersonal coordination and mutual role adaption Melendez-Calderon et al. [1]. These help humans to determine how and when their partner's goals and the overall interaction states change, allowing them to enhance their movements Takagi et al. [2]. Learning how and when the interaction states change in pHHI has important implications for physical human-robot interaction (pHRI). A robot which can accurately infer the current interaction state can use that information to adjust its behaviour to better complement the human partner during pHRI. In [3], we presented a feature extraction method to perform online classification for distinguishing between interaction states during pHHI as an effort to understand how two human partners' interactive states change over physical collaboration. This paper summarizes our classification results using the data collected during a dyadic object transfer using Madan et al.'s behavior taxonomy [4].

II. BACKGROUND
The data was collected using a virtual environment where human dyads interact through the haptic channel [4]. 40 volunteers, who got randomly matched to form dyads, participated in the study. The dyads collaborated in order to move an object in between target configuration. Two scenes with various scenarios that involve rotation and translation movements were created to provoke a range of different interaction patterns inducing conflicts and harmony. Madan et al.'s taxonomy assumes that there are three main types of interaction in any collaborative task between humans: T1. Harmonious interaction, T2. Conflicting interaction, and T3. Neutral interaction. Using this assumption we observed the frequently emerging patterns from the interaction and classed them into 6 task dependent interaction pattern classes as shown in Figure

III. METHODOLOGY
The dataset consists of variable length annotated interaction segments. In order to perform online classification we set a short window to extract a small sequence to be used for feature extraction. We extract 2 seconds worth of features every 1.5 seconds. The window parameters are empirically set such that enough data is mined for accurate classification while also taking into account human reaction times.

A. Online feature extraction
Before we begin with the extraction process, we prepare the raw timeseries data for processing. We assign every data point a label that matches the annotation defining the class of the corresponding interaction segment. As we iterate through the interactions, our window encounters segments that belong to more than one class. This can result in ambiguity of the segment's class and training on such segments can reduce the performance of our model. We deal with such ambiguity by checking the prominence of each class, and drop the window if the difference between the most prominent and second most prominent label is less than 20%. From windows that are not dropped, we extract features using the feature definitions in [4]. For each window, we compute the mean, standard deviation, median, and interquartile range for each of the variables. The feature set contains 48 features, which are normalized before being used for training and testing.

IV. RESULTS
We investigate online classification performance of Support Vector Machine (SVMc) and Gaussian Process (GPc) classifiers in two layers of Madan et al.'s hierarchy, namely on both task-dependent and task-independent behaviours. The performance of our model is evaluated using confusion matrices and by reporting the correct classification rates.

A. Experiment 1: Online classification of interaction patterns
In the first experiment, we investigate how our approach performs in distinguishing task-dependent interaction patterns. Figure 2 shows the confusion matrices. Our results indicate that SVMc reaches a 78.04% accuracy, whereas GPc achieves an accuracy of 80.79% on the online feature set, with 2.75% improvement on the performance of SVMc. In the second experiment we look at our model's performance for distinguishing task independent behaviors, namely harmonious, conflicting and neutral interactions. The SVMc and GPc achieved 83.31% and 83.40% accuracy respectively. The confusion matrices are shown in Figure 3  The following table summarises the previously discussed online classification performances and compares them to offline classification performance.

V. FINDINGS
Our experiments indicate that haptic data can be used for accurate classification of human interaction types and patterns in real-time. We also demonstrate our windowing method as a viable online feature extraction method for timeseries classification to identify interaction states during ongoing physical collaboration. The results indicate that both GPc and SVMc perform well at online classification of interaction states with our feature extraction techinque. GPc achieves a slightly better accuracy but at the cost of much longer training time.

VI. FUTURE WORK
This study acts as a first step to build a proactive robotic partner, which can assist a human, while being aware of the interaction state that the partners are in. Our study also demonstrates that haptic data is extremely useful for physical interaction inference. In future work we intend to design and experiment with more sophisticated haptic features, to see how much useful information can be carried through the haptic channel. We also aim to combine haptics with other modalities, such as vision and muscle activity in order to build a more comprehensive model for interaction and individual user states and intentions. This model could then be used in HRI to define proactive robot behaviours and/or role arbitration as described in [5].