Twist It, Touch It, Push It, Swipe It: Evaluating Secondary Input Devices for Use with an Automotive Touchscreen HMI

Touchscreen Human-Machine Interfaces (HMIs) inherently demand some visual attention. By employing a secondary device, to work in unison with a touchscreen, some of this demand may be alleviated. In a medium-fidelity driving simulator, twenty-four drivers completed four typical in-vehicle tasks, utilising each of four devices -- touchscreen, rotary controller, steering wheel controls and touchpad (counterbalanced). Participants were then able to combine devices during a final 'free-choice' drive. Visual behaviour, driving/task performance and subjective ratings (workload, emotional response, preferences), indicated that in isolation the touchscreen was the most preferred/least demanding to use. In contrast, the touchpad was least preferred/most demanding, whereas the rotary controller and steering wheel controls were largely comparable across most measures. When provided with 'free-choice', the rotary controller and steering wheel controls presented as the most popular candidates, although this was task-dependent. Further work is required to explore these devices in greater depth and during extended periods of testing.


INTRODUCTION
Touchscreens are increasingly becoming the primary display and control interface in cars. Research has shown that, in an automotive context, such devices can be more effective for common tasks (e.g. simple menu selection) and typically attract more positive responses from drivers, compared to other in-vehicle devices [4]. Nevertheless, touchscreens inherently demand some visual attention, due in part to designers' slavish adherence to skeuomorphic interface elements, even in the automotive domain, to reflect previously physical buttons -the absence of genuine tactile cues means that drivers are forced to visually sample the interface to 'find' controls and view task progress. Consequently, common in-car tasks, such as adjusting music volume, could demand too much attention if conducted on a touchscreen-centric infotainment system. This can result in deleterious effects on driving performance and vehicle control, thereby elevating the risk to drivers and other road users [12].
Nevertheless, touchscreen interfaces have captured the attention of automotive designers and appear to be the current, favoured in-vehicle HMI solution, with enticing interactive interfaces often embedded within the centreconsole of vehicles. As a consequence, there has been significant research interest in exploring how to mitigate the visual (and manual) demand elicited by such devices. This has taken a number of guises, including designing interactive on-screen elements to minimise visual demand (e.g. button colour, contrast, size, number [7]), comparing different list-scrolling techniques [11], and identifying simple, intuitive 'short-cut' gestures [3,6]. Other novel techniques, such as expanding touchscreen targets based on drivers' mid-air finger proximity have also been explored [1]. Theoretical modelling has also been used to highlight concerns in proposed designs much earlier in the design cycle [10]. In most case, recommendations are typically made in line with visual/manual distraction guidelines [e.g. 12]. However, such investigations are yet to reveal a viable solution, often serving only to highlight 'bad' designs rather than offering 'good' solutions.
An alternative approach, explored here, is to employ a secondary input device to work in unison with the touchscreen. The aim is to enable drivers to execute the most demanding tasks (or parts of tasks) using a less-Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request  visually demanding secondary device, thereby easing the visual/manual burden of the interaction as a whole, while maintaining the overall appeal and flexibility of the touchscreen. By utilising a secondary device, drivers can be reintroduced with physical anchors, akin to using traditional buttons/switches, thereby allowing the device to be located and operated without visual attention. Moreover, additional haptic cues can be provided during operation, such as 'clicking' through options in a list, thereby further reducing the need for 'eyes-off-road'. Although it is recognised that some touchscreens can also provide haptic cues, e.g. to simulate a button press [13], such 'soft' buttons still require vision to locate them and button activation cues often fail to fully deliver the complex cutaneous sensations associated with traditional, physical buttons [15].
Using a secondary input device with a touchscreen may also provide usability and physical ergonomics benefitssuch devices need not be placed in or near to drivers' normal line of sight, as would be expected and recommended for visually demanding in-car displays [9]. Consequently, such devices can be positioned in more ergonomically and anthropometrically-appropriate locations, thereby reducing fatigue effects during operation, and potentially alleviating handedness problems. The additional provision of a between-seat arm-rest (common in many modern vehicles) is also likely to lead to better operational accuracy compared to situations where devices are located in the upper centre-console, as drivers' arms are supported during operation [14].

Overview of Study
Although there has been significant research effort investigating different input devices/HMIs in cars, and a corpus of literature exists, there has been very little consideration of the combined effects or benefits or using devices together. The study therefore aimed to first understand the impacts of using alternative input devices on driver distraction, and then elicit drivers' preferences for a secondary input device/s that could be used in combination with a touchscreen. This was explored by allowing drivers to use their chosen secondary device/s in combination with a touchscreen during a 'free-choice' drive conducted towards the end of testing.

Participants
Twenty-four people took part in the study: 11 male, 13 female. Mean age, was 32 years, with ages ranging from 21 to 51 years. Twenty of the UK participants were righthanded and four were left-handed. Participants were experienced and active drivers (mean time with UK licence, 12.5 years; range 4-31 years; mean current annual mileage, 7495 miles). All participants were self-selecting volunteers who responded to advertisements placed on-line and around the University of Nottingham campus, and were reimbursed with £10 (GBP) of vouchers as compensation for their time.

Apparatus, Design and Procedure
The study took place in a medium-fidelity, fixed-based driving simulator at the University of Nottingham ( Figure  1). The simulator comprised the front half of a right-hand drive Honda Civic car positioned within a curved screen affording a 270° viewing angle. A bespoke driving scenario was created using STISIM (v2) software, to resemble a standard 3-lane UK motorway, and projected onto the screen using three overhead projectors. Participants were required to follow a lead vehicle ("as if going to a shared destination"), which travelled at a constant speed of 65mph, and wore SensoMotoric Instruments (SMI) eye-tracking glasses to record their visual behaviour.
Participants were asked to complete secondary tasks using each of four input devices that have commonly been considered in a driving context:

Rotary Controller (RC). Located between driver and
passenger seats, the rotary controller provided rotary input in addition to 4-way joystick and button presses.

Steering Wheel Controls (SWC). A Sony Vaio
Bluetooth laser mouse (model VGP-BMS80) was installed within the left spindle of the steering wheel. The device allowed 4-way directional control in addition to optical swipe and button press input. 3. Touchpad (TP). The touchpad was located between the driver and passenger seats, and provided 4-way swipe, button press and character/gesture recognition, using fingertip input. 4. Touchscreen (TS). An HP EliteBook 2740p tablet computer was located within the centre console.
Devices were positioned in typical locations within the car ( Figure 2) and were all designed to be used in conjunction with the touchscreen, which acted as a display when the touchpad, rotary controller and steering wheel controller were being used, and also as a control interface during a 'touchscreen-only condition. An arm rest was provided to support participants' arm movements when using the touchpad and rotary controller.
Prior to testing, participants received full training and guidance for each device, and for all tasks, until they were deemed to be competent. During testing, participants completed all four tasks using each device while driving, providing subjective feedback between devices/drives. Device and task order were counterbalanced to avoid learning effects. After testing all four devices, participants undertook a fifth, 'free-choice' drive, in which they were able to choose any device (or combination of devices) to complete each of the four tasks.

Tasks
The four tasks under investigation were representative of in-vehicle driving-related activities and were enabled using a bespoke, test interface (Figures 3-6):    Input/interaction techniques naturally differed between devices and participants were required to complete tasks using the native input techniques for each device. This ensured that participants were able to experience the full functionality of all devices, thereby allowing more robust conclusions to be drawn, especially regarding preferences and relative performance. For example, to move through a list, participants were required to swipe the touchscreen and touchpad, rotate the rotary controller, and press the steering wheel controller. To enter text, participants used an onscreen alphanumeric keyboard to select characters -either by touching or stepping through the menu using the rotary and steering wheel controls -but were required to 'write' each number individually on the touchpad using their left index finger.     The following measures were captured and reported: • Secondary Task Time -recorded from the touchscreen.
• Visual Behaviour -total glance time (TGT), mean glance duration (MGD) and number of glances (NG). Glances were classified as 'on' or 'off-road'. Off-road glances reflect visual attention directed at both the device for control and the touchscreen for feedback. • Driving Performance -speed, lane keeping, headway, captured from the STISIM simulation computer. • Workload -NASA-TLX mean workload rating [8].
• Emotional Response -ratings of 'dominance', 'arousal' and 'control', obtained from the Self-Assessment Manikin (SAM) questionnaire [2]. • Subjective Ratings and Preferences -ease of use, interferes with driving, device preferences/liking. • Actual and Perceived Use of devices during free-choice.

RESULTS
Unless otherwise stated, 2-way ANOVAs were conducted to examine the effects of device and task on each measure, with post hoc Tukey corrections for multiple comparisons. All figures show standard errors bars, where appropriate.

Secondary Task Time
There were main effects of device (F (3,357) = 40.99 p < 0.0005) and task (F (3,357) = 4.52 p < 0.0005) on secondary task time. Planned comparisons show that participants took significantly longer completing tasks using the touchpad and were quickest when utilising the touchscreen (Figure 7).

Number of Glances
There were main effects of device (F (3,240) = 33.98 p < 0.0005) and task (F (3,240) = 3.51 p = 0.02) on number of glances. Participants took significantly more glances when undertaking tasks using the touchpad. The fewest glances were observed with the touchscreen (p < .0005) (Figure 8). There were also main effects of device (F (3,240) = 5.21 p = 0.002) and task (F (3,240) = 2.91 p = 0.04) on number of 'long' glances (over 2.0 seconds), with significantly more long glances associated with the touchpad compared to all other devices (p max = 0.047) (Figure 9).

Total Glance Time
There were main effects of device (F (3,240) = 22.95 p < 0.0005) and task (F (3,240) = 4.01 p = 0.01) on total glance time. Total glance time was significantly longer when undertaking tasks using the touchpad. Shortest glance time was associated with the touchscreen (p = .009) ( Figure 10).

Mean Glance Duration
There were main effects of device (F (3,240) = 2.78 p = 0.04) and task (F (3,240) = 4.88 p = 0.003) on mean glance duration. Planned comparisons show that the mean glance duration was longer for the touchscreen compared to the rotary controller (p = 0.045) (Figure 11).

Speed
There was a main effect of device on standard deviation of speed (F (3,357) = 19.72 p < 0.0005), with greater variability in speed evident when participants were using the TP, compared to the SWC. The standard deviation of speed associated with the TS was lower than all other devices ( Figure 12).

Lane Position
There was a main effect of device on standard deviation of lane position (F (3,357) = 7.97 p < 0.0005), with more variability evident when using the TP, compared to SWC and TS ( Figure 13).

Headway
There was a main effect of device on standard deviation of headway (F (3,357) = 10.68 p < 0.0005). Using the TP resulted in the greatest variability in headway, compared to all other devices. Headway variability associated with the TS was also significantly lower than RC ( Figure 14).

Workload: NASA-TLX
There was a main effect of device on mean workload (F (3,92) = 7.95 p < 0.0005). Mean workload associated with the touchpad was significantly higher compared to all other devices (p max = .007) ( Figure 15).

Self-Assessment Manikin (SAM)
There was a main effect of device for SAM ratings of 'pleasure' (F (3,92) = 20.59 p < 0.0005) and 'dominance' (F (3,92) = 10.33 p < 0.0005). The touchscreen was deemed to be most pleasurable to use (p max = .016), and the touchpad, least pleasurable (p max < .0005), compared to other devices. Additionally, participants felt more in control when using the touchscreen compared to the touchpad (p < .0005), but least in control when using the touchpad (p max = .001). Ratings of 'arousal' were comparable between all devices ( Figure 16).

Ease of Use While Driving
There was a main effect of device on participants' rating of ease of use (F (3,91) = 19.55 p < 0.0005), with participants rating the rotary controller, steering wheel controls and touchscreen as significantly easier to use while driving than the touchpad (Figure 17).

Interferes with Driving Task
All devices were deemed to interfere equally with the driving task (F (3,91) = 2.20 p = 0.09).

Overall Liking
There was a main effect of device on participants 'liking' of devices (F (3,91) = 23.02 p < 0.0005), with participants indicating that they liked the touchpad least (all p < 0.0005). The touchscreen was liked more than the steering wheel controls (p = 0.035) (Figure 18).

Device Preferences
Participants were asked to rate/rank their preferred device for each of the four tasks, by placing a marker for each device on continuous linear scales. Positions were measured and interpreted as a 0-100 interval scale, where a rating of '100' indicated 'best/most preferred'. There was a main effect associated with participants' most and least preferred device for all tasks: menu navigation (F (3,92) = 35.29 p < 0.0005); list selection (F (3,92) = 10.33 p < 0.0005); text entry (F (3,92) = 12.03 p < 0.0005) and map manipulation (F (3,88) = 21.03 p < 0.0005) (Figure 19). Participants overwhelming preferred the touchscreen, to all other devices, for the menu navigation task, text entry and map manipulation tasks (p max = .007). For the list selection task, there were also high preferences for the rotary controller: ratings for both rotary controller and touchscreen were significantly higher for these tasks than those for steering wheel controls and touchpad (p max = 0.01). The touchpad was least preferred and received the lowest ratings compared to other devices for the menu navigation and list selection tasks (p max < .0005). For text entry and map manipulation, the touchpad received similar ratings to the rotary controller and steering wheel controls.
Participants were also asked to indicate the device (rotary controller, touchpad or steering controls), that they would most, and least, prefer to use in conjunction with the touchscreen for all tasks. The rotary controller was selected by more than 54% of participants as their 'most preferred' device. In contrast, the touchpad was deemed to be 'least preferred' by over 58% of participants.   During the final drive, participants repeated all four tasks but were given the opportunity to use whichever device or combination of devices they desired. Participants were then asked to record the amount of time they believed they spent using each device for each task. Results can be seen in Figure 20. These were then compared with actual use, obtained from analysis of the video recordings (Figure 21).

DISCUSSION
The study compared four different input devices and considered which device (rotary controller, touchpad or steering wheel controls) was most appropriate to use in conjunction with an in-car touchscreen HMI. Objective measures (secondary task time, visual behaviour, driving performance) consistently revealed shortcomings associated with the touchpad -secondary tasks took the longest time to complete, it invited the most glances (many of which were longer than 2.0 seconds -a common predictor of heightened risk [12]), and TGT was significantly longer than when using other devices. Using the touchpad also had the greatest impact on driving performance measures (indicated by variability in speed, lateral lane position and headway).
Overall, participants did not like the touchpad -it was associated with higher perceived workload and was identified as least pleasurable to use ("frustrating", "slow to use"), although some positive comments were received ("liked the 'concept'"; "character recognition generally very good"). Participants also felt least in control when using the device, found it more difficult to use while driving, and believed that it interfered more with the driving task than other devices. The touchpad was also highlighted as least preferred for menu navigation and list selection tasks, although it was more favourably considered (equally as popular as the rotary controller and steering wheel controls) for text entry and map manipulation. It was the least preferred device to use in conjunction with the touchscreen, overall.
As a whole, these results appear to comprehensively preclude the touchpad as a viable candidate to support drivers during the accomplishment of secondary tasks while driving, although it is noted that there were some technical problems in the implementation of the device during the study that may have influenced results -this was particularly noticeable for tasks requiring 'dragging' (moving through lists, manipulating the map). In general, character recognition (used for text entry and the initial stages of the list selection tasks) was very good on the touchpad. However, some right-hand dominant drivers struggled to form certain more complex characters (e.g. the number '8'), as they were required to use their nondominant left hand. This is likely to have influenced both objective performance and subjective opinions.
In terms of secondary task performance, visual behaviour and the effect on driving, the rotary controller and steering wheel controls were largely comparable. Subjectively, these devices were also equally popular, although the rotary controller was identified as the preferred device overall to use in conjunction with the touchscreen (taking all devices and tasks into consideration). However, it was evident that using the touchscreen on its own to complete tasks was quicker and required fewer glances than the other devices, in most situations. Nevertheless, MGD was notably the longest when using the touchscreen, though still significantly shorter than the 2.0-second recommended 'safety' threshold [12] at 1.3 seconds. This suggests that participants were more comfortable extending glances to the touchscreen, possibly due to the location of the device, close to their normal line of sight. It may also be a reflection of the fact that during the touchscreen-only condition, the device provided both control and feedback functionalities. Even so, the lowest variability in headway and lane position were associated with the touchscreen, suggesting that drivers were able to maintain primary task performance.
The touchscreen was also identified as the most pleasurable device to use, and was identified as "familiar, accurate and fast". Participants felt more 'in control' using the touchscreen, and generally liked the device more, specifically identifying it as their preferred device for menu navigation, text entry and map manipulation tasks; it was also equally as popular as the rotary controller for list selection. The popularity of such devices in everyday society means that touchscreens are increasingly familiar and 'intuitive' to use. Nevertheless, the rotary controller and steering wheel controls were also popular ("very good for moving through list" (RC); "comfortable to use" (RC); "expectations high" (SWC)).
Given the results, one may conclude that using a touchscreen alone is perfectly acceptable. Indeed, discounting the touchpad, the visual demand associated with both the rotary controller and steering controls appears to be no better than the touchscreen. Nevertheless, using a touchscreen is always likely to demand some visual attention, and their typical location invites anthropometrical issues (e.g. arm instability), leading to potential errors and fatigue. In contrast, physical devices, such as rotary controllers and steering wheel controls, permit eyes-free use, and their expected locations may afford arm support.
Furthermore, during the study, people relied on the touchscreen for progress/feedback, and this also demanded visual attention. However, one would expect this demand to reduce over time, as drivers become familiar with the interface, tasks and the operation/impact of the secondary device control actions. Additionally, for part of the study, single device operation was enforced, to allow direct comparisons to be made. In reality, there are aspects of each task that may be better suited to specific devices. This is likely to reduce the overall visual burden and improve the efficiency of all tasks. For example, during the final 'free- It is also noteworthy that participants' perceptions of the devices they used during the 'free-choice' drive were very different to their actual selections. For example, nobody stated that they used the steering wheel controls for list selection and yet this was actually employed for 26.4% of the time, on average. Similarly, participants indicated that the touchpad was selected for map manipulation and menu navigation (14.4% and 5.4% of the time, respectively), and yet neither device was actually used for these tasks during the 'free-choice' drive. It is worth noting that there were a few situations (4 noted) where participants switched to a different device shortly after starting a task -it is unclear whether this was because they were unable to complete the task as initially anticipated (e.g. due to technical or usability problems) or simply because they changed their mind. Nevertheless, this may have influenced their perceptions of use.

CONCLUSION
Overall, there was general consensus that the touchscreen on its own was most preferred and least demanding to use for all tasks. In contrast, the touchpad was least preferred and most demanding; the rotary controller and steering wheel controls were largely comparable across most measures. There is also good evidence that a combination of devices was employed by participants when provided with 'free choice' -this is likely to reduce the visual burden, compared to touchscreen-only operation, although further work should explore the candidate devices in greater depth and over extended periods of testing. Based on these results, both the steering wheel controls and rotary controller appear to be good candidates to for future investigations. Future work should continue to consider secondary devices to be used in conjunction with an invehicle touchscreen to reduce demand and improve performance, but should also consider practicalities during real-world use (e.g. the effect of vibration on performance and accuracy).