Exploring the Use of Mid-Air Ultrasonic Feedback to Enhance Automotive User Interfaces

Employing a 2x2 within-subjects design, forty-eight experienced drivers (28 male, 20 female) undertook repeated button selection and 'slider-bar' manipulation tasks, to compare a traditional touchscreen with a virtual mid-air gesture interface in a driving simulator. Both interfaces were tested with and without haptic feedback generated using ultrasound. Results show that combining gestures with mid-air haptic feedback was particularly promising, reducing the number of long glances and mean off-road glance time associated with the in-vehicle tasks. For slider-bar tasks in particular, gestures-with-haptics was also associated with the shortest interaction times, highest number of correct responses and least 'overshoots', and was favoured by participants. In contrast, for button-selection tasks, the touchscreen was most popular, enabling the highest accuracy and quickest responses, particularly when combined with haptic feedback to guide interactions, although this also increased visual demand. The study shows clear potential for gestures with mid-air ultrasonic haptic feedback in the automotive domain.


INTRODUCTION
Touchscreens appear to be the current de facto automotive user interface, with some research supporting their application for certain tasks in vehicles (e.g. simple menu selection), compared to other physical devices [1]. However, touchscreens inherently demand visual attention. This is due in part to designers' slavish adherence to skeuomorphic elements to reflect previously physical buttons, and is further exaggerated by the smooth surface and notable absence of genuine tactile cues. Clearly, any visual attention directed towards a touchscreen means that a driver may not be adequately attending to the road situation, and this can detriment driving performance and vehicle control, thereby elevating the risk to drivers and other road users. This is a particular concern when in-vehicle glances extend beyond 2.0-seconds [2].
Although research aiming to minimise the visual demand associated with touchscreens in vehicles has been prolific (e.g. [3]), it is important to understand that the number and duration of glances directed towards an in-vehicle device are in fact defined by two elements: the inherent and underlying visual demand associated with undertaking a specific task or interaction, and the driver's personal motivation or desire to engage visually with the interface. The former is defined by the characteristics of the interaction and interface (e.g. number, layout and size of targets on a touchscreen) [4] and can therefore be reduced through artful designs and novel interaction techniques. In contrast, the latter is motivated by what the driver deems to be 'acceptable' based on the current driving situation and their own attitudes and opinions [5]. This has even led to the classification of drivers in terms of their visual behaviour, resulting, for example, in so-called 'long glancers', i.e. drivers who are more inclined to take their eyes of the road for periods greater than 2.0-seconds [5].
Whereas attempts to minimise the visual demand of touchsurfaces may go some way to disincline or dissuade such drivers from looking at the device, proposed solutions are fundamentally limited by the individual behaviour and motivations of drivers. Thus, while a visual stimulus exists, drivers may still choose to direct their attention to the screen, irrespective of whether this is actually required to complete the task or interaction presented to them. Consequently, more radical approaches that have the power to eliminate the need (or temptation) for vision completely need to be explored. A number of novel technologies and innovative operating concepts have thus been proposed.
gestures can lack accuracy and/or the capacity for complex or multiple interactions, and present the potential for inadvertent operation. Even so, developments in technology, combined with lucrative commercialisation opportunities, have pioneered devices such as Microsoft Kinect, and means that gesture control is now ubiquitous within some application domains, such as gaming. However, gestural interfaces still often require, or actively invite, visual attention (for example, to view a visualisation of the user's hand/gesture, or confirm menu selections), and may also require the user to learn several different gestures -particularly where a number of potential options exist.
In an automotive domain, the use of gestures as an input modality has also been considered [7,8,9]. Although research generally indicates that gestures are rated more highly (more 'enjoyable') than conventional controls, task success rate and 'time to goal' are typically highly variable (particularly for more complex gestures or tasks), with the highest successes and shortest task times associated with 'simple' and 'natural' gestures. Authors also warn of the additional time required to learn more complex manoeuvres, and potential cultural differences associated with gestures [7]. Nevertheless, commercial in-vehicle applications do now exist (e.g. [10]), although these are often designed to be used in conjunction with an existing touchscreen display or interface.

Haptics
An alternative approach to reduce the visual demand of existing touch-surface interfaces is to augment routine interactions with additional information, such as sensations that can be 'felt'. Employing haptics naturally reduces the need for vision as users are not required to visually confirm selection or activation, but this has traditionally still required physical contact with the interface. For example, haptic sensations have been added by varying the friction coefficient of the touch-surface by vibrating it with ultrasound [11], or physically changing its shape [12], and this has also been investigated in a driving context [13].
Attempts to enable haptic feedback above a surface (i.e. without the need for physical contact with the surface) have also been explored but initially required users to wear an additional device to detect or guide their hands or fingers, for example, based on electromagnetic actuation [14] or vibrators [15]. While these allow users to move their hands freely without physically contacting the surface, the requirement to wear an additional device limits the potential for spontaneous interaction and can sacrifice simplicity and accessibility [16]. Moreover, in a divided attention context, such as driving, wearing an additional secondary device may interfere with primary task execution.
In contrast, Hoshi et al. [17] proposed the use of ultrasound to stimulate receptors in the human body. The approach, further developed and brought to market by Carter et al. [16] (who coined the term 'Ultrahaptics', and first recognised the potential in the automotive domain), uses the principle of acoustic radiation force, i.e. the force generated when ultrasound is reflected. Therefore, no physical contact or additional wearable device is required. Instead, ultrasound is focused directly onto the surface of the skin, where it induces vibrations in the skin tissue. The displacement triggers mechanoreceptors within the skin generating a haptic sensation [18]. By focussing the acoustic radiation force and modulating this at 200Hz (the frequency at which vibrations are perceptible by human skin mechano-receptors), the illusion of a virtual 'target' in mid-air can be created. Moreover, by synchronising the phase delays of multiple transducers, different focal points can be generated which are perceivable as different mid-air targets, or interface elements [16]. The ultrasound used has a frequency of 40kHz and thus the smallest interface element has physical diameter equal to the wavelength of 8mm Combining mid-air haptic sensations with simple, intuitive gestures therefore has the potential to eliminate the need for vision completely. For example, discrete virtual buttons could be presented in three-dimensional space to replicate the familiar layout of a traditional touchscreen interface (i.e. as an ordered array of buttons). Once identified, selections could be made by physically moving the hand downwards (to emulate pressing a button). The approach has the added advantage that users are not required to remember the semantic meaning of different gestures, or locations of interface elements, but rather use their sense of touch to locate the 'button' based on targeted haptic feedback, and then use a simple, intuitive gesture, such as a 'press' to activate it. As such, existing 'mental models' of interface layouts (e.g. a 2x3 structured array of buttons) can be logically applied. Additionally, the virtual array of targets can be placed anywhere in three-dimensional space and is therefore not bound to an existing surface or infrastructure.
In an automotive domain, this offers potential benefits in anthropometrics and physical ergonomics, as well as space savings in vehicles.

Overview of Study
The aim of the current study was to explore the use of gestures augmented with mid-air haptic feedback (created using ultrasound) in a driving context. Therefore, in the interests of scientific rigour (i.e. to ensure a fair and unbiased comparison in a 2x2 experimental design, with independent variables of touch/gesture and with/without haptic feedback), a touchscreen remained present throughout, acting as the interface during 'touch' conditions and providing an abstracted view of selections during 'gesture' conditions, even though a visual display would not strictly be required for the latter. Consequently, it is not expected that visual demand would be eliminated completely when using gestures-and-haptics, although this is a perfectly realistic goal in future investigations and evaluations.

Apparatus
The study took place in a medium-fidelity, fixed-based driving simulator at the University of Nottingham ( Figure  1). The simulator comprises a right-hand drive Audi TT car positioned within a curved screen, affording a 270 degrees forward and side contiguous image of the driving scene via three overhead HD projectors, together with rear and side mirror displays. A Thrustmaster 500RS force feedback steering wheel and pedal set are integrated faithfully with the existing Audi primary controls, with a dashboard created using a bespoke application and re-presented on a 7-inch LCD screen, replacing the original Audi instrument cluster. The simulated driving environment was created using STISIM (v3) software, and comprised a three-lane UK motorway with both sides of the carriageway populated by moderate levels of traffic, and authentic road signage and geo-typical roadside terrain.
Mid-air haptic sensations were created using the Ultrahaptics touch development kit (TDK) 1 , installed in the centre of the car (between driver and passenger seats) ( Figure 2), as might be expected for such an interface. This location naturally lends itself to comfortable 'open palm' interactions, and eliminates potential safety concerns associated with using directional ultrasonic waves. The TDK employs a 14x14 ultrasonic transducer array board to create three-dimensional mid-air sensations that could be best described as gentle, pressurised airflow on the palm of the hand. A Leap Motion 2 camera is used to detect and track the driver's hand movements and localise sensations.
Interaction techniques and textures were developed in collaboration with Ultrahaptics Ltd. using the Ultrahaptics software development kit (SDK) integrated with Unity, to replicate multiple target arrays ('buttons') and a graduated 'slider bar'. For buttons, active regions utilised four focal points to create perceivable 'button' shapes that were fixed in three-dimensional space (but not bound to a single plane in x-y space). Edges and the space between buttons were defined by the absence of haptic feedback, with the size and layout of button arrays directly corresponding with the onscreen representation. For slider-bar tasks, the centre of the slider-bar was determined by the participant's first openhand gesture (i.e. not fixed in three-dimensional space). Thereafter, the slider-bar interface allowed approximately 20cm movement in either direction.  The 'car following' paradigm was adopted as the primary driving task [19]. At the start of the scenario, a yellow car was presented ahead of the participant's vehicle on the motorway. This began moving when the participant started driving, and travelled at a variable speed (between 65 and 75mph). Participants were instructed to follow the lead car, which remained in lane one, at a distance that they deemed to be 'safe and appropriate'. While following the lead vehicle ('the primary task'), participants were asked to interact with the in-vehicle interface ('the secondary task') using four different techniques in a 2x2 experimental design (touch versus gesture and with/without haptic feedback). As stated, the touchscreen remained present during all conditions, providing visual feedback during the 'gesture' conditions.
Each participant was provided with training and familiarisation using the touchscreen and gestures (with and without haptic feedback). This occurred firstly whilst stationary (i.e. seated in the car) and secondly, while driving. For each technique, the participant was required to demonstrate three consecutive successful interactions (i.e. using the correct behaviour and selecting the correct target without any false activations), before they were deemed to be competent. Each participant was subsequently asked to undertake four experimental drives. During each drive, participants were presented with a different interaction technique, resulting in four drives, or conditions, with the order of exposure counterbalanced between participants:

Touch No Haptics (TN):
Tasks were completed using a conventional touchscreen with no haptic feedback.

Touch with Haptics (TH):
Tasks were completed using the touchscreen enhanced with ultrasonic haptic feedback aiming to guide the participant's hand towards the touchscreen (i.e. haptic feedback was provided when their hand was in close proximity to the screen).

Gesture No Haptics (GN):
Tasks were completed using simple gestures (identified using the Leap Motion sensor) but without haptic feedback. 4. Gesture with Haptics (GH): Tasks were completed using the same gestures enhanced by haptic sensations.
Bespoke interfaces were created comprising monochromatic, interaction elements of ersatz in-vehicle tasks, i.e. discrete 'buttons' and a continuous 'slider bar' (Figure). While conventional interfaces would likely include more complex interaction elements (e.g. multiple buttons in elaborate configurations and colour schemes), the intention in using an abstracted interface was to explore the behaviour associated with isolated, constituent elements and thus avoid potential confounds associated with more intricate designs as well as potential differences in the semantic interpretation or actuation of specific tasks. In addition, the chosen techniques (button selection and sliderbar manipulation) are highly representative of current automotive touch-surface interface elements, and of particular interest given that they pose different interaction characteristics (i.e. discrete versus continuous).
For button selection tasks, participants were provided with either a 2, 3 or 4-item structured menu (with targets numbered consecutively) (e.g. Figure 3), and were asked to select a specific target item (either by touching the screen or using a gesture) with a pre-recorded voice message. For example, "On this three-item menu, select two". To achieve this for the 'gesture' conditions (GN and GH), drivers were required to locate the correct button (either by using visual feedback, or identifying the relevant haptic sensation) and make a simple downward movement of their open hand to simulate a button press. For slider bar tasks, the pointer was initially placed in the centre of the slider bar, and participants were asked to increase or decrease the value by a specified amount, up to five increments in either direction (for example, "Please increase the value by three"), by dragging the pointer on the touchscreen or using mid-air gestures. For gesture conditions, participants were required to initially 'select' the pointer by making an open-palm gesture. Participants were then required to move their open hand right or left to increase or decrease the value, and then 'grab' (by making a

Measures and Analysis Approach
To record visual behaviour, participants wore SMI eyetracking glasses (ETG) (visible in Figure 2), with gaze data analysed using semantic gaze mapping. Off-road ('invehicle') glances were subsequently defined from the moment the driver's gaze started to move towards the in-car display, to the time it returned to the road scene (i.e. including the transition time from and back-to the road). Thus, a single in-car glance could comprise several fixations on the in-vehicle display.
Secondary task performance was determined through measures of accuracy (percentage of correct button/sliderbar selection, and cumulative slider-bar 'overshoots') and task-time. For slider-bar tasks, the task-time comprised the 'reaction time', i.e. the time from the delivery of the task instruction to the start of the interaction (when the Leap Motion sensor detected the hand and initiated the haptic sensations, or the participant made contact with the touch surface), and 'interaction time' reflecting the time that each participant took to manipulate the interface and make their selection (using either the appropriate mid-air gesture or touch). For button selection tasks, only total task time was recorded (given that for button selections using the touch surface, there was no discernible 'interaction' time).
In addition, driving performance data were captured from STISIM. These were used to calculate standard deviations

Visual Behaviour
Based on current guidelines [20], visual behaviour is presented as the number of off-road glances (NOG) (Figure  4), the total off-road glance time (TGT) (Figure 5), and mean off-road glance duration (MGD) (Figure 6). The number of off-road glances longer than 2.0-seconds are also presented. For clarity, these relate to each condition (or drive) (i.e. all 18 button and 20 slider-bar tasks, respectively) ( Figure 7). 'Off-road' (or 'in-vehicle') glances comprise all visual behaviour directed towards the interface (i.e. the screen and/or transducer array).

Number of Off-Road Glances (NOG)
For button-selection tasks, there were no significant differences identified in NOG for either Touch or Haptics ( Figure 4). However, there were significantly fewer glances over 2.0-seconds ('long glances') associated with the touchscreen (F(1,126) = 15.3, p < .001) (Figure 7). There was also a significant interaction for Touch*Haptics (F(1,126) = 4.22, p = .042), indicating that adding haptics to gestures decreased the number of long glances, whereas the number of long glances associated with the touchscreen increased when haptics were added. A similar trend can be observed for the number of glances overall.
For slider-bar tasks, there was a significant difference in NOG associated with Touch (F(1,125) = 4.0, p = .047), indicating fewer off-road glances associated with gestures. Adding gestures had no significant effect on the number of off-road glances overall, but tended to reduce NOG associated with the gesture interface. There were no significant differences in the number of long glances associated with slider-bar tasks, although there was a trend for fewer when haptics was added.

Total Off-Road Glance Time
For button-selection tasks, total off-road glance time was significantly lower for Touch compared to Gesture (F(1,126) = 4.93, p = .028) ( Figure 5). Nevertheless, adding haptics reduced TGT for the gesture interface, but extended TGT for the touchscreen.
For slider-bar tasks, there were no significant differences identified for either Touch or Haptics, although adding haptics to the gestures tended to reduce total off-road glance time.

Mean Off-Road Glance Duration
For button-selection tasks, the mean off-road glance duration was significantly shorter for Touch compared to   Gesture (F(1,126) = 5.69, p = .019) ( Figure 6). There was also a significant interaction for Touch*Haptics (F(1,126) = 6.97, p = .009), indicating that adding haptics to gestures decreased the mean off-road glance duration, whereas mean off-road glance duration increased when haptics were added to the touchscreen.
For slider-bar tasks, significant differences were found for Haptics (F(1,125) = 4.40, p = .038) (Figure 6), showing that the provision of ultrasound haptic feedback reduced the mean glance duration for both the touchscreen and gestures. No significant differences in MGD were found associated with Touch for slider-bar tasks.

Accuracy
For button-selection tasks, there was a significant difference between Touch and Gesture for accuracy (percentage of selections made correctly) (F(1,47)=13.95, p = .001), with the greatest success achieved when using the touchscreen. In addition, haptics provided benefit in terms of improved accuracy for the Touch condition but not for Gesture (F(1,47)=2164.9, p < .001).
There was also significant differences between Touch and Gesture for slider-bar 'overshoots' (F(1,46)=202.8, p < .001) (Figure 8), showing that fewer errors of this type were made when using gestures (both with and without haptics). In addition, when interactions were enhanced with haptics, benefits were more evident during the Touch condition than with Gesture (F(1,46)=1848.0, p < .001). There were no significant differences identified for percentage of correct selections for the slider-bar tasks between Touch and Gesture, although again, haptics tended to benefit the Touch condition more than Gesture for this measure.

Task Time
Total task-time comprises both time to respond to the task instruction ('reaction time', i.e. moving the hand to the active zone of the Leap Motion sensor, or making contact with the touch-surface), and the time to undertake the task itself ('interaction time', i.e. manipulating the interface and making a selection). For button-selection tasks, there was a significant difference between Touch and Gesture (F(1,46)=187.3, p < .001) (Figure 9), with a two-way ANOVA showing that Touch was significantly quicker than Gesture for total task-time, both with and without haptic feedback. It was not feasible to split total task-time for button-selections using the touch surface as there was no discernible 'interaction' time, and therefore no comparison could be made with Gesture for this measure.
When task-time was broken down into 'reaction time' and 'interaction time' for slider-bar tasks, it was evident that the time taken to undertake the task (i.e. the interaction time) was significantly shorter when using gestures (F(1,46)=43.15, p < .001) (Figure 10), whereas reaction time (i.e. the time to respond to the task instruction) was quicker for Touch compared to Gesture (F(1,46)=232.0, p < .001) (Figure 11).

Driving Performance
There was a significant difference for SDLP (F(1,45)=407.0, p < .001), with pairwise comparisons suggesting that SDLP was lower during Touch-only compared to Gesture-only (p = .039). However, when this was corrected for multiple comparisons, differences were no longer significant (Figure 12Figure). There were no significant differences revealed in SDHW between conditions.

Preferences
Participants were asked to rank the four conditions in order of preference. Pairwise ranking was used to systematically compare each condition with each other. Table 1 and Table  2 show pairwise scores and rankings for button-selection tasks and slider-bar tasks, respectively. For button selection tasks, participants tended to prefer the touchscreen, whereas gestures-with-haptics was preferred for slider-bar tasks.
These ratings were supported by comments made by participants. For example, regarding button-selection tasks: "The touchscreen was good because you could see what you pressed." Even so, some participants recognised the limitations associated with the touchscreen: "The touchscreen was difficult to actually get…and you do have to actually look at the number…it meant I had to look at the screen for longer, which was more distracting." Support for gestures-with-haptics was evident for both button-selection and slider-bar tasks: "You don't even need to look, you can kind of feel. So the haptic feedback helps with that"; "It gave me more confidence that I choose the correct number"; "The haptics slider was probably the easiest, because as you moved it gave you a click-click-click feedback." It was also clear from participants' comments that gestures alone (i.e. without haptic feedback) were more challenging to use: "Gesture control without haptics was difficult because you couldn't tell what you were activating."

DISCUSSION
By employing ultrasound to deliver discrete haptic sensations that can be felt on the palm of the hand, the illusion of an array of virtual buttons, or other interaction elements such as a slider-bar, can be created in mid-air and subsequently actuated using gestures [16]. This has the potential to remove the need for vision when interacting with an interface, and is therefore of particular relevance in an automotive context, where a driver's visual attention may already be consumed by the primary driving task. The aim of the current study was to investigate the use of midair ultrasonic haptic feedback to enhance interactions in this context, with a particular focus on understanding the impact on drivers' visual behaviour. It is worth highlighting again that as part of the experimental design, the touchscreen remained present during the 'gesture' conditions. In the absence of haptic feedback, this was arguably necessary to support the use of the gesture interface. However, when gestures were enhanced by mid-air ultrasonic feedback, visual feedback was not strictly required, and therefore the presence of the touchscreen may have inadvertently attracted additional visual attention in this situation. Thus, the study was unlikely to reveal the full potential of 'visionfree' interaction that gestures-with-haptics could enable. Nevertheless, there were some interesting visual behaviours revealed.
For example, when making button selections, the touchscreen attracted fewer long glances (>2.0-seconds) than the gesture interface. In addition, the total off-road glance time and mean glance duration was shorter when using the touchscreen for button selections. However, the addition of mid-air haptic feedback increased the number of long glances made to the touchscreen (as well as TGT and MGD), whilst actually reducing the visual demand of the gesture interface, evidenced by fewer long glances, and a significant reduction in TGT and MGD. In contrast, for slider-bar tasks, the gesture interface attracted the fewest number of glances, but again, the addition of haptics tended to reduce the number of glances (and in particular, the number of glances over 2.0-seconds), as well as the total off-road glance time and mean glance duration.
Therefore, although there is some evidence (based on the visual performance measures) to suggest that the touchscreen alone performed better than gestures for button selections, it was not possible to reduce the visual demand further by providing haptics. Instead, providing haptic feedback to the touchscreen actually increased visual demand. In contrast, the provision of ultrasound haptic feedback to gestures significantly reduced the visual demand associated with this interface, and for all tasks (i.e. button-selections and slider-bar manipulation). This is an important finding as interfaces are becoming less reliant on single button presses (which can be limited in their scope), and increasing incorporating novel interaction elements and techniques, that may be better serviced using gestures. Moreover, results suggest that current concerns regarding the potential additional demands associated with such interfaces may be alleviated through the careful provision of ultrasound haptic feedback.
There were, however, notable benefits in terms of accuracy (for the slider-bar task), when the touchscreen was enhanced by haptic feedback. Nevertheless, the best performance overall (in terms of the percentage of correct responses and minimising target 'overshoots') was associated with gestures and haptics. Conversely, utilising gestures for button selections appeared to extend total tasktime. Examined more closely (i.e. by segregating 'reaction time' and 'interaction time' for the slider-bar tasks), it was clear that the additional time was associated with the drivers' 'reaction', i.e. the time taken from the delivery of the task instruction to the start of their interaction. For the gesture/haptics condition this included the time that the For gestures not bound in physical space (i.e. the slider-bar tasks), a key factor is the successful accomplishment of interactions was participants' initial hand-position. Primarily, this was to ensure that the Leap Motion system could detect the hand, but it was also important to ensure that there was adequate space to complete manoeuvres. For example, poor initial hand placement could hinder sliderbar value increments due to the physical presence of the steering wheel (i.e. if interactions were started too close to the wheel), whereas decrementing the slider-bar could be difficult due to limitations in participants' reach, if their initial hand placement was too far away. Thus, participants were encouraged to carefully locate their hand in threedimensional space before commencing each interaction, and this would likely have taken additional time and effort. This self-imposed formality would be expected to reduce as drivers' familiarity with the technology increases.
In addition, even when located 'correctly', the Leap Motion camera was required to detect the driver's hand and initiate haptic sensations. Consequently, there was also an inherent hardware/software latency (in addition to the time for the driver to physically move their hand into position), and this is likely to reduce in future implementations as the flexibility and capability of the technology improves.
There were also some differences apparent in driving performance measures (in particular, SDLP), with better lateral vehicle control evident when participants used the touch-surface, compared to gestures without haptics. However, when the gestures were augmented with ultrasound sensations, vehicle control was comparable, suggesting that while gestures on their own may detriment driving performance, the additional provision of mid-air haptic feedback could negate deleterious effects on driving performance (i.e. making them comparable to using the touchscreen on its own). However, these effects were small and therefore further longer-term driving studies are recommended to explore this further. Support for the gestures and haptics is also evident in the preference ratings, with 'gesture with haptics' identified as the most popular (by far) for the slider-bar task, and the second most popular for selecting buttons. The fact that the touch-surface was most popular for button selections (and achieved the shortest task times for these) is unsurprising given that touchscreens are now common in many contexts, and the interaction itself (i.e. touching the screen) remains perceptively quicker and easier than locating and activating a virtual mid-air button (which was a novel experience to many of our participants). However, it is worth noting that the Audi TT simulator utilised during the study is a compact vehicle and has a characteristically small interior.
As such, the touchscreen was located close to the driver (placed in front of the centre console) (see Figure 2), and therefore generally within easy reach. This might not be the case in larger vehicles, such as SUVs etc., where touchscreens may be placed outside of easy-reach zones.
There was also recognition for the potential benefit of using gestures and haptics revealed through the post-study interviews, with many participants recognising that gestures and haptics could enable completely vision-free interactions as familiarity and usability improves; similar claims could not be made for current touchscreen technology.
It is worth noting that the gestures, associated haptic sensations and experimental interfaces were developed specifically for the study, and therefore all results are based on this bespoke implementation, experimental set-up and 'post-hoc' installation. Thus, some of the performance metrics associated with the combination of touchscreen and haptics, for example, may simply reflect a poor integration of these two technologies. Moreover, future implementations would likely be seamlessly installed within vehicle interiors. In addition, further developments in the ultrasound technology and automotive-UX design are likely to offer benefits in terms of improved usability and reduced response latency for all interfaces. For the gestureswith-haptics, in particular, there is further scope to develop novel gestures and distinct haptic sensations to help drivers differentiate and select targets. Moreover, future gestureswith-haptics interfaces need not be bound by the traditional restrictions of a visual interface (e.g. a limited physical space to present a finite number of elements), and this could dramatically increase the scope for novel, multifarious interactions.
Finally, it is worth highlighting that the activities under investigation were purposefully chosen to be task-oriented and not goal-oriented. Therefore, participants were required to make repeated selections and manipulations using a rudimentary interface with limited 'real-world' functionality or appeal. This was a necessary experimental constraint to avoid confounding effects, but it is recognised that in practice, drivers would have a goal in mind when interacting with an in-vehicle device, such as increasing music volume. This would not only provide feedback (e.g. the music gets louder), but might also not necessitate the accuracy demanded during the study (cf. moving the sliderbar by 3 increments). While these factors may affect realworld behaviour with such a system, the intention in conducting the study was to provide a robust and controlled investigation of gestures enhanced with mid-air haptic feedback compared to a traditional touch-surface interface, in a driving context.

CONCLUSION
The study evaluated the novel use of ultrasound to emulate discrete mid-air buttons and a graduated slider-bar, activated using gestures, in a driving context. By comparing this with a traditional touch-surface interface, the study

Session 1: Haptics and Gestures
AutomotiveUI '18, Toronto, Canada shows clear potential for gestures enhanced with mid-air ultrasonic haptic feedback in the automotive domain, with reductions in visual demand, shorter interaction times, and improved accuracy (for slider-bar tasks) evident when haptic feedback was provided. The combined gesturehaptics interface was also very popular amongst participants, who rated it particularly highly for 'continuous' slider-bar manipulations. Further work is required to optimise sensations and interactions, for example, to improve the speed of response during discrete button-selection tasks. In addition, future work could seek to locate the technology within more representative driving contexts, where vibrations, vehicle movements, and other demands of real-world driving may impact on usability.