Isolating the Effect of Off-Road Glance Duration on Driving Performance: An Exemplar Study Comparing HDD and HUD in Different Driving Scenarios

Objective We controlled participants’ glance behavior while using head-down displays (HDDs) and head-up displays (HUDs) to isolate driving behavioral changes due to use of different display types across different driving environments. Background Recently, HUD technology has been incorporated into vehicles, allowing drivers to, in theory, gather display information without moving their eyes away from the road. Previous studies comparing the impact of HUDs with traditional displays on human performance show differences in both drivers’ visual attention and driving performance. Yet no studies have isolated glance from driving behaviors, which limits our ability to understand the cause of these differences and resulting impact on display design. Method We developed a novel method to control visual attention in a driving simulator. Twenty experienced drivers sustained visual attention to in-vehicle HDDs and HUDs while driving in both a simple straight and empty roadway environment and a more realistic driving environment that included traffic and turns. Results In the realistic environment, but not the simpler environment, we found evidence of differing driving behaviors between display conditions, even though participants’ glance behavior was similar. Conclusion Thus, the assumption that visual attention can be evaluated in the same way for different types of vehicle displays may be inaccurate. Differences between driving environments bring the validity of testing HUDs using simplistic driving environments into question. Application As we move toward the integration of HUD user interfaces into vehicles, it is important that we develop new, sensitive assessment methods to ensure HUD interfaces are indeed safe for driving.


INTRODUCTION
Future in-vehicle displays may provide visual information to users by overlying graphics through the windshield and onto the surrounding environment, advancing potential capability of in-vehicle displays. These advanced head-up display (HUD) interfaces must be assessed for glances away from the road as problematic, and resulting guidelines (e.g., AAM, 2002;ISO, 2006;SAE, 2000) indicate that in-vehicle displays should encourage drivers to return glances back to the road ( ). Thus, researchers often assess in-vehicle displays by focusing on glance behaviors, such as the duraareas of the road or surrounding environment.
One established assessment method is Senders' visual occlusion method (Senders et al., 1967), which considers the central visual demands but disregards information gained using peripheral vision (Burnett et al., 2013;. While ignoring peripheral visual cues may be valid for HDD testing, to gather information using peripheral vision while using the display. Another prevalent assessment method is the National Highway Transportation Safety Administration's (NHTSA's) Eye Glance in a Driving Simulator (EGDS) method, in which display acceptability is determined by average display glance duration, percentage of time looking at the display, and total time with the NHTSA, 2012). While glancebased methods of assessing display safety have been validated for use with traditional in-vehicle head-down displays (HDDs), no such validation has taken place for use with novel displays of applying current NHTSA assessment methods to emerging technologies such as HUDs. The study presented herein is an important step in determining whether two critical elements of common in-vehicle display assessment methods are suitable for HUD interface assessment: (1) glance durations toward the display, and (2) the driving environment. In order to test these elements, we applied a novel method to systematically control glance duration and visual istic driving environment as a replacement for national assessment standards, especially given the unique nature of HUD usage.

Visual Attention Toward In-Vehicle Displays
tal part of understanding driving performance, especially when assessing in-vehicle visual displays (Cotter et al., 2008). Drivers must rapidly process and respond to dynamic visual infordisplays contribute additional visual load. Even driving-related information displayed within the vehicle can be dangerous if focusing visual attention toward the display causes drivers to in-vehicle visual displays can be especially dangerous due to increased information quantity as information already present in the real world must be processed along with added virtual graphics in the case of HUDs, or as graphically rich HDDs provide detailed maps on increasingly large touch-screen displays. These visually rich displays may require more visual attention to process through the information, ultimately increasing the risk of driving accidents (NHTSA, 2010). The risk is especially present when the display requires or encourages for more than 2 s (referred to in the literature as a "long glance"; Klauer et al., 2006;NHTSA, 2012;Zwahlen et al., 1988 cade) to an area of interest (AOI) combined and saccades within that AOI (NHTSA, 2012) A new glance begins when a saccade leaves one AOI (e.g., roadway) and moves into another data collected using HDDs, before the widespread emergence of HUDs. Therefore, the impact of HUD interface design and usage on drivers' behavior and performance is not yet fully understood. Furthermore, researchers haven to yet determined how best to measure visual distraction and resulting safety associated with HUD interfaces.
HUDs allow drivers to receive information important consideration for in-vehicle display design (Wittmann et al., 2006). It is possible that toward HDDs-most likely because drivers using HUDs may leverage peripheral vision for lane keeping and other basic visual tasks associated with driving (Horrey & Wickens, 2004a). As such, traditional methods of assessing visual as "on-road" since these glances are in the direction of the driving scene. Yet, peripheral vision drivers must also attend and respond to roadway events (Horrey & Wickens, 2004a). In this case, glances toward HUDs could be considaccommodate away from the road scene and onto the focal plane of the HUD; this is likely to result in both visual and cognitive distraction. A recent study suggests that even when HUD graphics are presented at the same focal depth as the real-world reference (e.g., a lead vehicle), there is a cognitive cost to switching between the graphic and real-world reference (Gabbard et al., 2019). Therefore, throughout this work, we consider glances to the graphics on the HUD Indeed, changes in drivers' glance and driv- (Donkor, 2012). Researchers have employed a for HUDs including visual search tasks (Smith et al., 2015(Smith et al., , 2016(Smith et al., , 2017, navigation tasks (Bolton et al., 2015;Liu & Wen, 2004), verbal response tasks (Horrey & Wickens, 2004b), Horrey & Wickens, 2004c;Kim et al., 2013;Liu & Wen, 2004 tasks that systematically demanded drivers' glance duration, was managed within the study design. In studies where visual attention was ticipants distributed road and display glances HDDs (Bolton et al., 2015;Horrey & Wickens, 2004b;Smith et al., 2015Smith et al., , 2016Smith et al., , 2017. Because roadway glances and driving behavior fering driving behaviors may have been caused in part by changes in adopted glance behaviors. Additional research is needed to understand underlying causes of changes to driving performance and the implications of these changes for assessing new HUD interfaces for safe, on-road use.

Driving Environment
In driving simulator-based research, the driving environment includes the driving scene and outcomes Teh et al., 2014). However, driving environment is not frequently displays is often conducted under non-binding NHTSA guidelines, whereby participants follow a single lead car traveling at a constant 50 mph on a straight, two-lane road with little or NHTSA, 2012). However, past formance indicated driver workload increased trol, headway, and lane keeping (Teh et al., 2014). Further, driving environment can impact glance scenario may not elicit authentic driving behavior . Thus, though glance patterns while using HUDs and HDDs will likely it is unclear whether these changes maintain similar patterns. Because physiological indicators like glance allocation are used to predictor changes in workload ( ) and, ultimately, driving behavior, researchers must understand and validate these glance-based assumptions for HUDs. If changes in glance from changes found while using HDDs, then there is further evidence for establishing new methods of assessment.
Hypotheses participants' driving behavior and vehicle control changes when glance duration varies while while participants used HDDs and HUDs to complete a visually demanding task in two different environments. We tested two hypotheses for this work: H1: As the duration of focused visual attention toward a display increases, driving performance deteriorates more quickly when using HDDs compared to HUDs.

H2:
Simple driving environments (e.g., NHTSA-prescribed) are less likely to than driving environments which include dynamic elements (e.g., curves and other vehicles).

METHODS
The study took place at the University of Nottingham, UK, and was approved by the University's faculty of Engineering Ethics Committee and the Institutional Review Board at Virginia Tech (#17-563); informed consent was obtained from each participant.
Participants drivers (M = 6357.5 miles per year) with a valid driver's license for at least 2 years (M = 14.75 years) participated in the study. Participants were aged 18-65 years old (M = 33.95 years) and self-reported that they had normal or corrected-to-normal vision. No participants based HUDs.

Driving Task
Participants completed a series of driving tasks using the car-following paradigm (Brookhuis et al., 1994;NHTSA, 2012) in our UK-based driving simulator, while complying with UK driving laws. The lead car remained in the left lane of the road throughout all drives but on the driving environment, described below.
Conventional environment. Our conventional driving environment adhered to NHTSA guidelines specifying that the lead car travel at a constant speed of 50 mph on a straight, two lane road (NHTSA, 2012). The conventional stimuli to divert visual attention away from the focused visual attention task. Participants inia lead car appeared on the road directly in front of participants' simulated car. Participants continued to drive, following the lead car at a safe distance, while completing secondary (focused visual attention) tasks. The conventional environment allowed drivers to anticipate and respond to the behavior of the lead car and the roadway.
Realistic environment. In our realistic driving environment, participants followed a variable-speed lead car on a multi-lane road traveling in the same and opposite directions, with the UK national speed limit of 70 mph, appropriate to this type of roadway . The environment included varied speeds, additional road curvature, and increased volume of other cars to provide more realistic mittent lead car "comfort braking" (Large et al., 2018;Pampel et al., 2019), which occurred up the same speed as participants, meaning that the lead car speed was variable and determined by the speed at which participants drove (but they did not know that this was occurring).

Focused Visual Attention Task
At the beginning of each drive, we verbally instructed participants to maintain safe control of the vehicle and follow the lead car at a safe driving distance (primary task) while completing focused visual attention tasks to control task). To complete these tasks, participants focused visual attention on the selected display and watched a single white letter changing every .1 s until it randomly paused for .4 s, at which point participants read aloud the paused letter. This method encouraged participants to maintain foveal attention directed to the display for a predetermined glance time. To successfully complete the task, participants could not look away from the stimuli until the task ended and the screen changed to a blank screen (HDD) or became fully transparent (HUD).
We selected durations of 1 s, 2 s, 5 s, 10 s, and 20 s for the focused visual attention task. However, during pilot tests for this study, HDD glances longer than 5 s resulted in crashes often enough that data loss became a concern. Thus, resulting data loss. Three repetitions of each glance duration (HDD: 1 s, 2 s, 5 s; HUD: 1 s, 2 s, 5 s, 10 s, 20 s) were randomly ordered within each drive such that participants were unable to short breaks between tasks so participants could refocus on driving. When a new task began, a car horn sound alerted participants to stimulus appearance, but participants did not know the duration. Participants wore eye-tracking glasses (ETG) to enable us to validate their visual behavior.

Equipment
We conducted the study in a medium-Factors Research Group Lab at University of Nottingham (UK). The simulator included a with rear and side mirror displays. Participants drove in both environments in a right-hand and with a Microsoft Surface Pro 4 Tablet was mounted using the suction cup mount seen in Figure 1. We displayed the focused visual attention task in white font on the displays using time embedded slides in PowerPoint, and forward-facing view using SensoMotoric matched the visual angle for the tasks such that plays was larger than the suggested 0.25" for in-vehicle displays (Green et al., 1993).

Procedure
After participants consented, we seated them in the driving simulator and helped them adjust ETGs, and calibrated the software. We then was correct through the eye-tracking video feed (Figure 1). The purpose of the calibration was to ensure that participants viewed the projected letters at the same location relative to the lead After calibration, participants undertook a practice drive in the simulator. We instructed participants to drive 70 mph (the UK national speed limit) in the realistic environment and 50 mph in the conventional environment (in line with NHTSA recommendations). Once participants were familiar with driving in the simuattention task. Participants subsequently undertook a second practice drive while simultaneously doing the focused visual attention task.
balanced): three in realistic and three in conventional environments. Participants drove with no display (baseline), HUD, and HDD. During the baseline drive, participants drove for 5 min with no secondary task. Between drives, participants took a break, if desired. All participants were

ANALYSIS
obtained from the ETG to validate our method glance duration, glance duration frequency, and total glance time allocated to each AOI, that is, the road, display (HUD or HDD), or other vehicle instruments (e.g., mirrors and speedometer). Therefore, the method elicited similar visual behavior and division of visual attention regardless of display type, something that has until now not been systematically demonstrated in HUD driving research.
driving performance, we collected lateral and longitudinal vehicle control data. We calculated lane position (LP) according to SAE J2944 10.1.1.1 (Option A), meaning that the lateral position was determined relative to lane center (Green, 2013). Standard deviation of lane position (SDLP) was derived from lane position (Cotter et al., 2008). Because the lead car drove istic driving environments, we used minimum distance to collision (MDC) to assess longitudinal vehicle control.
for HUD drives, denoted as HUD-20, (2) 5 s of data for HDD drives, denoted as HDD-5, and (3) to compare HUD to HDD (Combined-5). Thus, duration for each display type individually and duct our analysis, we subdivided each dataset into sequential epochs of 1s duration. Since participants did not know the focused visual attention duration when they began each task, they could not predict how long they would need to all glance tasks would have similar characteristics for a given display type, regardless of the total focused visual attention duration.

RESULTS
For lateral and longitudinal data in the HUD-20 and HDD-5 datasets, we conducted a repeated-measures ANOVA with focused visual attention duration (5 s or 20 s), sequential time (1-5 s or 1-20 s), and driving environment (realistic or conventional) as our independent in the model. In the Combined-5 dataset, we conducted a repeated-measures ANOVA (as above) with display type (HUD or HDD) as an additional independent variable. We determined p < .05.

Lane Position
on lane position in all three datasets, with the third repetition resulting in a lane position closest to center than the second repetition, which was further to the right for all datasets (Table 1).
ment and display in the Combined-5 dataset. While all conditions resulted in lane positions slightly right of center, post hoc testing showed that when using HDDs, participants drove further to the right (more negative) in the Note. *Differences between levels found in post hoc testing is indicated by "level 1 > level 2," where the level with the larger mean is listed first.
conventional environment than the realistic environment ( Figure 2).

Standard Deviation of Lane Position
environment and sequential time on the SDLP (Table 2). Post hoc testing and Figure 3 show that the realistic environment resulted in higher SDLP than the conventional environment for all three datasets. In the Combined-5 and HDD-5 datahigher SDLP than the second and third epochs. In the HUD-20 dataset, the second epoch was associated with higher SDLP than several other epochs (8 s, 5 s, 17 s).
play and sequential time, with the fourth and epochs. The third HDD epoch was associated HUD epochs. Thus, there was a pattern of participants' SDLP increasing with passing time were not present with the HUD.

Minimum Distance to Collision
For all three datasets, the conventional environment resulted in longer MDC than the realistic (Table 3, Figure 4). In the Combined-5 dataset, HDD use resulted in longer MDC than HUD use.

DISCUSSION
two assumptions underpinning current glancebased display assessments: (1) glance duration can be used to predict driving behavior, and (2) we systematically controlled focused visual impact of more realistic driving environments on drivers performing visually demanding tasks. In general, we found that both display ipants' driving behavior when visual attention was controlled.

Durations
As we systematically controlled participants' performance deterioration associated with HDDs compared to HUDs (H1). We found no signifuse was associated with increasing SDLP over time which was higher than when using HUDs. The trend in the HDD data suggests SDLP may increase until intervention occurs (e.g., looking back to the road). When controlling for visual attention duration toward HUDs, participants showed no marked increase in SDLP over the participants using the HDD showed increased SDLP as the task duration increased, especially after 2 s. In particular, the third, fourth, and all associated with higher SDLP (i.e., degraded lateral vehicle control) than the same epochs of changes in both lateral and longitudinal vehicle control measures between displays, with HDD use resulting in more rapid and diminished driving performance than HUD use. The Combined-5 data showed participants using HDDs allowed more distance between their car and the lead vehicle compared with HUD use, which is indicative of more conservative driving (Brookhuis et al., 1994). Because it suggests that participants were less comfortevidence of deteriorated driving performance relative to HUD use. vehicle control support H1, suggesting that drivers may sustain longer visual attention toward in driving performance. There are many potential display types, including increased use of periph-  to HDDs. However, two theories provide possivehicle control. First, Senders et al. (1967) posits that time looking away from the road, and in this case toward HDDs, results in increased uncertainty, which impacts drivers' behavior (Senders et al., 1967). As participants maintained glances toward the displays, their visual uncertainty about the state of the road may have increased more rapidly during HDD tasks because participants could not leverage their peripheral vision as they could when using the HUD. As uncertainty increased, drivers may have been less aware of their lane position resulting in over-or under-compensation for changes in lane position, ultimately impacting their SDLP. A second theory concerns gaze concentration ers primarily focus on one point in the road (their lane position variation (Li et al., 2018), supporting the decreased SDLP evident with HUD use. While Senders' theory would support HUD use in vehicles because degraded lateral vehicle control was lower due to lower uncertainty when participants used HUDs. However, if HUD use indeed causes ing, HUDs may negatively impact drivers' ability to respond to roadway events-as seen when using AR applications in other domains (Kerr et al., 2011). While these two theories may result in safer, it is important to note that both theories may be evident in this study. It is possible that HUDs while also causing new problems. Therefore, furtheories and to determine design implications.

Driving Environment
Characteristics of the driving environment can impact driving performance (Horrey & Wickens, 2004b;Senders et al., 1967), yet some assessment methods, such as EGDS (NHTSA, 2012) specify one type of driving environment. We therefore driving environments on driver performance and ing performance decrements in the more realistic environment (H2).
ing environment on lane position, but the realposition than the conventional environment during HDD use. The road geometry slightly the realistic and two lanes in the conventional), ception of space and the resulting position they between positions were small (less than one foot), so the real-world implications are likely minimal.
In all three datasets, the realistic environment resulted in higher SDLP (lateral instability) than the conventional environment, suggesting the realistic environment was more challenging to sequential time and display in the HDD-5 dataset ronment was associated with higher SDLP than in the conventional environment for all epochs. Moreover, later epochs in the realistic environment were associated with higher SDLP than early epochs, showing an increase in SDLP over timethis was only present when participants used HDDs. This supports H2 in part, because driving performance deteriorated more quickly in our realistic driving environment than in the conventional, but only when using the HDD. Thus, participants' between the two displays.

Assessment Methods
Because many in-vehicle display assessments are based on glance behaviors (e.g., NHTSA, 2012 might be assumed to have a similar negative toward HDDs. Yet, currently accepted assessment techniques were developed using data collected from HDDs. While participants in our study drove similarly when using both displays in the condriving behaviors in the realistic environment.
ently, depending on the display. In other words, not and HDDs in prior research can be attributed to selected glance behaviors. This is important because while the NHTSA EGDS method is commonly used to assess HDDs, it only includes one type of driving environment that is not representative of all, or arguably any, real-world scenario. Because HUDs and HDDs ronments, we cannot assume that results from a driving. Assessing glance behavior in simple environments, like our conventional environment, their ambient or peripheral vision, as evidenced by the driving performance measures. In other words, even when the duration of focused visual attenbetween HUD and HDD. Thus, assessing HUDs driving behavior developed with HDDs may be inadequate. Instead, we must develop new methods that are valid for each display type.

Long Glances
Prior research into drivers' glance behavior indicates that there is a 2-s threshold for glances away from the road, above which the likelihood of Klauer et al., 2006). In our study, the HDD was most in-keeping with for HDD after 2 s of focused visual attention. Thus, our study suggests that one contributor to increased crash risk at 2 s could be the result of increased lateral instability. However, we did not HUD, which may suggest that HUDs are a safer alternative to HDDs because they permit glances without hindering lateral vehicle control. It might also mean that drivers using HUDs are able to maintain lateral control for longer than the widely accepted 2 s, and new "safety" thresholds could be established for HUDs. While it is not possible to determine a new threshold from our results, it appears that visual attention focused on HUDs situations.

LIMITATIONS
ing simulator study included a relatively small n = 20). Future work should be ticipants as well as on-road studies.

CONCLUSIONS AND FUTURE WORK
This work has uniquely contributed to drivingrelated research by providing a systematic method road glances"). Applying this, we found that drivusage even when visual attention did not. Further, simplistic driving environments commonly between display type, whereas a more realistic ences in vehicle control. Thus, measures implying that driving performance can be determined based on glance pattern alone in simple environments are NHTSA EGDS test may provide poor recommendations when assessing HUDs. Because of this, we must pursue other methods of assessing driver behavior and performance to ensure safe on-road interactions. Assessing HUDs in visually rich environments may be required to provide realistic feedback on drivers' potential performance while using this type of display. Further, standard recommendations, such as the widely accepted 2-s rule, should be evaluated for HUDs in future work to help designers quickly assess potential dangers of using these displays.

ACKNOWLEDGMENTS
This material is based upon work supported by the National Science Foundation under Grant No. 1816721. The authors wish to thank Amanda Carroll who assisted in data collection and analysis.

KEY POINTS
Visual attention has been closely linked with driving behavior and is commonly used to assess in-vehicle visual displays. Augmented reality head-up display (HUD) usage behaviors than traditional in-vehicle displays. Even when glance behavior is controlled, HUD tive to traditional (head-down) displays. and traditional in-vehicle displays.