"It's small talk, jim, but not as we know it.": engendering trust through human-agent conversation in an autonomous, self-driving car

The use of speech has been popularised as a human-vehicle interface in the automotive domain. While this is most often associated with alleviating concerns of driver distraction and cognitive load, the study explores whether the presence of conversation could, in and of itself, engender trust in the technology, based on our understanding of speech in humans. Thirty-four participants were transported in a fully-autonomous, self-driving 'pod' vehicle, accompanied by a natural-language, conversational interface ('UltraCab'), which was delivered using Wizard-of-Oz methodology. Emergent, trust-related themes were identified from the conversation that took place between participants and UltraCab, posing the question of whether participants sought an emotional connection with the vehicle, or whether conversation remained primarily functional. Implications for trust and the design of conversational interfaces are discussed.


INTRODUCTION
Autonomous, self-driving vehicles are expected to revolutionise everyday travel with anticipated benefits of improved road safety, efficiency, comfort and mobility, but are not without their own set of unique Human Factors issues [20]. First experiences are likely to be in driverless 'pods' that operate in specific contexts, for example, 'geo-fenced' environments such as university campuses, airports etc. [29], with several examples already under evaluation. Nevertheless, major concerns have been expressed regarding the public's willingness to adopt the technology [21], particularly relating to issues of trust [17]. The development of trust is also an important component of successful human-human conversation [11]. The study therefore explores whether people's conversational behaviour (for example, the strategies and mechanisms they use to establish and calibrate trust), is the same when the second interlocutor is the vehicle itself.

Trust
Trust has been defined as an individual's willingness to depend on another party because of the characteristics of the other party [33]. Routed in human sociology and psychology, the concept of trust was originally attributed exclusively to interpersonal relationships, and can be viewed as a mechanism to reduce feelings of uncertainty [27].
The concept of trust has also been used to elucidate the relationship between users and technology [19]. In this context, trust is considered to be the extent to which people believe that the technology will perform effectively and without a negative or injurious outcome, and can therefore be influenced by factors such as reliability [19]. Trust consequently shapes individuals' attitudes and ultimately determines their behaviour, such as the extent to which they rely upon the technology and their operational strategies towards its use [25,26].
Evidence shows that an effective strategy that can influence users' understanding of and relationship with a non-human entity is to add 'human' elements [10,15,34]. Factors such as a discernible face or body (particularly if provided in a social context) can encourage users to endow the entity with human qualities (motivation, characteristics, behaviour etc.) -inspired by the said feature. This can change the value that users place on the item, for example, the level of trust it affords, and can influence how they behave towards it [8,24,30]. Even so, it is worth highlighting that, in so doing, people do not impute human personality in all its subtle complexity, but rather "paint with broad strokes, thinking only of those traits that are useful to [them] in the [24].
One such feature is speech. Philosophical debates identify speech as one of the quintessential markers of humanness [30]. It is the primary means of social identification amongst humans and implicates more parts of the brain than any other function [28]. Moreover, speech is peppered with salient, socially-relevant, cues above and beyond the lexical content, that humans are experts at extracting and comprehending using vocal characteristics such as pitch, cadence, speech rate and volume. These are subsequently used to provide systematic guidance for determining gender, personality and emotion-specific actions, such as who to like and trust [30,31].
Trust is also an important component of effective conversation between humans. In this context, trust is used to establish common ground and ensure mutual understanding [11]. Moreover, conversational partners use their exchanges to calibrate trust in each other and to declare their own trust [30,31]: speakers employ strategies, such as 'small talk' (or so-called, 'phatic communication'), humour, politeness, and vague language (deliberately imprecise language) to build trust over time [9,11].
Extensive research has demonstrated that humans appear to lack the wherewithal to overcome evolutionary instincts and behaviour, and respond to vocal utterances from a computer in a similar manner to other people, ascribing humanlike characteristics and attending to talking machines as if they were interacting with another human [30]. The presence of speech and conversation embedded within technology is therefore a powerful catalyst, encouraging the listener and interlocutor to imbue essential human characteristics to the host [34]. Indeed, manipulating speech interfaces has already been used to exploit these automatic responses. For example, different digital 'personalities', created by varying the vocal characteristics and language content of spoken language interfaces, have been shown to influence trust, performance and learning, and even consumers' buying habits in commercial contexts [30].
The use of speech has also been popularised as a human-vehicle interface in the automotive domain. While this is most often associated with alleviating concerns of driver distraction and cognitive load [18], it can also create a more natural user experience [1,5]. Moreover, given aforementioned literature, it would appear feasible that interacting using natural, spoken language (with a vehiclebased agent as the conversational partner) could also influence occupants' perception of the vehicle itself (and by association, the technology it proffers), with initial investigations already providing some support for this hypothesis [2]. Nevertheless, it remains unclear what form human-agent 'conversation' in this context should take.

Conversational Agents
Existing spoken human-agent interactions, such as those with Intelligent Personal Assistants (IPAs) like Amazon Alexa, are often limited to isolated question-answer pairs [32]. In aiming to inform the design of 'appropriately' conversational agents, Clark et al. [13] explored what people valued in human-human conversation and how they felt this should manifest in human-agent communication. Responses show that in general, interviewees favoured a more utilitarian definition of conversational qualities when interacting with a technology-based agent, repudiating the need for developing bond and common ground with the host. In particular, trust was discussed exclusively in functional terms, emphasizing factors such as security, privacy and transparency rather than emotional trust (i.e. sharing personal information and vulnerabilities to increase social bond), as would be expected in human conversations.
Nevertheless, Large et al. [22] demonstrated the adoption of 'normal' conversational behaviours during the practical accomplishment of tasks when participants interacted with a highly-capable conversational agent (delivered using Wizard-of-Oz) in a simulated driving context. In the study, language usage patterns indicated that interactions with the agent were fundamentally social in nature, and mirrored many of the routine behaviours observed during human conversation. For example, participants were polite, actively controlled turn-taking during the conversation, and used back-channelling, fillers and hesitation, as they might in humanhuman interaction (HHI). Furthermore, participants expected the agent to understand and process complex requests mitigated with hedging words and expressions, and peppered with vague language and deictic references requiring shared contextual information and mutual understanding [7].
As such, the design of conversational user interfaces -and the conversation they permit -remains an interesting and complex area, with prospective users apparently declaring expectations that are different to what occurs in practice. To inform this debateand recognising that one of the aims of conversation is to build and strengthen positive relationships with others -we have explored the use of human-agent conversation as a means to engender trust in an autonomous, self-driving vehicle -a context in which issues of trust and acceptance are paramount [17]. While initial results show that there is an increase of perceived trust in situations where an agent was employed to talk to passengers (compared to situations absent of an agent -see [23]), the aim of the current paper is to identify indicators of trust that can be gleaned from the conversation that took place between human and car.

Participants
A representative sample of experienced drivers were recruited to take part (n=34), comprising 17 male and 17 female participants. Ages ranged from 21-58 years with a mean age of 40, driving experience from 3.5-40 years with a mean of 20.6 years, and selfreported annual mileage from 6,000 to 60,000 with a mean of 15,691. All participants were employees of Jaguar Land Rover.

Apparatus and Study Design
The study was conducted in the Urban Development Lab (UDL) ( Figure 1) indoor testing environment in Coventry using a fullyautomated driverless pod supplied by the RDM Group. The pod operated autonomously throughout the study. The testing environment was presented as an urban scenario, with shop fronts and associated buildings projected onto timber constructions and perimeter screens to emulate typical commercial premises. The layout enabled multiple routes to be followed, which were presented to participants as different journeys (for example, a shopping excursion, visiting a friend etc.). The full study incorporated a comparative analysis of three different interfaces: a touchscreen, a voice command interface and the conversational agent ('UltraCab') -specifically aiming to explore the impact of these on users' trust and affective experience. Each interface was experienced in a separate drive in a counterbalanced, within-subjects design. Participants were thus tasked with interacting with each of the interfaces, while undertaking their journey in the autonomous pod.
During each drive, participants were exposed to three usescases: entertainment (news, sporting headlines, music etc.), office/scheduling (managing and creating to-do lists, calendars and emails) and system notifications (requests for vehicle diagnostics etc.), with information content equivalent between interfaces. In addition, participants were able to undertake a final, free-choice journey, and thus completed four drives in total (see Large et al. [23] for full details of the study design).
Participants were recorded using a GoPro camera for subsequent analysis, as well as a second camera for real-time streaming and observation. A small booth on the edge of the testing arena housed a professional actor, who delivered the conversational dialogue (UltraCab) in real-time during the relevant drive using a Wizardof-Oz approach [14]. Interlocutions were conversational in nature and based on a script developed during earlier focus groups and informed by previous studies [22]. Participants were specifically told that they were interacting with a prototype 'agent' interface and could talk with it using free-flowing, conversational language.
During each drive, which lasted approximately eight minutes, participants were presented with various 'trust challenges' inspired by the methodology employed by Antrobus, et al. [2]. For example, the pod abruptly halted, claiming to have detected a pedestrian in the road ahead, and informed the participant that the journey would resume once the pedestrian had moved; in practice, the roadway ahead was clear. In addition, 'functional' trust challenges, exploring security and privacy issues were presented. For example, the vehicle requested access to participants' personal email accounts and diaries to resolve calendar conflicts etc.
After completing all four drives, a post-study interview was conducted to elucidate findings. Each participant was in attendance for approximately 1½ hours, including briefing and debriefing.

Analysis Approach
As part of the larger study, a number of measures were captured to determine participants' trust, affective experience, preferences etc.; these are reported in Large et al. [23]. In summary, the journey in which participants were accompanied by the conversational agent (UltraCab) invited the highest ratings of trust, and significantly increased the pleasure and sense of control over the journey experience, despite equivalent use-cases and information content in all three conditions. Moreover, 88% of participants chose the conversational agent to accompany them during their final 'free-choice' drive. Consequently, the primary focus of the current analysis is to explore the nature of the dialogue and conversational exchanges that took place with UltraCab in more detail, and to extract salient elements relating to trust, in particular to understand the higher ratings. A further intention in doing so was to identify whether participants engaged in genuine 'conversation' with the agent (thereby enabling them to make an appropriate assessment of whether to trust them -and by extension the autonomous vehicle -as they might with another human), or whether the conversation remained highly functional in nature, despite conversational aspects (yet ultimately still influenced trust).
The paper therefore also aims to explore whether there was evidence that people still sought an emotional connection with the technology, despite their apparent protestations to the contrary revealed in earlier research [13]. The aim is not only to identify the value of employing human-agent conversation to engender trust in the technology, but more fundamentally to unveil how trust should, or could, be engendered and developed in a human-agent conversational relationship.
Each participant's 'conversation' with UltraCab was transcribed and underwent deductive thematic analysis to provide a comprehensive understanding of the data based on prior research [6]. Specifically, we drew guidance from previous papers on in-car assistants, relational agents and linguistics to identify trust markers of small talk [3,16,22], pronouns [4], and common ground (manifested as trust through expertise during the interaction) [11,12,22].

RESULTS AND ANALYSIS
The following sections present trust-related themes that emerged in the conversations between participants and UltraCab. Representative extracts from the participants' dialogue with the agent are included and accompanied with relevant literature from the fields of both human communication and human-computer interaction (HCI). It is worth highlighting that the examples do not provide an exhaustive list of all the occurrences in which they have emerged, but rather provide illustrative examples of the salient trust-related features of the language that exists within the conversations that took place.

Small Talk, Introductions and Greetings
At the start of each drive, UltraCab initiated interactions with a greeting -stating its own name and requesting the name of participants, introducing itself and its conversational capabilities, and finally describing the automated driving scenario. Such small talk can often be found at the beginning of interactions to build and establish rapport, trust and interaction styles, and can also be found further along in interactions to help reinforce these perceptions [16]. Nevertheless, similar conversational elements were not commonly observed within the data. This echoes similar findings by Clark et al. [13], in which participants view machines as basic dialogue partners lacking in humanlike conversational capabilities. While the authors noted small talk common in conversations with strangers and in transactional conversations, in this analysis there were few examples to be found.
In one example where it appeared to be present (Excerpt 1), P19 provides the topic of discussion -in this case talking about the football club they support and the building of the club's new stadium. In one sense, this can be seen as transactional talk in that UltraCab is fulfilling information requests made by P19. Conversely, this may also be seen more as interpersonal as P19 is providing information to UltraCab under the knowledge that UltraCab has presented itself as being able to learn from participants and tailor itself accordingly. In this sense, P19 may be providing the necessary information to build on the common ground between P19 and UltraCab. P19 also informs UltraCab that its own information may be incorrect. However, as with Clark et al. [13], this common ground building may be viewed more as the ability of a system to personalise itself towards a user, where P19 is signalling specific preferences to the system rather than aiming to achieve an emotional bond per se. The lack of emotional small-talk (i.e. emphasising interpersonal relationships and de-emphasising the task-based aspect of interaction) could be a consequence of UltraCab's task-based focus of interaction and participants responding accordingly. Instead, beginnings of interactions are focused on greetings, for example, Ultra-Cab requests the names of participants, and introduces itself and its purpose. Common responses to these greetings and introductions were minimal, with participants providing limited responses to the system. These responses also included nonverbal actions (nodding by P13; yawning by P15) or simply silence, the latter perhaps indicating participant is waiting for UltraCab to engage in further activities, such as beginning driving, or they simply felt no need to respond. This contrasts to HHI, where silence in introductions may be considered rude in certain contexts. Other responses share similar reciprocity to openings, greetings and small talk in human interaction. In Excerpt 3, P4 mirrors some of UltraCab's turns in asking what the system would like to be called. Other responses include more requests about the system's capabilities or the task at hand ("Is it possible to stop on the way?" [P8]), or go straight into requesting services from the system ("Play some music" [P15]; "Okay. Can you check my emails?" [P16]). Asking for further information from the system may represent a human-agent version of small talk, whereby the system presents itself and its capabilities, and a user may engage in requesting more information if this is not deemed sufficient (akin to a one-sided 'sizing each other up' [16] Excerpt 3: Example of a reciprocal response to UltraCab. These responses to initial introductions and beginnings of interactions may also provide an insight into a user's perceived trust in a system. Non-questioning, acceptance and acknowledgements [P1, P5, P9, P13] may show a level of acceptance in a system's presented capabilities. Further system interrogation [P8, P10, P16, P18] suggests more information is required about a system, a task, or the context at hand, indicating that the user is still building and/or calibrating trust. Proceeding with commands or requests right away [P15] may also indicate trust in a system, or a user's desire to test and confirm its capabilities.

Pronouns as Possible Social/Trust Markers
During the interactions, participants used a variety of pronouns when referring to different activities. Initially, the system refers to the journey as "your destination" (i.e. that of the participants). However, participants often refer to the journey inclusively of the system, i.e. "our destination" (e.g. Excerpt 4), perhaps owing to the participants being a passenger and the system appearing to be the driver. Similarly, P9 asks the system "what time are we going to arrive?" highlighting that both system and participant will be arriving at the destination. In other examples, P8 declines the system when it offers to book an appointment in their calendar ("No. Let's leave it"), where the inclusivity may hinge on the system performing the act, but P8 having the final decision on its content and whether the booking should be implemented. Elsewhere, P26 refers solely to the calendar as "my calendar" indicating this is something belonging to the participant, whereas the process of 'booking an appointment' may be considered a joint venture. The use of inclusive pronouns in these examples may help to distinguish which activities are socially marked (conducted between participant and system) -activities in which trust is shared -and those which are considered to be individual (i.e. system or participant only).
P5: How long 'til we reach our destination? UltraCab: Estimated time of arrival: 6 minutes.
UltraCab: I have just performed a diagnostic test on the batteries. They are in good working order, but will be due for recalibration and optimisation in three months. Would you like me to book an appointment for this, for the 10th of December, at 3 o'clock? I cannot see any appointments in your calendar for that day.

Trust of Expertise throughout Interaction
During the interactions, there are occasions when participants indicate varying degrees of trust in the system, both within and outside of the designated trust challenges. For example, in Excerpt 8, P22 is aware the system recognises the 'Premier League' as a league in the English football system and consequently asks about the 'Championship' (the football league below the Premier League). 'Premier League' is a more distinct and specific noun, whereas 'Championship' can have multiple meanings and requires contextual information to provide a relevant answer. UltraCab's initial response to football-related questions may indicate P22's level of trust in its expertise and competencies in responding. Similarly, the system's response updates P22's mental model of what the system is capable of. This may be acting more like HHI conversational interaction, where consecutive turns are built upon one another in a dynamic process of developing common ground, as opposed to isolated question-answer pairs (e.g. [13,32] Regarding the designated 'trust challenges', many of the participants were accepting of the events or did not make it clear they had any concerns or further questions. There were some occasions where participants requested further clarification or transparency as to these events (e.g. Excerpt 9). In doing so, like P23, they may adopt the conversational style of interaction (i.e. not having to restate utterances as individual requests, but referring to other aspects of the interaction). Consequently, they are able to use conversational styles to update their trust in the system and their mental model of what it is capable of, while indicating further initial transparency may also be beneficial ("I assume. . . the sensors are working now?").
UltraCab: I'm sorry. I have detected a problem. My sensors are unable to detect any objects. I am currently recalibrating them.
P23: Whilst moving? UltraCab: We have recommenced the journey. I'm sorry [Participant 23]. This has added 24 seconds to your journey time.
P23: I assume. . . the sensors are working now? UltraCab: Sensors have been checked.
Drivers were able to take a second drive with the UltraCab, based on their preferences (see: [23]). For those participants experiencing UltraCab twice, the evidence shows that using a conversational style also extended between drives as well as within drives. In Excerpt 10, P3 is exposed to the agent performing a diagnostic on its batteries during Drive 1. Consequently, in Drive 2, P3 asks if the system has performed a "systems diagnostic" when it encounters trouble, building on their knowledge and experience from the previous drive. Interestingly, P3 refers to "systems diagnostic" rather than "diagnostic on the batteries", suggesting that they believe that the system will understand the reference.

Drive 1
UltraCab: I have performed a diagnostic on the batteries. They are in good working order, but will be checked for recalibration and optimisation in three months. Would you like me to book an appointment? P3: Yes please. UltraCab: Okay.

DISCUSSION AND CONCLUSION
This study explored the interactions between a conversational agent in an autonomous vehicle and its users to explore the nature of the dialogue and identify salient elements relating to trust. In doing so, this work examines the extent to which people engaged in 'actual conversation' with the agent and what evidence there was that people sought emotional and interpersonal connections through their interactions. Overall, it is evident that even when conversational interaction is afforded by the agent, people often still interact in a command-based style, and only occasionally embrace a more conversational approach. However, when they do engage in conversation with the agent, participants will refer to knowledge gained or utterances produced in previous turns within an interaction, as well as between interactions, signifying a move from traditional isolated question-answer pairs (e.g. [32]). Moreover, there is evidence of a shared experience between the passenger and the agent, through the use of joint pronouns (we, us, our etc.) in the formation of questions and utterances. Participants also sought clarification on system capabilities and processes, for example, and sometimes sought extended discussion on further information retrieval relevant to them (e.g. a specific football club) -a strategy perhaps best described as 'functional' small talk. In contrast, participants perform very little in the way of interpersonal interaction. This means that exchanges were not necessarily restricted or regimented by social norms and expectationsusers appear quite comfortable not responding to all agent-initiated dialogue: something that would be considered rude in HHI. This again signifies a marked change from our current understanding of 'conversation'.
Many of the conversational features identified by Large et al. [22] are also present here. However, despite these conversational elements and strategies, the 'conversation' itself perhaps more accurately mirrors Clark et al. [13] in that people perceived the technology as very tool-like, with mental models of functional rather than social machines. As such, elements such as 'small talk' are more commonly employed to seek further information or to personalise the system rather than necessarily to build rapport or establish common ground.
Overall, the evidence suggests that even when 'conversational' capabilities are present, this is understood by users to mean that it is possible to 'have a conversation' with the technology, in so far as questions may be asked using a natural, 'conversational' language approach. However, the interaction is not perceived as a 'conversation' in the same manner that one would attribute such interactions amongst human partners -as touched on by Porcheron et al. [32].
Thus, while participants in the study show evidence of conversational approaches and strategies, the matters of 'conversation', and in particular, trust, are still highly functional and task-based rather than interpersonal or social -people expect and perceive the UltraCab to be very tool-like, despite its apparent human-like capacity to 'engage in conversation'. This may also partly stem from people's mental models of current interfaces that afford little in the way of actual conversation, and focus more on requests to the system and responses to the user [32].
Even so, there may be situations in which people break away from the traditional isolated question-answer structures when they know it will be successful, and use referents in utterances for information or reference points from previous turns. As such, it is recommended that human-agent conversation in an automotive context should seek to enhance functional aspects of trust (concentrating on privacy issues etc.) rather than necessarily aiming to build an emotional relationship with the driver, but be ready to engage in driver-initiated 'functional' small talk (requiring higherorder artificial intelligence capabilities), as and when required. It is worth noting, of course, that the situation may be different in other contexts. For example, building emotional trust may be more salient for natural language interfaces in healthcare, where a human patient may be required to disclose personal information to the system.
It is important to recognise that although the pod operated completely autonomously, the experience was still very much experimental, in that the use-cases lacked real-world implications and consequences, and the overall journey experience was somewhat limited. Participants were thus required to actively engage with these pseudo tasks (for example, allowing the system to manage their calendar appointments and conflicts) and the environment. In addition, by employing a Wizard-of-Oz approach to deliver the conversational agent, there may be aspects of the interactions that were unrealistic (for example, we specified completely error-free understanding and enunciation). However, our intention in taking this approach was to exceed current state-of-the-art conversational interfaces, while conforming with our understanding of users' expectations of future talking-technology. Future work will continue to explore the potential benefits of employing conversational interfaces in an automotive context to understand the range of benefits (in addition to elevating trust) that they may offer, with the ultimate goal of creating design guidelines specific to this domain.