(Over)Trusting AI Recommendations: How System and Person Variables Affect Dimensions of Complacency

Harbarth, Lydia; Gößwein, Eva; Bodemer, Daniel; Schnaubert, Lenka

Abstract

Over-trusting AI systems can lead to complacency and decision errors. However, human and system variables may affect complacency and it is important to understand their interplay for HCI. In our experiment, 90 participants were confronted with traffic route problems guided by AI recommendations and thereby assigned to either a transparent system providing reasons for recommendations or a non-transparent system. We found transparent systems to lower the potential to alleviate workload (albeit not to neglect monitoring), but to simultaneously foster actual complacent behavior. On the contrary, we found performance expectancy to foster the potential to alleviate workload, but not complacent behavior. Interaction analyses showed that effects of performance expectancy depend on system transparency. This contributes to our understanding how system- and person-related variables interact in affecting complacency and stresses the differences between dimensions of complacency and the need for carefully considering transparency and performance expectancy in AI research and design.

• Journal Article Type: Article
• Acceptance Date: Dec 29, 2023
• Online Publication Date: Jan 22, 2024
• Publication Date: Jan 22, 2024
• Deposit Date: Jan 23, 2024
• Publicly Available Date: Jan 25, 2024
• Journal: International Journal of Human-Computer Interaction
• Print ISSN: 1044-7318
• Electronic ISSN: 1532-7590
• Publisher: Taylor and Francis
• Peer Reviewed: Peer Reviewed
• Pages: 1-20
• DOI: https://doi.org/10.1080/10447318.2023.2301250
• Keywords: Complacency; human-AI interaction; performance expectancy; transparency; trust
• Public URL: https://nottingham-repository.worktribe.com/output/30110889
• Publisher URL: https://www.tandfonline.com/doi/full/10.1080/10447318.2023.2301250