Toward Understanding the Design of Intertwined Human–Computer Integrations

Mueller, Florian ‘Floyd’; Semertzidis, Nathan; Andres, Josh; Marshall, Joe; Benford, Steve; Li, Xiang; Matjeka, Louise; Mehta, Yash

Abstract

Human–computer integration is an HCI trend in which computational machines can have agency, i.e., take control. Our work focuses on a particular form of integration in which the user and the computational machine share agency over the user’s body, that is, can simultaneously (in contrast to a traditional turn-taking approach) control the user’s body. The result is a user experiencewhere the agency of the user and the computational machine is so intertwined that it is often no more discernable who contributed what to what extent; we call this “intertwined integration”. Due to the recency of advanced technologies enabling intertwined integration systems, we find that little understanding and documented design knowledge exist. To begin constructing such an understanding, we use three case studies to propose two key dimensions (“awareness of machine’s agency” and “alignment of machine’s agency”) to articulate a design space for intertwined integration systems. We differentiate four roles that computational machines can assume in this design space (angel, butler, influencer, and adversary). Based on our craft knowledge gained through designing such intertwined integration systems, we discuss strategies to help designers create future systems. Ultimately, we aim at advancing the HCI field’s emerging understanding of sharing agency.

Citation

Mueller, F. ‘., Semertzidis, N., Andres, J., Marshall, J., Benford, S., Li, X., Matjeka, L., & Mehta, Y. (2023). Toward Understanding the Design of Intertwined Human–Computer Integrations. ACM Transactions on Computer-Human Interaction, 30(5), 1-45. https://doi.org/10.1145/3590766

• Journal Article Type: Article
• Acceptance Date: Nov 20, 2022
• Online Publication Date: Sep 23, 2023
• Publication Date: Oct 31, 2023
• Deposit Date: Mar 14, 2025
• Publicly Available Date: Mar 17, 2025
• Journal: ACM Transactions on Computer-Human Interaction
• Print ISSN: 1073-0516
• Electronic ISSN: 1557-7325
• Publisher: Association for Computing Machinery (ACM)
• Peer Reviewed: Peer Reviewed
• Volume: 30
• Issue: 5
• Article Number: 73
• Pages: 1-45
• DOI: https://doi.org/10.1145/3590766
• Keywords: Human-Computer Interaction
• Public URL: https://nottingham-repository.worktribe.com/output/25395160
• Publisher URL: https://dl.acm.org/doi/10.1145/3590766
• Additional Information: Received: 2021-05-20; Accepted: 2022-11-20; Published: 2023-09-23