Pulling Back the Curtain on the Wizards of Oz

Porcheron, Martin; Fischer, Joel E.; Reeves, Stuart

Abstract

The Wizard of Oz method is an increasingly common practice in HCI and CSCW studies as part of iterative design processes for interactive systems. Instead of designing a fully-fledged system, the 'technical work' of key system components is completed by human operators yet presented to study participants as if computed by a machine. However, little is known about how Wizard of Oz studies are interactionally and collaboratively achieved in situ by researchers and participants. By adopting an ethnomethodological perspective, we analyse our use of the method in studies with a voice-controlled vacuum robot and two researchers present. We present data that reveals how such studies are organised and presented to participants and unpack the coordinated orchestration work that unfolds 'behind the scenes' to complete the study. We examine how the researchers attend to participant requests and technical breakdowns, and discuss the performative, collaborative, and methodological nature of their work. We conclude by offering insights from our application of the approach to others in the HCI and CSCW communities for using the method.

Citation

Porcheron, M., Fischer, J. E., & Reeves, S. (2020). Pulling Back the Curtain on the Wizards of Oz. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW3), Article 243. https://doi.org/10.1145/3432942

• Journal Article Type: Article
• Acceptance Date: Oct 31, 2020
• Online Publication Date: Jan 5, 2021
• Publication Date: 2020-12
• Deposit Date: Jan 12, 2021
• Publicly Available Date: Jan 28, 2021
• Journal: Proceedings of the ACM on Human-Computer Interaction
• Print ISSN: 2573-0142
• Electronic ISSN: 2573-0142
• Publisher: Association for Computing Machinery (ACM)
• Peer Reviewed: Peer Reviewed
• Volume: 4
• Issue: CSCW3
• Article Number: 243
• DOI: https://doi.org/10.1145/3432942
• Public URL: https://nottingham-repository.worktribe.com/output/5223052
• Publisher URL: https://dl.acm.org/doi/10.1145/3432942
• Additional Information: ©2020 Copyright held by the owner/author(s). Publication rights licensed to ACM.This is the author’s version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of the ACM on Human-Computer Interaction, https://doi.org/10.1145/3432942.