How UX Practitioners Produce Findings in Usability Testing

Reeves, Stuart

Abstract

Usability testing has long been a core interest of HCI research and forms a key element of industry practice. Yet our knowledge of it harbours striking absences. There are few, if any detailed accounts of the contingent, material ways in which usability testing is actually practiced. Further, it is rare that industry practitioners' testing work is treated as indigenous and particular (instead subordinated as a 'compromised' version). To service these problems, this paper presents an ethnomethodological study of usability testing practices in a design consultancy. It unpacks how findings are produced in and as the work of observers analysing the test as it unfolds between moderators taking participants through relevant tasks. The study nuances conventional views of usability findings as straightforwardly 'there to be found' or 'read off' by competent evaluators. It explores how evaluators / observers collaboratively work to locate relevant troubles in the test's unfolding. However, in the course of doing this work, potential candidate troubles may also routinely be dissipated and effectively 'ignored' in one way or another. The implications of the study suggest refinements to current understandings of usability evaluations, and affirm the value to HCI in studying industry practitioners more deeply.

Citation

Reeves, S. (2019). How UX Practitioners Produce Findings in Usability Testing. ACM Transactions on Computer-Human Interaction, 26(1), 1-38. https://doi.org/10.1145/3299096

• Journal Article Type: Article
• Acceptance Date: Nov 19, 2018
• Online Publication Date: Feb 6, 2019
• Publication Date: Jan 30, 2019
• Deposit Date: Nov 27, 2018
• Publicly Available Date: Nov 28, 2018
• Journal: ACM Transactions on Computer-Human Interaction
• Print ISSN: 1073-0516
• Electronic ISSN: 1557-7325
• Publisher: Association for Computing Machinery (ACM)
• Peer Reviewed: Peer Reviewed
• Volume: 26
• Issue: 1
• Article Number: 3
• Pages: 1-38
• DOI: https://doi.org/10.1145/3299096
• Keywords: CCS Concepts: • Human-centered computing → Usability testing; • Human-centered computing → Empirical studies in HCI KEYWORDS Usability testing; work practice; ethnomethodology; conversation analysis
• Public URL: https://nottingham-repository.worktribe.com/output/1314228
• Publisher URL: https://dl.acm.org/citation.cfm?id=3299096
• Additional Information: © ACM, 2019. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Computer-Human Interaction, vol. 26, issue 1, 22 January 2019,; http://doi.acm.org/10.1145/3299096
• Contract Date: Nov 27, 2018