How many testers are needed to assure the usability of medical devices?

Borsci, Simone; Macredie, Robert D.; Martin, Jennifer L.; Young, Terry

Abstract

Before releasing a product, manufacturers have to follow a regulatory framework and meet standards, producing reliable evidence that the device presents low levels of risk in use. There is, though, a gap between the needs of the manufacturers to conduct usability testing while managing their costs, and the requirements of authorities for representative evaluation data. A key issue here is the number of users that should complete this evaluation to provide confidence in a product's safety. This paper reviews the US FDA's indication that a sample composed of 15 participants per major group (or a minimum of 25 users) should be enough to identify 90-97% of the usability problems and argues that a more nuanced approach to determining sample size (which would also fit well with the FDA's own concerns) would be beneficial. The paper will show that there is no a priori cohort size that can guarantee a reliable assessment, a point stressed by the FDA in the appendices to its guidance, but that manufacturers can terminate the assessment when appropriate by using a specific approach - illustrated in this paper through a case study - called the 'Grounded Procedure'.

• Journal Article Type: Article
• Publication Date: Sep 1, 2014
• Deposit Date: Jul 30, 2015
• Publicly Available Date: Jul 30, 2015
• Journal: Expert Review of Medical Devices
• Print ISSN: 1743-4440
• Electronic ISSN: 1743-4440
• Publisher: Taylor & Francis Open
• Peer Reviewed: Peer Reviewed
• Volume: 11
• Issue: 5
• DOI: https://doi.org/10.1586/17434440.2014.940312
• Keywords: Evaluation cohort, Five-user assumption, Medical devices, product safety, Usability testing
• Public URL: https://nottingham-repository.worktribe.com/output/994436
• Publisher URL: http://informahealthcare.com/doi/abs/10.1586/17434440.2014.940312