Capturing user requirements in medical device development: the role of ergonomics

Measuring and fulfilling user requirements during medical device development will result in successful products that improve patient safety, improve device effectiveness and reduce product recalls and modifications. Medical device users are an extremely heterogeneous group and for any one device the users may include patients and their carers as well as various healthcare professionals. There are a number of factors that make capturing user requirements for medical device development challenging including the ethical and research governance involved with studying users as well as the inevitable time and financial constraints. Most ergonomics research methods have been developed in response to such practical constraints and a number of these have potential for medical device development. Some are suitable for specific points in the device cycle such as contextual inquiry and ethnography. Others, such as usability tests and focus groups, may be used throughout development. When designing user research there are a number of factors that may affect the quality of data collected, including the sample of users studied, the use of proxies instead of real end-users and the context in which the research is performed. As different methods are effective in identifying different types of data, ideally more than one method should be used at each point in development. However, financial and time factors may often constrain this.


Introduction
Developers of medical devices are increasingly subject to demands that they incorporate assessment of user requirements into their development processes. Such demands come, on the one hand, from agencies funding medical device research, many of whom now require some evidence in grant applications that user needs have been or will be addressed. On the other hand, there is an increasing recognition that poor usability increases the risks associated with medical devices. In response to this, and the new emphasis on patient safety by both governmental and non-governmental organizations, developers face more stringent requirements for usability testing.
Other areas of engineering, in particular those applied to many safety-critical industries, have incorporated user requirement research into their development processes for some time. Healthcare engineers have been slower to act. This is a particular concern given the remoteness of medical device developers from device users. Most such devices arise from either the desire to fulfil an unmet need in healthcare diagnosis or treatment, or because a scientific/technological advance offers an improved solution to a known problem. These are, of course, not mutually exclusive. Typically, pioneers of new devices are scientists or engineers with little experience as device users. Unfortunately, the result may be end-products which are sub-optimal for the application for which they have been developed.
Given these potential benefits of incorporating user requirements into the device development cycle, why has the healthcare device industry lagged behind other safety-critical industries in this respect? In the absence of hard evidence, we can only speculate. Possible reasons include difficulties in accessing patients and the need to obtain ethical approval (obviously for very good reasons, but nevertheless a lengthy and therefore expensive process), concerns about the vulnerability and/or communicative abilities of some patient groups, and fear that the time and cost involved will not be matched by the benefits gained (a prevailing, but not always accurate, view is that if a device is beneficial in terms of health outcomes then users will be willing to accept some discomfort during its application). Finally, developers may not think of medical devices as consumer items, given the distance between themselves and users that is created by healthcare procurement processes. As a result, issues such as aesthetics are often seen as either unimportant or unrelated to the device's commercial success. This paper has two aims. First, it examines the potential benefits, in terms of improved patient safety and device effectiveness, reduced need for product recalls and modification and greater commercial success, of incorporating 'user needs' at all stages of the design process. In this context, 'users' may include not only healthcare professionals, but also the patients who may use the device themselves or receive treatment using the device and, in some cases, carers who are relatives and friends. Second, it analyses existing methods for assessing user needs, drawing upon a recently completed review of the literature on methods for assessing user requirements in engineering and ergonomics. The relevance of these methods for the healthcare industry is examined and, where available, examples of their use in relation to medical devices are discussed. The appropriateness of different methods for particular stages of the design cycle and their relative costs in terms of both time and money are examined. Given the potential difficulty of accessing 'real' end-users, the use of proxies is also discussed.

Why study user requirements?
Over the last decade, there has been an increasing interest in addressing the requirements of users during medical device development, particularly to enhance patient safety. This topic was recently reviewed in this journal, with specific reference to patient monitoring, by Walsh Improve safety of devices Improved usability of devices Reduce device recalls Limit the need for ad hoc modifications Improve efficiency of users Improve patient outcomes and satisfaction Assist with obtaining development grants and Beatty (2002) who categorized the healthcare sector as a cognitively complex environment, similar to the nuclear and aviation industries. The authors highlight the importance of good design of medical devices and the role that device developers can play in improving patient safety, stating that 'it is time that patient monitor designers reasserted that they have their contribution to make in improving the safety of the monitored patient' (Walsh and Beatty 2002, p R129).
Although designing for patient safety and reduced human error is extremely important, satisfying user requirements in medical devices should also incorporate aspects of usability such as comfort, effectiveness, ease of both use and learning, training, hygiene requirements, maintainability and servicing, storage, labelling and so on. Attention to comfort, aesthetics and portability can affect patient readiness to follow a treatment regime, particularly for selfadministered devices. The potential benefits of incorporating user requirement research in the design process are summarized in figure 1 and discussed below.
Medical device users are extremely heterogeneous. For some devices, such as ventilators and surgical devices, the users will be exclusively clinicians. For others, such as asthma inhalers, the primary users will be the patients. In many cases, there will be numerous categories of users. Even when patients do not operate devices themselves, they will receive the diagnosis, treatment or other process and may contribute to assessing its value. This need to satisfy both operator and recipient differentiates medical devices from other ergonomics domains such as the design of work systems or consumer products. Merely addressing the clinical needs of the users and the safety needs of the system will not guarantee the success or prevent dissatisfaction or ad hoc modification of the device. Additional factors must be considered during design to ensure a good fit with user requirements. Devices must fit into the working and living patterns of the users to allow them to be used efficiently and effectively; ideally, users should not have to modify either the device or their working pattern or lifestyle in order for it to be used. The system in which the device will be used must also be considered: devices will be used in conjunction with other equipment by different users for the treatment of numerous conditions and in a variety of settings.
Investment in user requirements research benefits the developer as well as the user and the healthcare sector. The increasing recognition, among healthcare purchasers and regulators, of the links between good design and patient safety, clinical and general efficiency and satisfaction has led to formal requirements for user research by funders of development work on medical devices. For example, the UK Health Technology Devices (HTD) programme, which supports collaborative research projects on new and improved health technology devices, requires companies to involve users in decisions during design and development rather than simply using them as passive participants in research. Demonstrating both the need for and usability of products is important when seeking external funding, for example from venture capitalists.
Capturing user requirements for medical devices differs from general product development in a number of significant ways. Numerous barriers, including ethical and research governance procedures and the multiplicity of user groups and stakeholders, make user involvement particularly challenging. Medical device developers are under pressure to minimize cost and time to the launch of the product. When faced with the need to establish proof of concept and perform any necessary clinical trials, developers may not prioritize rigorous, structured attempts to capture the user requirements for the device. It is here that ergonomics research methods, which have been developed in response to the practical constraints of product development, are particularly appropriate.
To date, ergonomists have done little to tailor their methods and approaches to the needs of medical device developers. Many companies lack the resources or the will to pay for ergonomics expertise. Developers need practical advice about maximizing the benefits from time and money devoted to capturing user requirements. This is what the MATCH project and this paper aim to provide. The Multidisciplinary Assessment of Technology Centre for Healthcare (MATCH) is a UK collaboration between five universities funded by the Engineering and Physical Sciences Research Council (EPSRC), Department of Trade and Industry (DTI) and the National Patient Safety Agency (NPSA). MATCH aims to support the healthcare sector by creating methods to assess the value of medical devices from concept through to mature product. Although the MATCH research is being performed within an academic framework, the emphasis is on working with industrial collaborators to solve real problems.

Methods
An iterative and flexible search strategy was used to search ten online bibliographic databases covering engineering and ergonomic journals and trade publications. A list of keywords was compiled under the headings of 'methods', 'devices', 'user requirements' and 'users' including possible permutations of their names and American English spellings where appropriate. The list was added to throughout the review as more methods and tools were identified. None of the databases used an indexing method such as the medical subject headings (MeSH) used by the National Library of Medicine and each database required different searching methods. Boolean operators, quotation marks, wildcards and truncations symbols were used when appropriate.

Results
Most methods identified in the review were concerned with user-centred design (UCD), an ergonomic approach which focuses on users throughout planning, design and evaluation. It is recommended that user-centred design (UCD) should begin at the earliest stage possible, ideally at concept, and continue through an iterative design and evaluation process (figure 5). UCD uses a variety of research methods depending upon factors such as the stage of development the product is at, the type of users to be studied and the type of data required. The methods judged to have the greatest potential in medical device development are presented below, with a discussion of the type of data they generate and an assessment of their possible

Long-term observation of an environment
Focus is on learning without pre-conceived ideas or questions Provides qualitative, descriptive data Suited to identifying unmet or ill-met needs Very expensive and time-consuming application within medical device development. This is followed by a discussion of factors and issues that affect the type and quality of data collected.

Exploratory/scoping methods.
Ethnography. Identifying unmet or poorly met clinical needs is often the starting point for developing a successful new device. Qualitative methods, which are typically exploratory and open ended, are particularly suited to this task. Ethnography, with its emphasis upon discovery, is particularly useful in this regard (figure 2). The ethnographic researcher spends an extended period (often months or even years) studying users within their work and/or home environment, observing the behaviour of users, interactions between users and between users and devices. Recordings of verbal and non-verbal behaviour are supplemented with interviews and analysis of documentation. Such ethnographies can identify aspects of the work environment which are not obvious to users themselves. As D'Souza and Greenstein (2003, p 263) state 'often, this results in discovering latent needs-those needs of which a user is not yet aware, that when met, bring a high level of satisfaction'.
Sommerville et al 1994 used ethnography to study an air traffic control centre with a view to designing new computer systems. Researchers spent several months observing controllers' behaviour, interactions and task performance. They also read and analysed training and operational manuals and interviewed controllers to obtain further insights into the working environment. They gained detailed knowledge of the environment which allowed them to develop an appropriate computer system. For example, a number of repetitive tasks were identified and it was initially thought that these could be automated. However, analysis of interview and documentary data showed that these tasks incorporated important safety checks which would have been sacrificed if the tasks had been automated.
Ethnography could be used to identify unmet or poorly met needs in clinical contexts as observers note deficiencies in existing systems or opportunities for new devices. Studying devices in their context of use provides valuable information on device operation within the healthcare system, by real users and alongside other devices as well as information on clinical performance.
A number of examples of the use of ethnography in product design were identified, many of which were concerned with the development of complex computer systems (Beynon-Davies 1997, Simonsen and Kensing 1997, Viller and Sommerville 1999, Wales et al 2002, D'Souza and Greenstein 2003. The authors of these studies report ethnography to be effective for obtaining insights into the ways technology functions within organizations, and the cultures and sub-cultures they comprise, as well as individual work patterns.
Ethnography provides detailed and useful insights into the environment in which context of device use. However, it is extremely expensive and time consuming, therefore unlikely to  be practical for use by most product developers, at least in its purest form. No examples of ethnography being used for a medical application were identified in the literature. A cognate ergonomics method, contextual inquiry, has been developed which takes into consideration the practical constraints under which developers have to work but which provides the necessary awareness of social and organization issues for a smaller time and financial investment.
Contextual inquiry. Contextual inquiry (CI), also known as shortened ethnography, was developed by Holtzblatt and Beyer (1993) specifically to counter address the difficulties of applying traditional ethnography, with its heavy demands on time and resources, to product development. They also point to difficulties in translating the outputs from ethnography into well-defined user requirements, to inform the design process. CI consists of short, targeted observations and interviews that, to some extent, may be influenced by pre-determined research questions (figure 3).
Contextual inquiry involves the observer/interviewer shadowing workers as they conduct their work. The observer may ask questions about what is happening, why it is happening and how tasks could be improved by any device or system. The designer and the user work together to discover important taken-for-granted information that the user may not recognize as relevant or significant. By observing the researcher may identify areas that are deficient; for example, a user may not be performing a task as efficiently or safely as possible and there may be a number of different reasons for this. By asking the user questions, these reasons can be identified and solutions can be found that may include a new or improved device.
Most examples of CI identified from the literature addressed computer systems design; however, this method has also been successfully applied in medical contexts. Coble et al (1997) used CI during the development of a new clinical workstation software program. They completed 300 h of observations and 1300 h of data analysis over 3 months. Ten physicians were observed in a variety of clinical settings: inpatient, outpatient, primary care and general surgical activities. Following the observations, the researchers generated over 500 requirements, which were then reviewed and prioritized in collaboration with the physicians. This formed the basis of the user requirements document, which was subsequently reviewed by the users themselves.
The authors found CI to be a useful tool which captured an extensive list of user requirements for the product. However, they report that it was still challenging to translate these requirements into a workable functional specification and that further consultation with users is necessary throughout development to ensure that requirements are not lost or misunderstood as they are incorporated into design. Doherty et al (2001) used CI during development of the Cyberlink device, a brain interface that assists motor-impaired people with communication.
Group discussion with 8-10 users

Provides the opinion of a group not a number of individual opinions
Can be used at many points throughout development: To identify unmet or ill-met need To obtain specific ideas for a new device To obtain user opinions on a prototype device Requires a moderator to guide discussion Ideally will be supplemented by observational data The authors of this study found the method to be effective at identifying user requirements early in the design process.
Contextual inquiry is particularly useful for identifying areas of clear unmet need or where current devices are obviously deficient. Providing designers with information about ways in which current devices fail to meet user requirements could play a vital role in design improvement. The exploratory nature of this technique means that it is relatively expensive and time consuming to perform, which may prove to be a barrier for smaller enterprises. However, it is clear that some investigation into the context(s) in which the device will be used is important during device development.
Focus groups. Focus groups are used in a wide variety of industries and their popularity reflects the relatively low cost and time commitment involved. Often used within UCD, focus groups may be employed at a number of stages during product development: to identify unmet needs at the concept stage, to investigate the features and characteristics required by users and their relative importance, and to obtain feedback on prototype designs as part of usability tests (Gould andLewis 1985, Mello 2001). It has also been suggested that focus groups allow deeper issues to be investigated such as emotional bonding of users with products and cultural perceptions of products (McDonagh et al 2002). Studying groups of workers may also generate data on organizational issues, conflicts and tensions (figure 4). Batavia and Hammer (1990) used focus groups with wheelchair users (the intended user group) to specify requirements for a new assistive device and reported this to be effective at identifying requirements which were not currently met by existing devices. Garmer et al (2004) compared the use of focus groups with usability tests (see below) for detecting problems with an existing infusion pump to inform the design of a new version. Participants in each study were intensive care nurses, the users of the pump. Focus groups were effective at identifying contextual problems with the device and identified a number of issues that the participants reported would not have been raised in individual discussions, whilst the usability tests were effective at detecting problems that the user was not necessarily aware of whilst using the device. The authors conclude that neither of these methods would detect the full range of requirements on its own and that, ideally, a combination of methods should be used in device development.
Focus groups will not always be the most appropriate method. Some groups of users, such as children or those with mental impairments, may find individual interviews more conducive, and in some cases patients may be uncomfortable discussing aspects of their medical condition in a group setting. Data generated through methods such as focus groups will be restricted to what the participants are aware of and what they can recall and articulate. However, in many cases users may not note the deficiencies in the devices that they use, or may not be able to identify what improvements could be made to the treatment of their condition or their work pattern. As Holtzblatt and Beyer (1993, p 93) state, 'people are adaptable and resourceful creatures-they invent many workarounds and quick fixes to problems and then forget they invented the workaround. Even the details of everyday work become second nature and invisible'. Ideally, focus group discussions should be complemented by some type of observation of the situation or device under investigation in order to access information and contextual factors about the use of the device that users may not be able or willing to articulate.

Evaluative methods.
Methods for evaluating existing devices or new prototype devices may be applied at various points during device development to identify problems, to suggest possibilities for redesign or to establish the improvement of a new design over an existing product.
Usability tests. Usability tests are performed by asking either real users, or proxies, to perform typical tasks using the device under investigation. Usability is defined in ISO 9241-11 (1994) as 'the extent to which a product can be used by specific users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use'. Notably, this does not include safety which has been the primary focus of most work on improving usability of medical devices. A consequence of this is that the other important aspects of usability including efficiency, effectiveness and satisfaction may be overlooked during device development.
For any usability test, it is critical that appropriate tasks are selected to allow valid testing of the device. The context of the test also needs to be considered; if the usability test is performed outside the normal working environment, then this will need to be taken into account in interpreting the results. During usability tests, various data collection methods can be used to collect both qualitative and quantitative data. When testing an existing or prototype device to identify areas for improvement for example, qualitative data may be of most use. The developers may observe users perform the test to identify shortcomings or areas for improvement and/or asking the user to report their experiences in follow-up interviews or through a method such as the think-aloud method where the user verbalizes their thoughts and actions as they complete the task (Vaughan and Schwartz 1999). In its simplest form, a usability test may just involve users performing a number of typical tasks and then reporting their experiences of using the device, i.e. what, in their opinion, worked well and what was problematic. It cannot be assumed, however, that users will be able to detect and communicate all the problems that may arise in the use of a device by themselves. Some problems may be more noticeable to observers who are not preoccupied with meeting the demands of the task at hand.
Sometimes quantitative data are collected during usability tests, such as the time taken to complete a task using the device or the number of errors made whilst performing the task. These types of data are useful when comparing two devices, for example a prototype with an existing device, to demonstrate improvement. Quantitative data are also useful when communicating the results of usability tests to other stakeholders and decision makers. Results such as time to complete task or errors made during task are easy to understand and compare ( figure 6).
Due to the current focus on improving usability and reducing human error within the healthcare industry, usability testing is beginning to be used during medical device development. An example of this is the study reported above by Garmer et al (2004) who used usability testing along with focus groups in the design of a ventilator. The usability tests consisted of a number of different scenarios likely to be experienced whilst working in an intensive care unit. Outcome measures were both objective and subjective: errors during operation were recorded by observers and the users completed a questionnaire following the tests. The authors report that the usability tests were especially useful for identifying problems that the users were not aware of, but less effective at identifying contextual issues. A number of papers were identified that recommend using usability tests as part of an iterative design approach (figure 1); essentially, a number of design and evaluation cycles are performed until the goals of the device are met (Salvemini 1999, Vaughan and Schwartz 1999, Garmer et al 2002a, 2002b.
Used during prototype evaluation to identify usability problems with a device Often performed by members of design team 'thinking as users' Depends upon ability of evaluators to think and act like users May miss significant contextual issues Can be performed in-house to identify large problems before more extensive usability testing with real users Cognitive walkthrough. A technique that attempts to measure usability of a device without including true end-users is the cognitive walkthrough (CW), a technique designed to be performed as part of an iterative design and evaluation cycle. This method is performed by expert evaluators, usually members of the development team or usability specialists, rather than actual end-user. In CW the focus is on learning through exploration, the evaluator specifies the sequence of actions required to perform a certain task and then steps through that sequence to identify potential usability problems. The main focus is often upon how easy a system would be to learn for a novice user (figure 7).
Two papers reported the use of CW for a medical application. Patel and Kaufman (1998) performed a CW during the development of new medical informatics software as part of an iterative design procedure. Kaufman et al (2003) used cognitive walkthrough, in conjunction with traditional usability testing involving end-users, for the development of computer-based healthcare systems for use in patients' homes. The CW was performed by members of the design team to determine the cognitive demand that the system would place on a predominately elderly target population. Kaufman et al (2003, p 47) state that 'elderly users of a system are likely to have a lower tolerance for excessive memory or attentional demands' and as a result made a number of changes to the system to accommodate this. The authors of this paper were sensitive to the special requirements of their potential users stating that the method is predicated on a sound understanding of the target population. They were also aware of the difficulty of identifying all the problems that may be encountered by the user and for this reason also performed usability tests with true end-users and found that these tests identified a number of additional problems not anticipated by the developers during CW.
The main disadvantage of the cognitive walkthrough is that it is difficult for designers to truly think and act like users, especially novice users. In addition, some authors report that, as a consequence of evaluators' lack of domain and contextual knowledge, some of their suggestions may be inappropriate or make other tasks difficult or impossible (Wharton et al 1992). However, cognitive walkthrough is a low investment technique ideally suited to the early identification of design problems; relatively cheap and quick to perform it can be used to refine preliminary designs before usability testing with real end-users is performed on a later prototype.

Factors influencing user requirements research.
The use of more than one method. Many authors recommend using more than one method to capture user requirements during the development of a device (ISO 13407 1999, Lin et al 1998, Salvemini 1999, Garmer et al 2002a, 2002b. For example, a scoping method such as contextual inquiry or a focus group can be used to specify the needs and requirements of the users with an evaluative method such as a usability test then used at a later stage to determine whether these have been met. As different methods are effective at identifying different types of data (Garmer et al 2004) in an ideal scenario more than one method should be used at each point in device development; however, financial and time constraints may often prevent this. Table 1 presents a comparison of the methods discussed in this paper in terms of the stages in product development when they can be used, the time, cost and expertise required and the type of data that they generate.
Context. The contexts in which the medical device will be used must be addressed during development. Devices may be used by a number of different types of users, within different environments for the treatment of different conditions and will have to be used in conjunction with other devices and equipment and an appreciation of these factors is essential during development. There are also wider, organizational issues such as working procedures, team work, communication, safety culture and management attitudes that influence the effectiveness, usability and safety of a device. This does not mean that all user research has to be performed in the field, assessments performed in isolation may be entirely appropriate at certain stages of development, particularly for proof of concept work. However, this should always be supplemented by field tests where the compatibility of the device with other equipment and the environmental effects on device performance can be verified.
Sampling and the use of proxies. The ability to access appropriate participants will affect the methods open to developers. Time, financial, ethical and practical factors as well as commercial confidentiality may make it difficult or impossible for real end-users to be included in certain phases of product development. Including appropriate participants is an issue that can be especially problematic for medical device developers: accessing end-users within the healthcare sector is a difficult, time-consuming process due to the research governance procedures involved.
Due to the difficulty and cost involved in accessing and including 'real' end-users such as clinicians, patients and carers in their user requirement studies, developers may consider using proxies in their place, especially early on in the design process. For example, healthy participants may be used to test a device for usability in the place of patients or clinicians may be asked to provide the opinions or views of their patients for example, 'what factors do your patients say are important to them?' or 'do you think your patients would find this acceptable?' The developers themselves, or other personnel within a company, can also act as proxies for the end-user. This may be especially useful for evaluating early prototypes for major and more obvious usability problems. Using an easily accessible sample means that several design and evaluation cycles can be performed quickly and easily, making it possible to identify and fix major problems with the device early in development. If large problems have been solved beforehand, a more refined prototype can then be used for tests with real end-users later on in the development. This type of evaluation also has the benefit of being performed in-house and without the risk to company confidentiality that can be associated with methods that involve participants from outside the company. Testing products on the development team has been a traditional route to addressing user issues. However, while quick and efficient, the results can be fraught with bias and should never completely replace testing with representatives of the target users.
The use of proxies may also be appropriate for capturing the requirements of users who may have difficulty expressing their own opinions: for example some children or those with severe mental impairments. In these cases, it is critical that the appropriate proxy is chosen to represent the user.
There are a number of additional issues that will affect the methods chosen by developers to obtain user requirements. The intrusiveness of some methods discussed in this paper may mean that they are unsuitable for use within particular medical settings or may have to modified: for example when using contextual inquiry it may be inappropriate for the researcher to ask questions of the users whilst they are performing tasks or during patient consultations or certain surgical procedures. In such cases, questions and discussion may need to be conducted after the task has finished. It should always be borne in mind that any observation or intervention by evaluators will affect a change in the behaviour of those being observed and it is essential to take this into account in interpreting the results of such enquiries.
Cultural and social factors of either particular workplaces or wider communities may mean that some methods are less suitable than others. This should be addressed when considering methods that involve group activity or discussion. For example, the presence of managers or supervisors may influence the behaviour and opinions of other workers and the opinions of patients may vary according to whether doctors are present. Similar issues may arise in certain countries where, for example, culture may dictate that workers will agree with the most senior person present or women agree with men, and it may be difficult to predict the extent of these factors. In such cases, methods that involve one-to-one interactions may be more suitable.

Conclusion
Few papers provided any general recommendations on the factors that should be considered by developers when choosing methods to use within product development. There is a need for an academia and industry-focused guide to provide general advice and recommendations for developers and to help them quantify the relative costs and benefits of different methods.
Initiatives that aim to provide support for product/device/system developers face a common problem: tools for use in product development need to be contextualized, methods invariably need to be adapted for individual cases and specific user issues require specific methods. However, in order to be useful for prospective development advice on methods and tools has to be generic. The balance between generalizability and applicability is often difficult to achieve and we will be looking to work with industry to try to overcome this barrier.