Assistive technologies for severe and profound hearing loss: Beyond hearing aids and implants

ABSTRACT Assistive technologies (ATs) offer capabilities that were previously inaccessible to individuals with severe and profound hearing loss who have no or limited access to hearing aids and implants. This literature review aims to explore existing ATs and identify what still needs to be done. It is found that there is a lack of focus on the overall objectives of ATs. In addition, several other issues are identified, i.e. only a very small number of ATs developed within a research context have led to commercial devices, and there is a predisposition to use the latest expensive technologies and a tendency to avoid designing products universally. Finally, the further development of plug-ins that translate the text content of a website to various sign languages is needed to make information on the internet more accessible.


Introduction
According to the World Health Organisation, almost 5.3% of the world's population have a hearing loss greater than 40 dB. Specifically within the United Kingdom, there are more than 11 million people that have some form of hearing impairment. Almost 900,000 of these people are severely or profoundly deaf (RNID, 2017;WHO, 2017). Unfortunately, not all individuals with severe and profound hearing loss can either be helped by or have access to hearing aids or implants (e.g. cochlear, bone conduction). Sahin, Sagers and Stankovic (2017) reported that, by the end of 2012, only 324,000 people worldwide had received implants. LANCET (2016) also reported that the current production of hearing aids only meets less than 10% of the global need. Thus, providing additional assistive technologies (ATs) for individuals with severe and profound hearing loss, especially those who have no or limited access to hearing aid or implants, will likely offer additional benefits and a potential to improve quality of life, which has been reported to be poor (Fellinger, Holzinger, Gerich, & Goldberg, 2010). Having access to these technologies or devices will likely allow these individuals to do something that they could not do before or to do it more easily and independently, which may then lead to better general functioning in daily life.
Provision of ATs will also enable individuals with severe and profound hearing loss to use existing technologies more effectively. Unfortunately, the needs and characteristics of people with severe and profound hearing loss are often being overlooked in the design of various existing systems, applications or any other technologies. A good example of this is the UK government's websites which rely entirely on telephone support, without offering an additional option through their website for deaf and hard of hearing individuals to send their enquiry by alternative means, e.g. an email address or a contact form. The same issue was also encountered by the deaf and hard of hearing community with bank services (RNID, 2012). Furthermore, these websites do not consider the difficulties that deaf individuals may face in comprehending and reading their content. Several studies have shown that the reading performance and comprehension of deaf individuals is poor compared to that of hearing ones (Dolnick, 1993;Kyle & Cain, 2015;Marschark & Harris, 1996;Wauters, 2005). This is especially common for prelingual deaf individualsthose who were born deaf or became deaf before learning any languagewho find it hard to read a text which is written in a spoken language. Therefore, written material is often less accessible to deaf and hard of hearing individuals than information presented in sign language (SL). Specifically, Marschark and Harris (1996) support the view that the learning and writing progress of people with profound hearing loss is extremely slow. The view is further reinforced by a recent study which proved through reading comprehension testing that reading performance of deaf pupils aged 7-20 was equal to that of 7-year-old hearing children (Wauters, 2005).
The role of ATs, which will aid in the integration of deaf and hard of hearing individuals in wider society, is crucial. Through ATs, users can perform a variety of actions to achieve their particular goals. Over the last 18 years, various systems or devices specifically designed for deaf and hard of hearing individuals have emerged, e.g. devices that allow them to communicate with hearing communities who are not conversant with SLs, enjoy music and be aware of environmental sound. Unfortunately, existing reviews of ATs for deaf individuals are focused on either specific areas or technologies (e.g. Fajardo, Vigo, & Salmerón, 2009;Sorgini, Caliò, Carrozza, & Oddo, 2018;Suharjito, Anderson, Wiryana, Ariesta, & Kusuma, 2017). Consequently, it is difficult to acquire a holistic view of the current state of research related to ATs for deaf individuals and identify research areas that need to be addressed to bridge the gap between deaf and hard of hearing and hearing communities. This study aimed to fill this gap by performing a comprehensive review of the plethora of ATs which have been developed and researched to date. Since hearing aids and cochlear implants have been well researched and reported, this study is focused on ATs other than hearing aids and cochlear implants. Thus, from this point onward, the term "ATs" refers to any ATs or devices other than hearing aids and cochlear implants.
The present literature review will answer the following three research questions: (1) What types of ATs have been developed through research? (2) How are these ATs evaluated? (3) What are the results of the ATs' evaluation? It should also be noted that this study does not attempt to provide an answer on "whether or not ATs for individuals with severe and profound hearing loss are truly effective". The research in this area is still in infancy and, as such, the authors deem that, in contrast to for instance hearing aids and implants, more established research is needed to be able to answer this question.

Methodology
The literature search was conducted by utilising the Nottingham University search engine which sought journal and conference publications from various collections (including Scopus, IEEE Conference Publications, MEDLINE/ PubMed Citation, Web of Sciences, etc.). The search engine also automatically grouped the search results under various topics. The key words used for the literature search were "assistive & device & deaf". Only articles that were published between 2000 and 2017, written in English and available in full text were included. Articles that were on irrelevant topics, as identified by Nottingham University search engine (e.g. Cochlear Implants, Cochlear Implantation and Anatomy & Physiology), were excluded. The abstracts of 344 articles were then examined. Only articles that explicitly reported development of ATs for deaf individuals were included. A total of 53 articles (50 ATs) were identified and included in the review. For each article, the followings three aspects were identified: (1) the purpose of the reported device, (2) the device evaluation methodology and (3) the outcomes of the evaluation. The articles were then grouped into different categories based on the purposes of the ATs. Table 1 (appendix) shows the AT categories and a summary of the included articles/studies.
For each AT category, commonalities related to the followings factors were identified: (1) the type of evaluation conductedtechnical and/or user-centred, (2) dependent variables (parameters) used for evaluation, (3) type and number of users involved in the evaluation (if applicable). Technical-centred evaluation refers to an evaluation that is based on pre-defined technical parameters which are associated with ATs performance; while user-centred evaluation is defined as an evaluation that involves participants and is aimed to obtain subjective and/or objective variables which are associated with ATs performance.

Types of identified ATs
ATs for communication with hearing community Various ATs have been designed to aid the communication between individuals with severe and profound hearing loss and the hearing community. There are three main types of ATs within this category, i.e. translation of speech to text, translation of speech or text to SL, translation of SL to speech or text. Individuals with severe and profound hearing loss mainly use lip reading and SL to communicate, i.e. using the visual channel to understand their interlocutors (Bahan, 2004;Padden & Humphries, 1998;Lane, 2015). In the UK, over 87,000 people use SL as their preferred method of communication (British Deaf Association, 2018). There are more than 200 SLs worldwide (Harrington, 2018).

Translation of speech to text
Mirzaei, Ghorshi and Mortazavi (2012) developed a device using augmented reality technology which enables a deaf individual to see the facial expression and speech of a narrator. The device translates the narrator's speech into text and imposes the text on the real-life video capture of the narrator. However, it was reported that the processing time took up to 10 s which indicates its unsuitability for real-time communication. Lee, Kang, Han and Ko (2016) proposed a portable device that performs real-time speech-to-text transcription with keyword spotting functionality. The performance of the device seems to be promising as it yielded a low rate of word error (20%). Unfortunately, neither the processing time nor the effectiveness of keyword spotting was not evaluated. Meanwhile, Kheir and Way (2007) designed an automatic transcription system that translates lectures and presents the transcription on a simple web page. The system was evaluated in a real lecture setting by students with severe and profound hearing loss individuals. The system was accurate 85% of time and deemed to be of enormous benefit by the students.
Translation of speech/text to SL Virtual avatars are usually used to deliver the translation of speech/text to SL. Unfortunately, to date, facial expressions during the presentation of SL, which are particularly important for SLs since they often affect the meaning of signs (Elliot & Jacobs, 2013;Liddell, 1983;Woll, 2001), have not been included in any existing ATs. Cox et al. (2002) demonstrated a system called "TESSA", which is used in post offices to translate the speech of the post office worker to SL. It consists of three sub-systems: a headset microphone, a display that presents a list of possible phrases to the post office worker, and a second display that is located in front of the deaf person and shows a virtual avatar interpreting the speech of the post office worker in British Sign Language. Observations of three different post offices' transactions were conducted in order for the researchers to collect More than 80% of user preferred to have the transparent sign language video and the concept was well accepted. 5 Li et al. (2014) The development of a system to translate text into virtual human animation that takes the effect of context on manual gesture and nonmanual gesture (facial expression, etc.) Not reported Not reported 6 Cox et al. (2002) A system that aids transactions between a deaf person and a clerk in a post office by translating the clerk's speech to sign language The evaluation involved deaf users (n = 6) and post office clerks (n = 3) and the following was evaluated: sign language, intelligibility and acceptability, time needed to complete transaction and acceptability of the system.
The intelligibility is about 61% and 20% of them as acceptable, transaction took much longer with TESSA and the acceptability of using TESSA is lower than without TESA 7 Kennaway et al.
A system (eSign) to translate text to signing gesture using an avatar-independent scripting notation aimed for website/web pages Assessment of the level of comprehension of the synthetic signing developed in eSign (single sign, signed sentences and text chunks) with deaf participants (n > 15) 70-75% comprehension rate for single sign, 40% for sentences and text chunks for initial version of the system and 90% and between 58-71%, respectively, for an improved system. 8 Rekha and Latha (2014) A mobile-based sign language translation device for automatic translation of Indian English speech language to sign language transcripts of a natural dialogue between the workers and the customers. A set of 115 phrases was used in the initial system and after the completion of the user trials, almost 255 phrases were added in the system. In order to capture signing and develop the virtual avatar, various sensors were on a person to record the motion of hands, upper body, arms and facial expressions (Cox et al., 2002). One limitation of the "TESSA" system is that it is contingent on a number of phrases; therefore, it cannot be applied to any other contexts apart from the post offices. Moreover, it cannot be as effective as it could be as the translation was not independent of certain phrases and unable to recognise natural language (Elliott, Glauert, Kennaway, Marshall, & Safar, 2008). This was reflected in the results of the evaluations conducted concerning the effectiveness of the system. It was observed that the transaction took longer to complete with the "TESSA" system than without. Moreover, two out of six participants found the transaction with the "TESSA" very difficult due to confusion caused by the inappropriate use of signs (Cox et al., 2002). In addition, the system eliminated any social interaction between the post office worker and the deaf person, as both of them looked on screens throughout the transaction.
Another system similar to "TESSA" is "Thetos" which was used in medical settings. "Thetos" translates a text of the Polish spoken language to SL and presents that on a screen through an animating virtual character (Suszczanska et al., 2007). The difference between this system and "TESSA" is that it deals with natural language and not "ready-made"pre-storedphrases. A question typed into the system will be analysed to decide appropriate animation to convey the same message in Polish SL. However, as a result of using animated virtual characters to deliver Polish SL, the system faces difficulties in introducing lexical and non-lexical facial expression into the SL being generated .
In 2013, a team of researchers influenced by the "TESSA" system demonstrated a mobile-based system, so called "SignSupport", which supports the communication between deaf patients and pharmacists (Motlhabi, Glaser, & Tucker, 2013). This application was designed for deaf people who visit a public pharmacy to obtain medicine. It allows the pharmacist to input text which is then translated into SL. The information given by the pharmacist is presented on the mobile screen via video, enabling a deaf individual to get detailed instructions on how to take the dispensed medication. However, it does not support conversation that could possibly arise as it was designed in such a way that it answers most common questions asked by patients before they actually asked the questions. Another drawback of the system is that the use of pre-recorded videos to convey the SL required further verification to ensure their clarity and context relevancy.
Rekha and Latha (2014) demonstrated a mobile interface, which offers an automatic translation of the Indian English spoken language to SL. A cloud database was developed, in which all the captured images of the SL (static and dynamics gesture) were stored. The speech is translated to text and then is presented in SL through a 3D avatar on the screen. While use of cloud database enables real-time translation of text to SL and mobile technology offers ubiquity, it could also present a problem especially in a situation in which a fast internet connection is unavailable.
The poor language literacy skills of deaf people have results in a lot of online information becoming inaccessible. This has prompted translation of text on websites to SL via virtual avatars (El-Gayyar, Ibrahim, & Wahed, 2016;Kennaway, Glauert, & Zwitserlood, 2007;Kouremenos, Fotinea, Efthimiou, & Ntalianis, 2010;Li, Yin, Wang, & Kong, 2014), pictures (Ditcharoen & Cercone, 2010) or transparent SL videos (Debevc, Stjepanovič, & Holzinger, 2014). Kennaway et al. (2007), the only study that involved deaf individuals to evaluate the effectiveness of the translation of text on websites to SL via virtual avatars, developed a plug-in that presents the content of a website in SL through an animated virtual character. By clicking on something such as a paragraph, label, caption and so forth, a synthesised SL animation of that content is made available. Although the comprehension of SL for translating sentences ranged from 58% to 71%, deaf people found the particular tool extremely helpful and they suggested that it would be nice if similar tools were available in train stations, airports, hospitals and other public services, so that they are able to independently access information (Kennaway et al., 2007).

Translation of SL to speech/text
Many researchers have tried to support the interpretation of SL to enable a deaf individual to communicate back to a hearing individual without relying on an interpreter. Some of the existing systems make use of the Kinect 3D depth camera (Agarwal & Thakur, 2013;Lang, Block, & Rojas, 2012;Li, Lothrop, Gill, & Lau, 2011;Trigueiros, Ribeiro, & Reis, 2014). Some other systems use data from various sensors and/or web cameras to recognise particular gestures (Al-Jarrah & Halawani, 2001;Allen, Asselin, & Foulds, 2003;Brashear, Starner, Lukowicz, & Junker, 2003;Paudyal, Banerjee, & Gupta, 2016;Sarji, 2008). Unfortunately, all of the above studies evaluated the performance of their system only on a single or individual signnot sentences and against a very limited number of SL vocabularies. Lang et al. (2012), a study that evaluated the use of Microsoft Kinect to recognise SL vocabularies more than any other studies, demonstrated the "Dragonfly", an open-source framework developed for German Sign Language recognition. Kinect as well as Hidden Markov Models were used for a comprehensive recognition of various signs. Through their study, they proved that without using CyberGlove (Kessler, Walker, & Hodges, 1995) -"a whole hand input device"and only with the use of a 3D depth camera, the detection of various gestures is accomplishable. However, they have not mentioned the context in which this system could be applied and how it could aid deaf people to communicate with hearing people. The numerous disadvantages of Kinect, such as the fact that it tracks humanlike objects located in the background (Zhao, Naguib, & Lee, 2014), were also not discussed in the study. Therefore, these issues were not taken into consideration in the design of the framework. Nevertheless, it can be used for the development of educational games for deaf children.
Another notable interface is "HandTalk" (Sarji, 2008) which consists of low-cost gloves with sensors and mobile phone with Bluetooth and converts basic signs of the American Sign Language (ASL) to speech. Flex sensors are located within the glove to detect the bending and flexing of the fingers and hence different signs of the ASL. It converts the signs into voice by interacting with the mobile phone via Bluetooth. However, while it has the potential to be offered as a low-cost portable device, it could not detect fast alternating gestures and further work was needed to address this issue.

Two-way translations between speech/text and SL
Only one research study addressed two-way translations between speech/text and SL. Escudeiro et al. (2015) developed a PC-based bidirectional translator between Portuguese Sign Language and Portuguese text. The system utilised Microsoft Kinect, Sensor Gloves and a virtual avatar performing SL. However, the study only reported the results of the translation from SL to text as the text to SL had yet to be developed. Furthermore, the translation from SL to text was only evaluated for accuracy with a very limited number of SL vocabularies.
ATs for communication aid among individuals with profound and severe hearing loss Cavender, Ladner and Riskin (2006) sought to develop a mobile-based technology for a low-bandwidth communication between deaf and hearing people through video. Based on their findings, they decided to increase the quality of the video in the face region and decrease it in other regions (Cavender et al., 2006). Despite the limited processing power of mobile phones, they found a way to reduce the complexity of video processing without affecting video quality.

ATs for emergency situation
The ubiquity of smart phones has prompted mobile applications, such as LifeKey (Slyper, Ko, Kim, & Sobek, 2016) and SOS Phone (Paredes, Fonseca, Cabo, Pereira, & Fernandes, 2014), that are designed to support deaf people to report an emergency event quickly without the need to rely on a hearing person. A custom keyboard was developed through which the deaf users can summarise the emergency by tapping through different categories (type of incident and exact location within the area). The users can also set up a custom introduction to state their identity, which is sent along with their location. A user-centred design was implemented and through an iterative process, various prototypes of the interface were developed based on the input of deaf users and people from the emergency staff. Unfortunately, no summative evaluation on the effectiveness of the system was performed and the applications remained as prototypes.
Meanwhile, another study (Fujii et al., 2007) focused on creating a natural disaster alert system for deaf individuals via SMS and visual displays installed at their homes. Unfortunately, similar to the other studies, no summative evaluation on the effectiveness of the system was performed and it was not developed any further.

ATs for perceiving music
How can we aid deaf and hard-of-hearing individuals to perceive music? This question was addressed by several researchers who tried to propose systems and provide solutions to this issue. Nanayakkara, Wyse, Ong and Taylor (2013) have investigated how the use of visual and haptic displays would assist and enhance the musical experience of deaf people. Specifically, a haptic chair and a visual display were developed. On the display, abstract motion graphics representing various attributes of the music were presented, and the haptic chair provided vibration feedback according to the music. Experiments were carried out with 43 participants from whom 15 were profoundly deaf. Fifty-four per cent of the participants preferred the music accompanied by the haptic chair only, whereas the others preferred the music accompanied by both of the systems. Both of the systems were revised according to the results of the experiments and the participants' comments. Human gestures, as well as 3D motion graphics, were added in the visual display and they were compared against the 2D music representations. Participants preferred watching human gestures, as according to their reports they were "more musical". Another study (Araujo et al., 2017) further confirmed, despite the limited number of deaf participants involved in the study, that the use haptic feedback enabled deaf participants to perceive the rhythm and energy that were present in the music.
Other studies were focused on how the visualisation of music can enhance entertainment for deaf individuals (Pouris & Fels, 2012;Zhou, Sim, Tan, & Wang, 2012), and some others on how vibrotactile systems can help deaf people to learn a musical instrument and to explore different sounds (Marshall & Wanderley, 2006;Petry, Illandara, & Nanayakkara, 2016). A recent study conducted by Petry et al. (2016) was about the development and implementation of a portable sensor-based device called MuSS-Bits, which is divided into two parts. The "Sensor-Bit", which records sounds from instruments, laptops and other sources and communicates that to the "Display-Bit", which can be worn or held by users. The "Display-Bit" converts the sounds into vibration and visual feedback. The two systems communicate via WiFi to achieve real-time feedback (Petry et al., 2016). Unfortunately, a thorough evaluation on MuSS-Bits was not conducted.

ATs for alerting and everyday use
The development of different types of alerting devices for individuals with profound and severe hearing loss is one of the most common topics of interest. There are already numerous commercially available devices such as doorbells, smoke detectors (Domingo, 2012), alarm clocks and sound detectors (Mielke, Grünewald, & Brück, 2013). There are two common approaches in assistive alerting devices for everyday use, i.e. wearable, network devices and mobile devices. Ren, Meng and Chen (2006) proposed a system that networks these commercially available ATs so the individuals with profound and severe hearing loss can have sufficient information about their environment at one place. This kind of system, so called the Wireless Assistive Sensor Network, can be applied to different contexts such as homes, public schools, churches, theatres, etc. However, their work was only at an early stage and limited to the evaluation of their proposed concept via computer-based simulation. Ravid and Cairns (2008) implemented an iterative usercentred design approach and proposed a mobile alert device called "Vibe", which receives data from other wall-mounted devices, as well as from the user's environment in order to provide detailed information to the deaf users about the state of their surroundings. One limitation of this particular interface is that, in controlled environments such as a house, people usually leave their mobile phones in one place. Therefore, when there is an alert about an event happening and the users do not hold their mobile phone, they might not realise that it is vibrating. In cases like this, a wearable device could be valuable. Kumari, Goel and Reddy (2015) demonstrated a wireless system that notifies a deaf person about a visitor. It is called "PiCam" and it is consisted of two devices: a transmitter that is placed at the door and a wearable device, which receives information and alerts according to the data collected from the transmitter. The transmitter also interacts with the owner's mobile device by sending text messages when a visitor presses the doorbell. In addition, it captures a photo of the visitor and sends that to the wearable device along with the message that there is a visitor at the door. At the same time, the wireless device vibrates to alert the deaf owner of the visitor. All the information, such as the image of the visitor as well as the date and time of visit are stored in the server for later retrieval.
Most studies were limited to using neither network nor wearable technologies and relied on creating an application that can be installed on and utilised via mobile phones. For instance, Ketabdar and Polzehl (2009) utilised mobile phones to detect changes in audio patterns and notify the user through vibration and visual indications. However, the device does not recognise and differentiate different events, or sounds. Moreover, it does not provide information to the user about the location of the sound source. Bragg, Huynh and Ladner (2016) also created a mobile application that is trainable to detect various sounds from the environment. Furthermore, users can record a sound of interest and use it to train the system to recognise similar sounds in the future. However, it does not offer any indication about the direction of the sound source.
A few studies have also looked into integrating wearable and network devices. Gorman (2014) proposed a device, consisting of eyeglasses on which microphones and LEDs are attached, that focused in localising sound sources as well as directing the user there. On the other hand, Honda and Okamoto (2014) designed a device, attached on users' hair, that converts the sounds into vibration and whereby the louder the sound is, the more intense the vibration would be. Mielke et al. (2013), based on user requirements derived from literature review and individuals with severe and profound hearing loss, designed a system in which smartphone is used to detect and alert environmental sound. The device analyses the acoustic environment and notifies individuals with severe and profound hearing loss when a sound is recognised. From the requirements derived from the literature review conducted, the system had to identify the direction of the sound source and be able to suppress wind noise. Moreover, it had to be trainable in order for the users to be able to register an event not previously recognised by the system. All the occurring events and sounds detected had to be stored in a database, so all users could have the latest and fully updated version of the system. The deaf users are also requested that the device was small in size and had a battery that lasts for a whole day. Mielke and Bruck (2016) improved the designed further by incorporating a smartwatch that detects sounds from the user's environment, so called the "AUDIS wear". It was the first device able to recognise sounds from an outdoor environment. A prototype of the system was evaluated by deaf users and a longer field study is in the authors' future plans for developing the final device.

ATs for environmental sound awareness and localisation
Environmental sounds provide important information about occurring events and the current condition of our surroundings. However, people with severe to profound hearing loss are unable to perceive and localise them. A number of studies have attempted to address this by providing wearable devices that can help deaf people with environmental sound detection and localisation (Daoud, Al-Ashi, Abawi, & Khalifeh, 2015;Gorman, 2014;Kim, Choi, & Kim, 2014;Matsuda, Nakamura, & Sugaya, 2014). Some of the ATs necessitated the use of smart glasses equipped with microphones arrays and light emitting diodes (Gorman, 2014;, a belt that contains microphone arrays and delivers haptic feedback (Daoud et al., 2015), or simply wearable microphones (Matsuda et al., 2014). The evaluation of the devices' technical performance showed that much work is still needed due to the low accuracy and response time of sound localisation.

ATs for education purposes
Hearing deficiency makes learning hard and exhausting (Dolnick, 1993). Some researchers tried to aid the learning of the deaf by developing systems, which produce videos in SL for every learning material. They have also implemented new pedagogic methodologies to engage deaf people and to help them learn without experiencing difficulties (Drigas, Kouremenos, Kouremenos, & Vrettaros, 2005;Drigas, Vrettaros, & Kouremenos, 2004). Another study, in order to help deaf students to remain concentrated in a lecture, merged all of its components in one screen (Cavender, Bigham, & Ladner, 2009). The components included the lecture's slides, videos of the lecturer and the interpreter, and a collaborative notetaking tool. This system is called "ClassInFocus" and its interface can be configured based on the preferences of users.
On the other hand, Ng'ethe, Blake and Glaser (2015) designed a system that aids individuals with severe and profound hearing loss to learn at their own pace without depending on a teacher. They developed a prototype of an authoring tool on mobile devices to learn computer literacy skills. The application uses South African SL videos and images for each lesson. The lessons are structured corresponding to pre-existing teaching methods. Another educational application developed as a learning aid for deaf people is "LAMBERT". "LAMBERT" is a software that helps deaf children to learn a SL by scanning real objects. An RFID (radio frequency identification) tag is attached to each object and if it is scanned, several images of that object (e.g. different colours of an apple) as well as videos of a person and an avatar signing the scanned object are shown on a computer's screen (Parton, Hancock, & Du BusdeValempré, 2010).

The evaluation methodology of identified ATs
The literature review shows that approximately 76% of studies conducted an evaluation of their ATs. It is also apparent that, for most AT categories, the evaluation approaches and variables/parameters to evaluate the performance of them vary greatly. Variations of evaluation approach mostly appear in studies that adopted user-centred evaluation where interactions between users and ATs occurred. Most studies obtained subjective data such as usability issues, user experience and acceptability. Some studies also collected objective data such as time needed by end users to perform a given task. A closer observation on the user-centred evaluation reveals that most studies only involved a limited number of deaf participants. Another recurring theme that emerges with respect to usercentred evaluation is the lack of an evaluation with end users in a real environment setting, which also implies the need to allow prolonged end-user engagement with or exposures to ATs. Consequently, the aforementioned trends present a challenge for studies that aim to compare and/or establish effectiveness of ATs. Interestingly, less variations of evaluation approach are observed on technical-centred evaluation. For instance, the studies in two ATs (translation of SL to speech/ text and environmental sound awareness and localisation) employed more or less similar variables/parameters to evaluate the ATs performance. Details of identified commonalities for each identified AT is shown in Table 2.

The evaluation outcomes of identified ATs
As it has been explained previously, this study did not aim to establish whether or not identified ATs are effective for individuals with severe and profound hearing loss. Even if the authors attempted to do so, we would not be able to draw meaningful conclusions due to the reasons explained in the section "The evaluation methodology of identified ATs". However, the outcomes of the evaluation can be summarised and used as a basis to suggest further steps to realise the potential of identified ATs. Table 3 provides the details of the recommendation for each identified AT.

Discussion
The review shows that there is still a lot of work to be done with regard to ATs for severe and profound hearing loss individuals. Particularly, there appears to be a lack of focus or direction on the overall objectives of ATs to communicate with hearing communities. For instance, much effort has been dedicated to only support one-way communication between the hearing communities and individuals with severe and profound hearing loss. Yet, there seems to be little, if any, evidence that the outcomes of these studies were used as a foundation towards building two-way communication ATs. As a result, ATs that allow real-time two-way communication between individuals with severe and profound hearing loss and hearing communities are still lacking and limited.
One way to provide clear and unified objectives for communication aid technologies is perhaps by performing a largescale user study (e.g. online questionnaire, interviews, focus groups) and/or a small scale but in-depth study (e.g. observation, similar to study performed by Hannukainen and Otto (2006)) to identify the real communication needs of individual with severe and profound hearing loss. For the largescale study, this could be done by involving relevant stakeholders, i.e. not only individuals with severe and profound hearing loss but also their social networks (e.g. family, friends) and relevant government bodies and organisations. • Technical-centred evaluation.
• Recognition/accuracy rate was used as dependent variable for evaluation • ATs were evaluated on very limited vocabularies (not phrases) • Expanding the evaluation to recognise phrases and more vocabularies • Developing towards an ubiquitous translation of SL to speech/text Two way translations between speech/text and SL (n = 1) No commonalities were identified due to the limited number of studies Communication aid among individuals with profound and severe hearing loss (n = 1) No commonalities were identified due to the limited number of studies Emergency situation (n = 3) No commonalities were identified due to the limited evaluation conducted by the studies Perceiving music (n = 6) • User-centred evaluation.
• Different dependent variables were used for user-centred evaluation. • Strong evidence of involvement of real end users • Streamlining dependent variables for usercentred evaluation.
• Different dependent variables were used for user-centred evaluation due to different types of alerting devices • Strong evidence of involvement of real end users • Streamlining dependent variables for usercentred evaluation. • Evaluation with real end users in real environment setting Assistive device for environmental sound awareness and localisation (n = 4) • Mainly technical-centred evaluation.
• Error of direction estimation and/or response times were used as parameter for evaluation • Lack of involvement of real end users • Evaluation with real end users in real environment setting Education (n = 6) • User-centred evaluation.
• Different dependent variables were used for user-centred evaluation. • Strong evidence of involvement of real end users • Streamlining dependent variables for usercentred evaluation are needed. • Evaluation with real end users in real environment setting Table 3. Summary of evaluation outcomes and recommended steps to realise the potential of ATs.
Type of assistive devices Summary of evaluation outcomes Recommended steps to realise the potential of ATs Translation of speech to text (n = 3) The translation accuracy ranged between 80% and 85% in real-time setting.

To develop lightweight (mobile and ubiquitous) ATs
Translation of speech/text to SL (n = 10) Existing ATs had a limited ability to support realtime interaction in various environment settings To reconsider the purpose of the ATs, e.g. instead of supporting realtime interaction, it can be used to increase information accessibility in public places and online Translation of SL to speech/text (n = 9) The accuracy ranged between 87% and 97% in experimental setting. Haptic could potentially help music perception for deaf individuals, more so than visualisation.

To develop lightweight (mobile and ubiquitous) ATs
Alerting and everyday use (n = 7) ATs were still in early development phase and only allowed partial/really limited technical and usercentred evaluation.
To further the development of ATs to enable non-Wizard of Oz evaluation approach Assistive device for environmental sound awareness and localisation (n = 4) Response time was still too slow to support realtime application.
To focus on improving processing time of sound detection and localisation Education (n = 6) There was an early indication of the ATs benefit for deaf students To establish robust and uniform evaluation methodology However, identification of users' and stakeholders' needs alone is not sufficient. The prioritisation of needs and a feasibility assessment are also required. The former will provide the indication of urgency from the point of view of users and stakeholders, while the latter could be used as a guide of what is achievable now and what should be accomplished in the future. For instance, SL recognition presents many technical challenges that need to be resolved in order to achieve two-way communication between individuals with severe and profound hearing loss and hearing communities that are not versed in SL. With the current state-ofthe-art visual computing and computing power, the use of assistive device that enables SL recognition is likely to be limited due to portability issues. Therefore, in the meantime, attempts should be made to provide an alternative approach that is equally acceptable by individuals with severe and profound hearing loss.
Unfortunately, to the extent of the authors' knowledge, apart from an initial study conducted in United States (Maiorana-Basas & Pagliaro, 2014) such user study has not been completed. In addition to provide unified objectives and research direction, the availability of such a study would also help towards ensuring that the ATs developed are usable by individuals with severe and profound hearing loss. Currently, only a few of the current ATs for communication were evaluated with a sufficient sample size of real end users. This means that many usability problems are still uncovered. Additionally, participatory design methodologies are important for the design and development of such ATs. This will also enable individuals with profound and severe hearing loss to be involved throughout the development of the system and the researchers to uncover more specific and unconscious user requirements, motivations and needs.
While there have been examples of ATs that progressed from a research study to commercialisation, the examples are few and far between. This means that only a very small number of ATs developed within a research context reach individuals with severe and profound hearing loss. One reason for this may be the requirement of commercial firms to have patents related to commercial devices which means that innovation related to assistive devices designed within research studies are kept confidential and patented or protected. On the other hand, as shown by the literature review, due to a small market size, there are only a small number of commercial firms that produce ATs for individuals with severe and profound hearing loss.
Most of these devices were developed and produced in developed countries which unfortunately results in high purchase prices. Consequently, individuals with severe and profound hearing loss who live in developing countries are less likely to have the necessary financial means to purchase such expensive devices and remain unable to take an active role in the community. The tendency towards utilising the latest technology in ATs, e.g. augmented reality glasses, Kinect, also means that the final product will be costly, at least initially. Furthermore, using latest available technologies may also result in exclusion of individuals with severe and profound hearing loss such as the elderly who have been shown to be less receptive to new technologies. Therefore, assistive device for individuals with severe and profound hearing loss should also attempt to use affordable technologies that are already widely available. For instance, maximising the use of mobile phones through the creation of different apps that exploit their existing features such as microphone, camera, vibration alert, screen displays, GPS, etc. On the other hand, more complex, expensive and less portable ATs could be used by governmental bodies or large organisation to provide better public and commercial services.
With regard to the use of assistive alerting devices for everyday use outside the home environment, most of them, if not all, require installation of some systems that are dependent on the willingness of organisations or government bodies to adopt them. For instance, the "Deaf Alerter" (AlerterGroup, 2017) can be installed in any public building where deaf people may access. However, if the necessary installation is not being done, then the appliance will never be able to operate and transfer information about a fire event or any other incident to individuals with severe and profound hearing loss. Consequently, this minority group is unable to take ownership of some ATs designed for them. Similar issue also apply to ATs for education purposes. A possible solution, at least with regard to the use of assistive alerting device for everyday use, is to foster the implementation of a "universal design" or "design for all" approach in which the design of the product or environment would be made accessible for people with and without disabilities. Unfortunately, this design approach is rarely used in practice which means that modification of systems or products designed for hearing people is needed.
Further investigation needs to be conducted regarding increasing the accessibility of information contained within the internet through further development of plug-ins that translate the words of a website to a wide range of SLs. The development of plug-ins that translate the words of a website to various SLs will enable individuals with severe and profound hearing loss to access important and complex information faster. It will also eliminate the misinterpretations often caused by written content, mainly on websites where complex terminology or jargon is often used such as those of governments or banks. Until now, there are only a few studies concerning this matter and none of them managed to solve the problem entirely (Kennaway et al., 2007;iSigner, 2017). Better access to information contained within the internet would also mean that individuals with severe and profound hearing loss could be more proactive in finding further information on potentially suitable ATs that are commercially available. At the moment, people with disabilities reported that they had received little or no information about ATs and mostly relied on healthcare professional as their information source (Carlson & Ehrlich, 2005).

Conclusion
The development of ATs for severe and profound hearing loss individuals is an active area of research. Nevertheless, there is still a lot of work that needs to be done to allow these individuals to have equal opportunities and independence. Comprehensive user studies to identify the needs of individuals with severe and profound hearing loss together with their urgency and feasibility are needed to provide a better focus or direction on the overall objectives of ATs. Furthermore, further research needs to be done in order to make information on the internet more accessible by developing plug-ins that translate the words of a website to various SLs.