Development of a Low-cost Multimodal VR System for Engineering Design

Virtual Reality technologies are available at a lower cost than ever before. However, such systems are developed mainly for the consumer market, and inaccuracies in spatial judgements may make them unsuitable for specific applications like Engineering Design. There is evidence to suggest that the addition of haptic feedback may improve spatial judgements, but most commercially available haptic systems are impractical and unaffordable outside of specialist research settings and large enterprises. We describe the challenges for developing a multimodal VR system using only low-cost off-the-shelf technologies, and demonstrate a working prototype of a system which aims to overcome these issues.


Introduction
Virtual Reality (VR) technologies are becoming increasingly affordable, and it is now viable to obtain, Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. set up and use VR systems in everyday domestic and workplace environments. VR has the potential to improve design processes by allowing interactive and immersive exploration of products, systems and environments (see sidebar). However, it is known that distance judgement in Virtual Environments (VE) is often poor, which limits the usefulness of VR in many contexts.
This paper details development of a prototype multimodal system in a two-year feasibility study investigating the suitability of currently available lowcost, off-the-shelf technologies for Engineering Design, where accurate spatial judgement can be critical.

Spatial judgement in virtual environments
Although there are known issues related to distance perception in VE, the factors involved are complex. Most research has shown that spaces appear compressed in VE as compared with the real world (e.g. [7,14]); a review of 78 studies found that on average people judge distances in a VE at 74% of the size of real world distances [9]. However, this is not the case in all situations; for example, a study looking at exocentric distances (distances between two points in space) as opposed to egocentric distances (those from the observer to a point in space) found that participants actually tended to overestimate spaces in a VE [13], while studies closely replicating the user's physical surroundings in a VE found no significant difference in distance perception between the VE and real world [1].
Various factors may affect judgement of virtual space, including technical factors and hardware, human factors and individual differences, and compositional factors of the visual scene [9]. It is widely acknowledged that there is not yet comprehensive understanding of all factors affecting spatial perception, the specific effects of those factors, or of possible interactions between them. However, one suggestion in the literature is that haptic feedback may improve spatial judgements [3].

Issues with haptic feedback systems
Haptics can be used to give collision feedback in VE [2]. Previous studies investigating haptic feedback in VR suggest that it is more effective when co-located with visual feedback [8,11,12]. Devices like joysticks and gloves can deliver this feedback on the hand. However, in many Engineering Design applications (for example when evaluating physical ergonomics of a space) collisions could occur anywhere on the user's body.
Solutions to provide haptic feedback anywhere on the user's body exist, and have been developed for some time [6], but often involve cumbersome equipment. Systems such as full body suits can cause issues with comfort and practicality in an Engineering workplace context [4]. Coupled with the typical cost and limited availability of such systems, they are unlikely to see widespread use, particularly among SMEs not having extensive dedicated research facilities.

Off-the-shelf haptic feedback system
Full haptic prototype One solution to these issues is to use an array of vibrotactile devices (tactors) attached to the user's body, as an alternative to a full body suit. Our multimodal VR system was developed with tactors controlled by Arduino (Figure 1). Microsoft Kinect was used to track upper-body movement (elbow, shoulder and head) and Leap Motion for sensing finger Why use VR in Engineering Design?
VR permits varying levels of visualization and interactivity depending on which aspects of design require evaluation; virtual prototypes can therefore be used as design filters [5]. VR facilitates formative evaluation at earlier stages in the design lifecycle. VR can reduce the need for physical prototypes, which can:


Be costly to produce;  Be limited in visual or interaction fidelity;  Take time to produceeither design progress is halted during this time, or the produced prototype may already be out of date when completed;  Take time to modifythis means fewer, longer iterations.
As a result of these issues, there may be fewer design alternatives being considered, and commitment to nonoptimal design solutions.

Demonstrations
DIS 2017, June 10-14, 2017, Edinburgh, UK movement, which mapped the real human data onto an avatar displayed in the virtual environment. Kinect is a cheap and affordable optical system that captures depth data for human skeletons, and the SDK for Unity has human kinematics which allow it to control a human avatar in Unity. Leap Motion is the only noncontact finger skeleton and kinematic system that is widely available and cheap. Unity is a free and seatlicensed software that has large public support for codes, ideas and extensions to support new development. Unity integrates graphics and coding with features like direct control of graphics animation via code, which improve the development cycle and provide better control for design iterations. We used the Oculus DK2, an affordable off-the-shelf Head-Mounted Display (HMD) with head-tracking capacity.
Our first study [10] found that an array of multiple tactors delivering vibrotactile feedback at the specific points on the user's body is unlikely to be suitable for use in Engineering Design contexts. Participants rated this full haptic feedback system as significantly lower for comfort, acceptability and wearability, and higher for discomfort, as compared to the same VR system with no haptic feedback (i.e. wearing an HMD only).

Sensory illusions prototype
We investigated the use of a simplified version of the haptic feedback system, referred to as the "sensory illusions" prototype. Collision feedback is still delivered in the form of vibrations from tactors, but is delivered near to, rat her than exactly at, the point of contact on the user's body. This solution would permit a substantial reduction in the number of worn tactors (Figure 2), to improve comfort and wearability.
The prototype was developed iteratively with formative feedback from colleagues and Human Factors experts. A lab-based study was conducted to compare the full haptic system with the scaled-down haptic feedback system and to obtain broader feedback from Engineers. The majority of participants, who all had an Engineering background, felt that the sensory illusions setup would be more appropriate than either the full haptic system or a system with no haptic feedback at all (Figure 3). An ongoing current study ( Figure 4) investigates participants' performance in spatial tasks in a VE using the sensory illusions prototype compared to a system with no haptic feedback, to establish the potential task performance benefits with haptic feedback. Results so far suggest that participants make fewer errors in spatial judgements with the sensory illusions system.

Conclusions
Spatial judgement in VR is not the same as in the real world, which means VR is not always suitable for Engineering Design tasks, despite clear potential benefits. Adding haptic feedback may offer performance   improvements, but full body haptic systems are not acceptable to engineers. The Sensory Illusions system we present for demonstration uses low-cost off-theshelf technologies to create a wearable multimodal VR system, and early results indicate engineers make fewer errors with this than a visual-only system.