VR-Fit: Walking-in-Place Locomotion with Real Time Step Detection for VR-enabled Exercise

. With recent advances in mobile and wearable technologies, virtual reality (VR) found many applications in daily use. Today, a mobile device can be converted into a low-cost immersive VR kit thanks to the availability of do-it-yourself viewers in the shape of simple card-boards and compatible software for 3D rendering. These applications involve interacting with stationary scenes or moving in between spaces within a VR environment. VR locomotion can be enabled through a variety of methods, such as head movement tracking, joystick-triggered motion and through mapping natural movements to translate to virtual locomotion. In this study, we implemented a walk-in-place (WIP) loco-motion method for a VR-enabled exercise application. We investigate the utility of WIP for exercise purposes, and compare it with joystick-based locomotion in terms of step performance and subjective qualities of the activity, such as enjoyment, encouragement for exercise and ease of use. Our technique uses vertical accelerometer data to estimate steps taken during walking or running, and locomotes the user’s avatar accordingly in virtual space. We evaluated our technique in a controlled experimental study with 12 people. Results indicate that the way users control the simulated locomotion aﬀects how they interact with the VR simulation, and inﬂuence the subjective sense of immersion and the perceived quality of the interaction. In particular, WIP encourages users to move further, and creates a more enjoyable and interesting experience in comparison to joystick-based navigation.


Introduction
In recent years, due to technological advances, virtual reality (VR) applications have become exceedingly popular. The introduction of low-cost smart phone adapters, such as the Google Cardboard [1], also played an important role to boost the popularity and utility of VR in everyday applications. This is manifested in a huge rise in the number of mobile VR users worldwide, which increased from approximately 2 millions in 2015, to an expected 135 millions by 2020 [2]. In VR, users sense and react to rapidly changing 3D information made available through the use of appropriate software and hardware. Creating a realistic VR experience is closely linked to immersion, which creates an altered mental state by engaging the user's senses. However, unlike expensive head mounted displays (HMDs), mobile VR kits only are capable of limited levels of immersion. This is partially caused by the physical capabilities of mobile adapters and displays, which are much inferior to those provided by expensive HMDs. This creates a need for better design techniques to enable a smooth and realistic interaction between the user and the virtual world. We hypothesize that the way users interact with the environment is an effective force to increase the immersion and improve the user experience.
In contrast to passive VR applications, such as watching 360 videos, interactive applications impose the users to have an active interplay within the environment. One such interaction is to allow users to gain mobility in the simulated space, i.e. virtual locomotion, through virtual walking or running. While using HMDs, controlling the locomotion is not straightforward due to limited workspace and lack of visibility of the real world. Traditional locomotion methods using computer mice or gamepad interactions are typically insufficient to provide a realistic sensation of walking. The most common approach for virtual locomotion in VR is to use a hand-held controller for inputting motion commands. However this method is not realistic and introduces a cognitive process, meanwhile continuously reminding the user that he is in a VR simulation, negatively affecting immersion [3]. On the other extreme, omnidirectional treadmills (ODTs) are used to allow the user to walk on in all directions. However, ODTs require complicated design mechanisms, and are cumbersome and costly for personal use [4].
To allow users to wander around in virtual space, while still staying in affordable ranges, walk-in-place (WIP) approach for locomotion control was proposed [3]. In WIP, as the name implies, the user walks or runs in place, and steps are captured using sensor data. In this study, we compare the effects locomotion control on user experience by comparing joystick navigation to WIP. We investigate these effects within an exercise application, and investigate how the control technique would affect user behaviour when reaching the goals of the application, i.e. activity boosting, and user satisfaction.

Background
There are many different approaches for virtual locomotion. The simplest and most widespread approach, commonly used in computer games, is to use a joystick, which might come in the form of a game pad.
Step commands are given as affine transformations (rotation and translation) defined by the looking direction of the avatar and position advancement, all inputted through joystick input. However, controlling both rotation and translation through a joystick turns out to be confusing when using a HMD, hence is problematic as a motion control technique, and typically not used in such VR settings. An alternative way is to handle rotations with physical body rotations and controlling translations with the joystick [3].
In [3,5], it is argued that a real walking approach, where the user can actually walk during interaction, is remarkably better in terms of naturalness and simplicity for virtual locomotion. An optical tracking system can be used while implementing the real walking approach [3], however this approach is also limited since the tracked space and the virtual space are required to be of the same size. This problem can be solved using omnidirectional treadmills [4]. However, this solution is often expensive and bulky, which limits its use with a mobile platform.
In [6], it is proposed that using the walk-in-place (WIP) approach is a favorable alternative over the joystick and treadmill approaches. It was suggested that locomotion can be controlled using WIP as easily as using a joystick [7]. In addition to this, WIP has also been observed to be as practical as real walking [5]. In this study, we propose that the way users control the the simulated locomotion affects how they interact with the VR simulation. We present VR-Fit as a VR-enabled exercise application, and evaluate how task performance and the perceived quality of the activity, such as enjoyment, encouragement for exercise, interest, ease of use, and exercise intensity is affected by the use of WIP.
WIP relies on sensor information to detect user steps. The location and type of the sensor, step detection algorithm implementation, and noise reduction in sensor readings can result in a latency between action and reaction. In [4], Darken et al. argue that starting and stopping latency is a significant challenge while implementing WIP. The starting latency is the delay between the moment a step is recognized and converted into virtual motion, whereas the stopping latency relates to how long it takes the virtual motion to stop once the user stops taking any steps. In case latency is high, navigation quality will drop [7]. Different algorithms have been developed to deal with latency.
Step detection methods can be divided into the following three categories:

Peak Detection
This method uses successive peak and valley acceleration values to detect steps [8][9][10][11]. Mock et al. [10] use acceleration values and implement a step counter method which finds hills of consistently perceived acceleration values. When a hill is detected, one step is registered. In [9], it is observed that while walking, the measured vertical and horizontal acceleration values can be modeled as sinusoidal waves. Therefore, when a sinusoidal pattern is identified, a step can be detected. In [8], local maximums among a series of sensing measurements are identified. When a local maximum is identified, one step will be registered. [11] regards the local maximum between two different local minima to capture steps. This work implement an initial low-pass filtering on data to clean the noise in acceleration measurement. After that, they define a threshold to see if the difference between two adjacent local minimum and local maximum values is large enough. If this difference is greater than the predefined threshold, one step will be counted. Although peak detection is simple to implement, it is shown that this method may not consistently provide accurate results, especially when used with mobile phones, due to irregularities in the ways that the users are handling these devices [12].

Gait Analysis
Gait is defined as the pattern of walking, which is typically modeled through footfall patterns. At the moment of a footfall, the lower leg is perpendicular to the horizontal plane, i.e. the floor, and walking velocity is zero. In order to detect steps, zero velocity update method is used in [11,13,14]. In [15], gyroscopes are tied to users shoes to gather scalar values determining the foot speed. If this speed is smaller than a given threshold, one step is registered. In [13], a pressure sensor is installed in the soles of the users' shoes to detect zero velocity update while walking. In [14], the authors place inertial sensors to users' calves. Although these can effectively detect the users steps [13][14][15], the dependency on external sensors, and the need for good sensor placement induce a difficulty in using the approach for everyday use.
As discussed above, even though a gait analysis provides accurate step counts, it requires external sensors located on the body. In mobile VR, the use of smart phones allow us to use the sensors that are already built on board for step counting. However, since these sensors cannot be fixed to the lower extremities, e.g. legs, ankles, calves, as commonly practiced in gait analysis, extracting gait patterns through mobile sensing is not feasible.

Threshold Setting
In threshold setting [16][17][18], sensory data is compared to some threshold, and a step is registered if the reading is higher than the threshold. In [17], an inertial sensor is placed on the users ankle. When the measured data exceed a predefined threshold, a step is detected. Alternatively, in [18], the authors apply a similar approach to detect steps, in which they tie an inertial sensor on the users leg. In [16], Alzantot et al. suggest to use a finite state machine (FSM) in addition to setting thresholds to count steps. They define different states in the FSM as regular walking, walking briskly, and running, which exhibit different regular variations in motions. These regular variations are automatically classified into the states using threshold values. When detecting steps, the proposed scheme first identifies the state of a user, and then triggers a state transition if sensory values exceeds the threshold assigned for that particular state.
Due to relative simplicity of implementation, threshold setting is a viable alternative for step detection and step calculation in literature. [17,19]. The threshold method may fail to be effective when used with mobile devices in case of hand-held applications. In such applications, the unconstrained motion of the mobile device cause the data range to change abruptly, affecting the utility of the thresholds [19,20]. However, in our study, the VR headset is worn in the head, hence ensures a good frame of reference for sensors on the smart phone.
Hence, in our setting, we assume that these problems can be neglected without loss of generality.
In this study, we used the threshold setting method to implement virtual locomotion in a mobile VR application due to minimal sensor requirements and simplicity of implementation. One challenge in threshold setting method is the choice of thresholds, which can be affected by the movement patterns of users, and the properties of surface and shoes [19]. In order to investigate how our implementation is affected by such differences, we conducted a small-scale pilot study with 3 users as summarised in Section 5. The users freely walked in place on different surfaces, and we investigated how the detected step counts change across users and surface types. We verified that user and surface types do not radically affect the WIP performance in our application.

Fig. 1. Game screen for VR-Fit
In order to compare WIP and joystick-based locomotion interface, we implemented a fitness application, named VR-Fit. In VR-Fit, the user is placed in a forest setting, where he/she acts as a runner, fleeing from a fire (See Fig.  1). En-route to his/her escape, a firewall chases the user. The aim of VR-Fit is to motivate exercise, hence the user enters his/her daily step target, and is provided with the current number of steps while walking/running. In case the fire reaches the user, or the user exits the game, exercise records, e.g. current walking distance and steps remaining to achieve the daily goal, are updated. To make the fitness application complete, a profile review was added into the system. In the profile, the user is given options to review his/her profile, e.g. weight, height and age specifications, and set new daily goals. VR-Fit is implemented for iOS. Unity 3D and Xcode platforms are used to develop the software and the 3D environment.

Virtual Locomotion: WIP and Joystick Locomotion Algorithms
There are many virtual locomotion techniques, such as using joysticks for navigation, omnidirectional treadmills for real walking, and walking-in-place. In order to present the benefits and utility of WIP, we compare it with joystick-based navigation. We hypothesize that WIP is beneficial as it would encourage the user to do exercise.
The locomotion in VR-Fit is enabled through a threshold setting WIP technique. Once a step is detected, physical movement is translated into virtual movement to enable virtual locomotion. For this purpose, we use the data coming from the accelerometer and the gyroscope located on the mobile device. When a user starts walking/running in place, acceleration values in y-axis (perpendicular to the ground plane) are recorded and compared to predefined thresholds. If the acceleration exceeds the predefined threshold, a step is registered and motion is generated. Algorithm 1 shows how steps are detected and used in WIP to generate motion. Note that the HMD orientation is used to determine the moving direction of the generated motion in the virtual space. Joystick-based locomotion uses gaze direction and and joystick input to generate the motion. In particular, the user's head direction is used to set the virtual heading, whereas the velocity is set based on joystick input. Algorithm 2 details the joystick-based locomotion technique we used in the experiments.

Pilot Study: Evaluation of WIP Implementation
In order to evaluate the utility of WIP to generate consistent motion, we tested our step count detection algorithm. In particular, we were interested in seeing whether the threshold setting approach we adopted is able to estimate the step counts consistently without regard to user and environment differences. In order to validate this, we conducted a small-scale user study with 3 participants and observed the outcome of the step counting algorithm on a granite floor and a carpet.
The pilot study was conducted with two male and one female volunteers, whose ages ranged between 19-23. All three participants used the application on both types of floor (granite and carpet). The participants were asked to walk in place three times for each floor type and take 100 steps. The experimenter counted the steps and terminated the trials when 100 steps were reached. After the trials, number of estimated steps was recorded to validate step counting algorithm. As shown in Figure 2, the step count algorithm achieved consistent performance across users and different types of floors.

Experiment: WIP vs. Joystick-based Locomotion
In order to provide a comparison of alternative locomotion methods, we conducted an experimental study with 12 subjects. A within-groups factorial design was used for this comparative study. The participant pool consisted of 5 female and 7 male volunteers, whose ages ranged from 22 to 29.

Procedure and Experimental Design
We designed a controlled user study to compare between two different interaction techniques, WIP and joystick-based locomotion interface. The participants were randomly assigned to two equal-sized groups. The first group used the application firstly using WIP, followed by joystick-based locomotion. The second group firstly used the joystick-based technique and then the WIP interface. This design eliminated any ordering and learning effects. The participants were instructed about how to navigate using both methods prior to starting the experiment and all participants gave informed consent before participating in the study. The participants were explicitly told that the purpose of the application is to promote exercising, hence they were asked to walk in place regardless of the applied locomotion method.

Data Acquisition and Evaluation Measures
The estimated step counts are recorded during the experiment to indicate the actual activity level of the users. After the trials, a questionnaire was given to the participants to evaluate their experience under each navigation method. The participants were asked to indicate their level of agreement or disagreement on a 5-pt Likert scale (1 = Totally Disagree, 5 = Totally Agree) for a series of statements: 1. Activity: I got tired from exercise while using the application. 2. Enjoyment: I had fun while using the application. 3. Encouragement for exercise: The application encouraged me to do exercise. 4. Interest: I would be interested in using a similar application in the future. 5. Ease of use: I was able to easily wander around within the virtual environment. 6. Intensity: I felt like I was exercising intensely.

Results and Discussion
This section presents the results of the experiment in terms of the quantitative and qualitative measures defined in Section 5. Statistically significant differences between conditions are investigated using Wilcoxon signed-rank test in the case of questionnaire results. The differences between conditions in terms of step counts is investigated using one-way repeated measures ANOVA. Since Mean scores and standard errors of the means of the questionnaire responses are shown in Figure 3 (questions 1-6). The results indicate that the locomotion technique has a statistically significant effect on the activity level, manifested as increased fatigue (Q1) (p < 0.001), exercise intensity (Q6) (p < 0.001) and encouragement for exercise (p < 0.01). Even though the participants stated that they felt more tired due to exercise and thought that the exercise was more intense, they were significantly more encouraged to do exercise (p = 0.005) when using the WIP interface.
Looking at the estimated step counts, we observe consistent activity levels to those that were perceived by the subjects. As seen in Figure 4, the subjects walked more the WIP interface than they did with the joystick (p < 0.001). These results support the hypothesis that WIP was successful to get the users to move more when compared to joystick-based locomotion.
Qualitative results shown in Figure 3 also indicate that there is a statistically significant difference between WIP and Joystick interfaces for Q4 (p < 0.01), expressing an increased interest for the subjects to use WIP in other applications. The participants reported increased enjoyment when interacting with the WIP locomotion technique, as indicated by their responses to Q2 (p < 0.05), The responses to Q5 indicate that the participants were comfortable while using both interfaces, however, there was no statistical difference between two interfaces (p = 0.25).

Conclusions
This study investigated perceptual differences and variations in activity levels between WIP locomotion interface and joystick-based navigation when using a VR-enabled mobile fitness application. We designed and implemented a real-time step detection technique in a VR-setup to enable WIP locomotion in a virtual environment.
We validated our WIP locomotion technique in a pilot study, and verified that the proposed technique eliminates differences between users and the type of floors while detecting steps. Our results indicate that the way users control the simulated locomotion affects how they interact with the VR simulation, influence the subjective sense of enjoyment and attracts more interest while increasing physical activity levels, when compared to more traditional joystickbased locomotion.
In particular, WIP allows users to move further, and let them have a more immersive experience. Significant differences between qualitative survey results clearly indicate the potential of WIP as a viable technique that can be used in mobile VR settings.