VRtefacts: Performative Substitutional Reality with Museum Objects

We explore how a combination of manipulations and transitions can extend Substitutional Reality to create a highly personal Virtual Reality experience. Our design aimed to meet two challenges faced by museums: the limitations of object handling and the desire for visitors to create their own interpretations. Using a Research-through-Design methodology, we built a performance-led Mixed Reality (MR) experience that lets museum visitors physically handle 3D prints or scans of museum objects to share personal stories about them. The stories are recorded and donated to the participating museum. We reflect on the complex design and the findings gained from a two-day in-the-wild deployment to explore engagement and disruption through manipulations of physicality, visuals, and scale; the transitions between spaces; and a trajectory of storytelling performance. We chart a wide scope for Performative Substitutional Reality and draw implications for VR, MR, and performance-led research in any context.


INTRODUCTION
This paper explores a design that uses tangible representations of museum objects embedded within a VR environment to inspire personal storytelling. Passive Haptics [36,39] describes the practice of populating the real environment around a VR user with physical objects that aim to match the virtual objects they see in VR, while Substitutional Reality accepts some object mismatch and investigates how, and how far, that mismatch can be stretched [58]. We aimed to: 1, make a very accessible VR experience so museum visitors could touch, Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. DIS '20, July 6-10, 2020, Eindhoven, Netherlands. © 2020 Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-6974-9/20/07 ...$15.00. http://dx.doi.org/10.1145/3357236.3395459 move, and look up close at one 3D print and one 3D scan housed in a physical acrylic box; 2, explore reactions when some of those objects had been rescaled and none had the same weight or feel as the original; 3, motivate these visitors to explore their personal connections with their objects; 4, structure the experience so they could tell a coherent personal 'story' around each object; and 5, make them feel comfortable donating their stories to the museum for future public display. Our task was to juggle the competing goals within a single, brief, performance-led encounter for museum visitors.
Our primary contribution to the DIS community is what we learned about manipulations and transitions when navigating a very complex situation. We offer specific points of contact that pose surprising strengths, challenges, and opportunities arising from manipulations of physicality, visuals, and scale, and elements of the spatial and internal trajectories of participants' experiences. This work fundamentally follows the Research through Design methodology [31], especially in its generative and exploratory senses within the emerging design space of museum-situated VR/MR experiences. Our work also includes elements of performance-led research in the wild [11] and Performative Experience Design [62]. Although we video recorded all sessions and conducted interviews with all users, these findings only supplement our primary focus: our experimentation with emerging technologies deeply embedded in a Performative Substitutional Reality experience.
To provide context for the topics explored in this paper, we first set out the 2 challenges faced by museums that motivated the design: problems around existing object handling practices and the need for effective methods to solicit visitor interpretations. We then provide a quick background on the 2 main strategies we used to address them: the HCI research that underpins our definition of Performative Substitutional Reality, and story performance mechanisms. Next, we offer a detailed description of our design as deployed. We discuss our findings in terms of manipulations and transitions and conclude with the challenges our discussion has opened.

Object Handling in Museums
Museum Studies is the academic discipline of museums: their history, functions, methods, and audiences [72]. Much of the field's current thinking is occupied by 'new museology' -a pushback against elitism, curatorial authority and exclusionary practices [46]. A key aspect of new museology is what Howes has referred to as 'the rehabilitation of touch' [38, p. 259]: the idea that museums need to reconsider their traditional emphasis on visuals and allow visitors to physically interact with collections. The use of touch has many advantages: it stimulates more of the visitors' senses, which supports learning and engagement [61]; it creates a more varied experience; it forms a sense of 'intimacy' with those who made or used the object; and it can help visually impaired visitors to experience objects that would otherwise be inaccessible [38].
While touch is an advantageous addition to the museum experience, it is often not feasible. Visitors can harm museum collections by exposing them to contaminants found on skin [1], or subjecting them to structural damage through overuse, dropping, or scratching [60]. Additionally, some museum objects can be harmful to visitors; they may be heavy, have sharp edges, or contain hazardous chemicals such as arsenic or lead [53]. These practical issues mean that many museums have simple 'do not touch' policies in place, forbidding visitors from physically interacting with their collection.
Museums have found partial solutions to some of these issues, for example situating realistic, touchable replicas next to original objects which can make intricate details easier to see and feel. While such reproductions are often popular, it is frequently argued that 'real objects "speak" in ways that representations of those objects do not' -museums and visitors still value originals over replicas [27]. Other museums have designated handling collections made up of real objects, [15], but these also have limitations: staff supervision is often required, and nothing rare, valuable, or delicate can be used. Thus the objects that visitors are most interested in (often the things that the museums are best known for, like Birmingham Museums' Staffordshire Hoard [2] or the British Museum's Rosetta Stone [3]) are the objects least likely to be included.

Visitor Interpretation of Museum Collections
Questioning curatorial authority and presenting multiple perspectives (including those of visitors) are also integral aspects of new museology [64, p. 277]. Museums subscribing to this view often wish to solicit visitors' interpretations of their objects, to complement or even replace existing top-down curatorial interpretations. Psychologist Jerome Bruner [21] identified narrative, including performed narrative, as a means of forming individual understandings of culture. On this basis, many museums adopted a narrative-based approach [8], although the construction is nearly always in the hands of the professionals, with 'performed' contributions often relegated to structured enactments by schoolchildren or a 'talk-back area' where visitors can leave stories, presumably for the museum to keep and perhaps for others to find. A notable interactive exception is the CHESS project [52], though this also focused on museum authorship. However, museums are increasingly encouraging visitors to engage in the interpretation of collec-tions by asking visitors to respond to objects in various ways, and then using these responses as part of the object's interpretation [59], with [56] as an example from within the HCI community. As explained in our design section below, we saw visitor interpretation as a design aim in its own right and a key to sustained engagement with replica museum objects.
While museums sometimes solicit visitor interpretations, such as comments on their objects, these are rarely framed as 'donations'. Donations are usually seen as financial transactions, from token contributions to large bequests [49,4]. However, donations are a form of gift, and even the gifting of simple anecdotes told with a particular gift-receiver in mind has been shown to create emotional impact [63]. Even an impersonal donation of one's own objects, when transformed through an artistic process, can create both personal and cultural value for the donor [54]. Therefore, framing visitor interpretations in the context of donating personal stories around objects available for handling seemed to be a promising new option.

Substitutional Reality through Passive Haptics in VR
Our design aim was to explore how to engage users with a tangible replica embedded in a VR context in which object manipulation could solicit personal visitor interpretations. Interaction Design has a longstanding interest in touch. Early work on 'Graspables' [28] and 'Tangibles' [70] inspired the field of Tangible and Embedded Interaction, which explores the possibilities of using physical objects -creations and appropriated mundane items -as novel interfaces [14,12,13,22,28,40,41]). Tangibles also have a long history of use in museums contexts [26,37,50,57]. Several museums and art galleries have used VR in the past decade to better immerse visitors in collections (e.g. [53]) or to share collections not physically on display (e.g. [5]), but they have not used haptics.
Another approach lies in the haptic technologies that underpin VR [69]. A core proposition of VR is that it presents a world that can seeem 'real' to the point that a user can feel present and/or immersed in it [48]. This proposition implies that as many of a user's senses as possible should be stimulated. In practice, current VR primarily focuses on visual perception via head-mounted displays (HMDs). The user's other senses are still perceiving the physical environment around them. This introduces Sensory Misalignment [45] or Mismatch [35], where a user's eyes are telling them a different story from the rest of their body, which can negatively impact the experience or be exploited in creative ways. A large part of prior and current VR research focuses on minimising or exploiting the misalignment. While we cannot hope to do justice to the full tradition influencing our approach, we offer a sketch of the groundwork on which we base our approach of Performative Substitutional Reality. The 'performative' part is addressed best in the next section. The VR in VRtefacts sought to explore a new angle of Substitutional Reality, which is based on the concept of Passive Haptics [36,39], introduced above. Our 3D prints and the 3D scans housed in their acrylic box, as well as the chair and table, are instances of Passive Haptics in that they populated the environment within the visitor's reach with physical objects that matched the virtual objects. By matching the haptic expectations of the user, they could exhibit characteristics of real-world movement and to create a greater sense of 'real-world' presence in the virtual world [65]. While raising questions around flexibility and costeffectiveness, Passive Haptics can contribute to powerfully immersive experiences [23,69] under the right circumstances.
The challenge is increased when the user interacts with objects in VR that can be handled. The examination of these objects, particularly via touch, increases the likelihood that they will fail to meet users' expectations by dramatically increasing the sensory misalignment. Substitutional Reality [58,66], an extension of Passive Haptics, focuses on manipulating how users perceive reality [66]. It investigates the extent to which the misalignment is acceptable [58], how it can be overcome by other stimuli such as the more dominant visual [43], or how it can be repurposed through techniques such as redirected touching [43], haptic retargeting [6], and sparse haptic proxies [24]. Performative Substitutional Reality seeks to engage any of the possibilities opened up by Substitutional Reality in order to engage users in personal storytelling.

Story Performance Mechanisms
We felt that providing a 3D replica of museum objects in VR could deliver novelty and perhaps excitement but risked stopping at a shallow level of engagement. We therefore solicited visitor interpretations and used their storytelling process to frame and guide their interactions with the object. The following section briefly reviews similar work in interaction design, followed by the performance approaches we adopted.
Many existing designs for using digital technologies to solicit personal stories (which, when responding to museum objects, become 'visitor interpretations') have focused on writing and arranging text and/or media for later sharing (e.g. [44,42]). Interventions for telling or sharing spoken stories tend to have enhanced or guided conversation as their goal: for example, 4Photos [68] showed people gathered around a dinner table a selection of their personal photos, and Cueb [32] used colocated storytelling about personal photos to strengthen parentteen relationships. The TOTeM project brought stories told about personal objects into the public realm via IoT [7], while the Carolan guitar brought stories and music to a self-selecting, snowballing group [12]. Other projects for co-located media sharing explore relationships between media organisation and storytelling [33,30,16], 'shared remembering' [18], or multisensory conversational stimuli for the elderly [51], dementia sufferers [47], or school children [55]. Narrative storytelling is well explored by VR researchers, though nearly always in the context of structuring the stories that VR users navigate or interact with; few, if any, examples of user-generated storytelling exist outside of Rec Room's social VR spaces (https://rec.net/). Moreover, neither VR nor museum research navigates a middle ground between conversational reminiscence and carefully structured narrative.
For this, we turn to the Performative Experience Design (PED) methodology [62], which also relies on Bruner's understanding of narrative, especially of personal experience [20]. PED advocates using performance literature and practices that can meet the goals of a relevant design. Storyteller and theorist Mike Wilson [74] offers tools for transforming anecdotes that might arise in conversation into planned, non-conversational stories. His 'Performance Continuum' offers six parameters to analyse -or create -shifts from conversation to performed stories suitable for the wider public [75]. These parameters include the level of intensity, formality, consciousness (of performing), risk (of embarrassment), and rewards.
Participatory theatre, in which audience members are invited to contribute, is also important to consider, as it offers agency to people who would ordinarily expect only to spectate [73] in much the same way that museum visitors expect only to view from a distance. Theorist Gareth White [73] also discusses framing, aesthetics, self-awareness, risk, and other close parallels to Wilson's components of the Performance Continuum [75]. Finally, 'visitor interpretation' does not mean re-telling stories from other sources: it demands a personal contribution. Therefore, each story teller must also be a story maker. The canonical text Autobiography and Performance [34] describes making a personal story a continual process of 'engendering a coherent and continuous identity as we remind ourselves in the present of who we were in the past' [34, p. 95]. The professional performances analysed have been scripted and rehearsed over considerable periods of time before being performed, but the process of conjuring memories and associations from an object in order to share those thoughts (or fictions based on them) with other people is nearly universal and accessible in some form to almost anyone.

THE VRTEFACTS DESIGN
After exploring many possible routes, we decided to create a VR experience in which visitors could handle physical objects matched to high-fidelity 3D models, either as 3D prints or as 3D scans housed in a physical case. They explored these objects using both touch and sight with the aim of thinking up a personal story to tell about it, a video of which they would donate to the museum. Stories donated to the museum are visitor interpretations of its objects for the museum to collect, analyse, display, and store for future generations.

Our Museum Partner
Although VRtefacts can be adapted to virtually any museum, it is nearly impossible to describe without the context of the museum the iteration is built for. The partner museum provides both physical spaces for visitors and the artefacts from which the 3D models originate. Coordinating our goals with theirs also strengthened the importance of visitor interpretation through storytelling to the experience. This paper describes VRtefacts as deployed on 22-23 May 2019 at the Derby Museum and Art Gallery, part of the Derby Museums group, which houses a collection of local art and objects. The citycentre location was chosen to showcase their 'Museum of Making', which was under reconstruction at the time.

VRtefacts in Derby
Visitors were invited to take part from inside Derby Museum's collection space. A researcher explained the project to potentially interested visitors. They introduced the expectation of donating stories and wearing a VR HMD but gave no other clues as to what would follow (though some participants had heard that objects were involved). The researcher gained

Emotions and Collaboration in VR
DIS '20, July 6-10, 2020, Eindhoven, Netherlands written consent for the study. Because we sought visitor feedback, we formalised the consent process. Then the researcher ushered the visitor into the Art Room, a space for outreach activities located just off the main thoroughfare through the collections -still clearly part of the museum, though just as clearly not intended for unsupervised public use.
The Art Room was set up with all of the technical equipment required for VRtefacts, plus a table, chair, green screen, and camera mounted on a tripod (see Figure 1). Each visitor took part individually, with one researcher managing the VR environment and another researcher acting as a 'host' to guide their VR Substitutional Reality storytelling experience. The host welcomed each visitor to the Art Room, explained the mechanics of VR, confirmed the visitor's intention to donate their stories around museum objects, and asked visitors to abide by basic guidelines for user-generated content (no obscenities, defamation, etc.). The host then placed the HMD on the visitor's head and adjusted it for the participant's comfort.
Inside the VR, the visitor could see the Donation Hall, a large space meant to combine the importance of a venerable museum with the sense of an employee-only workspace. The chair in which they sat occupied the middle, with a table in front of it. As visitors were instructed not to get out of the chair, only those two items were initially within reach. The physical chair and table were fixed in place and mirrored their virtual counterparts. The visitors had no visible presence in VR other than a model of a generic headset matching the position of their HMD, for reasons detailed in the technical design.
The rest of the VR environment contained the six objects for which we had a 3D model. Each was shown in its own museum-style display cases, roughly 2 metres high and wide by one metre deep, three along the left side and three along the right. To make objects equally easy to see, they were well lit and scaled to be approximately the same size, at least one metre at their longest or widest. Thus all of the objects except the aircraft engine and toolbox (see below) were scaled up dramatically to fit in the display cases. Each exhibit had an interpretation panel next to it showing a close-up of the object, its name, and a brief description. Directly in front of and pointed at the user stood a camera on a tripod flanked by two large spotlights (see the bottom right quadrant of Figure 3.
The objects that lined the right-hand wall were the ones for which we had produced 3D printed replicas. The ones on the left had no 3D-printed replica, but were instead 3D scans, each shown in the hand-held vitrine. The visitors were not made aware of this distinction. Guided by the host, they were instructed to choose the object that most interested them. Depending in the object they chose, the host quietly retrieved the corresponding 3D-printed object -or 3D scan within the hand-held vitrine -and placed it on the table in easy reach of the participant. The researcher then activated that object's display in VR, causing it to materialise with an appropriate visual effect. Participants could touch, lift, move, and explore the object, and then tell a 'story' -any personal memories, thoughts about the object or other objects associated with it, or even fictional tales, as long as the participant thought them worth sharing. When they finished, the researcher made the object 'dematerialise' and the host asked them to choose an object from the other side of the hall. This instruction ensured that each participant experienced both a 3D print and a 3D scan. The materialising, exploration, storytelling, and dematerialisation process was repeated. The host concluded the VR session and removed the HMD before sending the participant to the next stage of their museum visit and welcoming the next participant. The host's guidance process is explained in more detail 'Performance-led experience design' below.

TECHNICAL DESIGN Object Choice and Preparation
Derby Museums had already digitised some of its collection into 3D models using a handheld structured light 3D scanner that provided high-enough fidelity visuals. Thus we chose objects from their archive of models. Initially we wanted two very large, two very small, and two at-scale objects, to investigate effects of scale at 'handheld' size. There were no very small objects available, so the smallest object included was a Glass Pipe. We excluded several objects due to their proportions. For example, an old and brightly coloured push mower would have made an eye-catching and tactile 3D model, but in order for the interesting bottom section to be perceptible in any detail, the less interesting handle would have to have been unfeasibly long. We also avoided fabrics, which visitors might expect to fold, stretch, or wear -which would not have been possible given the tracking technology available.
In total 6 3D models were chosen. The 3D prints were a scaled-down version of the Rolls-Royce Eagle Aero Engine that made the first transatlantic flight, a Wooden Boar's Head, and the aptly named Mystery Textile Equipment (see Figure 2). The prints were no more than about 25cm in their greatest dimension. They were securely mounted on top of 'plinths', clear acrylic boxes measuring 20cm x 20cm x 10cm that contained each object's assigned tracking device, detailed below. Visitors could lift the objects using either the object itself or its plinth, though they were warned against lifting the Engine or Textile Equipment by their most fragile parts. The 3D scans were the scaled-down Pattern Maker's Toolbox, the Ogoni Elu Bird Mask, and the Kilburn Glass Frigger Pipe. The models were scaled to fit inside the 'vitrine', a box measuring 15cm x 15cm x 30cm that appeared as a glass case in VR but in reality was an acrylic box housing its assigned tracker. The researcher could make any of the scans appear within the box according to the visitor's choice.

Making and Tracking the Artefacts
One method for making physical objects, which is increasingly common practice in museums, is to commission replicas [67]. However, this process is resource-intensive and not always feasible for complex objects such as the Engine. Instead, we used high-quality 3D printing technology to create geometrically exact copies of the 3D models: ABS plastic for the Boar's Head and Mystery Textile Equipment, and sintered polymer for the more intricate and delicate Engine. This allowed us to be much more flexible in our choice of 3D models and kept skill, time, and cost requirements down.
The next task was to make the objects trackable so that their corresponding VR versions followed their exact position and rotation at all times. Several technologies can achieve positional tracking of objects down to the millimetre. However, they usually involve costly, complex installation-based motion tracking systems using optical sensors or ultrasonic emitters. These approaches also require a dedicated space, and were therefore out of scope for our partner museum.
With the above in mind, we opted for HTC Vive (https://vive. com). Its ecosystem uses two Vive Lighthouses placed in the physical environment. The controllers and headset use these to determine position and rotation. Vive Trackers are accessories slightly smaller than standard Vive controllers. They attach to physical objects to control VR tennis rackets, game weapons, etc. The challenge was that the trackers, already less accurate than the controllers, also had to be fixed to the objects and become part of their geometry. In early tests, users tended to grasp the objects by the attached trackers, obscuring them from the Lighthouses and therefore breaking the tracking. This design limitation inspired idea of plinths and vitrine.
We also needed to capture the stories on video. We set up a functioning virtual camera in the VR environment situated in the physical camera's actual line of sight. This captured the user's headset and the disembodied object, useful when the visitor moved it in keeping with the content of their story. The result was a simultaneous 4-perspective video frame arranged in a 2x2 matrix. The video consisted of the first-person perspective of the user, a view from the virtual camera in the VR space, a real view from the matching camera in the physical space, and a composite view of these created by superimposing the user onto the virtual camera view using the green screen, as seen in Figure 3. Audio was picked up from the HMD's onboard microphone.

PERFORMANCE-LED EXPERIENCE DESIGN
As mentioned above, the researcher acting as the host for the visitor's experience did much more than explain the requirements of an object-handling or storytelling task. They performed for each visitor according to a carefully worded and structured script, from which they could deviate as necessary to suit each visitor's needs. The host's performance in turn guided the visitors through various transitions, most critically the transition from conversation (chatting about the object with the host) to a more intense, formally told, consciously constructed, risky (to their pride), and potentially rewarding mode of storytelling for donation to the museum, including possible public viewing. In the end, both host and visitor performed: the host for the visitor, and the visitor for an unknown future audience.
The host framed the experience by introducing the legal and practical necessities and preparing the visitors for VR, as explained above. The host then described the VR environment as the museum's 'Donation Hall', 'where the museum curators work on the objects they most want to hear your thoughts about'. The nature of those thoughts was subtly steered by the host asking each participant gently personal questions such as about relationship to the city of Derby. The host asked which object they found most personally appealing, then quietly placed the object or vitrine on the table in front of the visitor. The researcher then made the object appear to materialise from nothing on the table before the visitor's eyes, increasing the surprise of the encounter. We felt that seeing a static object in front of them would gently invite interaction and be less disconcerting than making it visible from the start and watching it wobble towards them through thin air, as the host's body could not be satisfactorily represented within the VR's look and feel.
The host managed the encounter as necessary through the use of voice. Sometimes, this required the host to instruct them to reach out for the object, to touch it directly rather than simply holding the mount. The host then encouraged thorough and thoughtful exploration of the object, including memories and associations around the object. Both the encounter and the

Emotions and Collaboration in VR
DIS '20, July 6-10, 2020, Eindhoven, Netherlands exploration (and sometimes the storytelling itself) involved the host explicitly validating the participant's perceptions and suggestions, building their confidence that whatever story they wanted to tell, no matter how tangential or piecemeal, would be valued. To make each story as coherent and self-contained as possible, the host balanced this conversational encouragement with the instruction to save up their actual stories for the moment that the VR environment's key lights and 'camera' would switch on. The aim of this was to delineate the moment that the storytelling performance would begin, with the slightly increased intensity, formality, consciousness, and risk involved. The host sometimes continued to question less talkative participants when it was clear they had more to say but were hesitant to say it. When the host judged the story was complete, they signalled the researcher to turn off the lights in the VR and asked the visitor if anything on the other side of the room appealed to them. Again, this ensured that each participant experienced one 3D printed object and one 3D scan held in the tall vitrine. After repeating the process of encounter, exploration, validation, and storytelling for the second object, the host removed the HMD and asked the visitor to confirm their willingness to donate their story. This brought the visitor out of their memories and imaginations within a virtual world and back to their real selves in relation to the real museum.

VRtefacts in Action
Twenty-four participants (7 male, 16 female, 1 non-binary), with ages ranging from 17-72 years, took part in VRtefacts over the two days of the study (3 aged 17-19, 9 in their 20s, 2 in their 30s, 4 each in their 50s and 60s, and 2 in their 70s). The preponderance of female participants is common in museum contexts, while dip in people of working age likely reflects the weekday scheduling. One was wheelchair-bound and another wore a wig for medical reasons; we easily adapted to accommodate their needs. Previous VR experience covered the spectrum from none to regular use of commercial headsets. Elderly participants tended to have less VR experience than younger participants, though they were if anything more enthusiastic about it. The majority of participants regarded themselves as frequent museum-goers, 16 had already visited Derby Museum, and 2 had never been to a museum before. By pure luck, the first objects chosen were nearly evenly split between scans (11) and prints (13), thus precluding any concerns about expectations being dominated by one or the other.
The interview questions covered the main topics addressed by VRtefacts, such as participant responses to physical interaction with the objects within VR, their perceptions of scale, etc. However, given our Research-through-Design [31] motivations, we anticipated that there would be at least as much to be discovered from unanticipated responses to this exploratory design for an emerging design space. Therefore, 3 of the authors conducted full inductive thematic analysis on the interview transcripts to uncover any unexpected themes or patterns [17], many of which shape and inform our discussion. Two authors also reviewed and took notes on the video recordings to supplement post-hoc interview data with in-the-moment reactions. However, this data resulted in a richer understanding of their responses but no new themes.

POINTS FOR DISCUSSION
We have established VRtefacts as more than an object-focused VR experience. We have sought to explain how it is deeply contextualised by its specific host institution, how it seeks a thoughtful and generative encounter with an object, and through that, with the visitor's own personal memories, associations, and imaginative wanderings brought on by the visual and haptic experience. We now sift through our exhaustive analyses of participant interactions and responses, along with our own well-documented design process, to reveal global reactions to the experience and distil from the rest the two points that we believe can contribute to future work in this design space: manipulations of physical objects, virtual objects, and scaling between the two; and transitions through physical, virtual, and personal (storytelling) spaces.

Global Reactions
As mentioned above, some participants immediately reached for the objects, while others had no idea that the image they saw had a physical dimension and had to be explicitly told they could touch it. Overall, initial engagement was tentative, though a few handled the objects casually and easily. The most common response for 3D prints was to reach out slowly, determine the object's dimensions by fingertip, lift it gently, hold it by its plinth while moving it to examine from multiple angles, then place it back on the table while storytelling. For 3D scans, the variation was to spend less time with fingertip exploration and more time viewing it from different angles. We were surprised at how very few held, manipulated, or gestured to these 'props' while telling their stories.
Some participants found the physicality of the objects to be the anchor points that scale could not provide: how much 'more natural' it was 'to interact...in a tactile way' (P8); or the way that touch 'helps ground you' (P11). One was so caught up in the physicality that they 'forgot about the fact that I was obviously just holding a model or a box' (P22). P11 explained how 'it's part of human curiosity to want to touch, to rotate it'. In other words, some took physicality for granted, and scale manipulations went unremarked.
Most participants were notably more engaged with 3D prints than with 3D scans (e.g. P1), though people with a strong interest in the subject of a scan could be just as engaged with it as with any object. The preference that most noted for 3D prints in interviews did not elicit complaints at the time and did not impede storytelling. This is also backed up by timings: average time spent preparing stories for scans and objects differed by only three seconds (1:11 for scans compared to 1:14 for prints), and average time spent telling stories about scans and objects differed by only 11 (1:46 for scans compared to 1:57 for prints). Most of that difference can be accounted for by the long time most who chose the Engine spent on their stories (longest average at 2:11 and longest story of all at 3:51).

Manipulating Physicality
We analysed the language people used to describe their interactions with the 3D prints and vitrine. Three categories emerged, Emotions and Collaboration in VR DIS '20, July 6-10, 2020, Eindhoven, Netherlands all of which can be seen to contribute to 'the rehabilitation of touch' [38]: speaking in terms of the physicality of the object such as its texture (15 participants), as a way of engaging directly with the object (touch facilitating engagement, 11 participants), and how touch enabled them to bring it close for a better view (touch facilitating visuals, 13 participants). These categories were not mutually exclusive, though for most people, one purpose dominated. We had expected complaints about texture, but the tracking of VR model to physical object was so accurate that it allowed participants' eyes to trump the evidence of their fingertips. Two discussed the texture of the wood (P21 for the Boar's Head and P3 for the Mystery Textile Equipment), despite both being made of plastic, and their most notable textures were artefacts of the printing process. Touch, especially perceived texture, was also a way of 'exploring' the object (P19, P24); clues as to age (P3), 'materiality' (P17), or 'structure' (P22); or pure pleasure: 'best time!' (P12).
Although several participants responded very matter-of-factly that touch was simply 'surprising' and made the experience more 'interesting' (e.g. P1, P4), several reached for more depth of meaning: 'it gives more of the things, of more involvement' (P12). For some it enabled 'a lot of freedom' (P2), 'a bit more of a real experience' (P20), or made the experience feel 'personal' (P12, P23). P19 exhibited interesting physical engagement with the Mystery Textile Object -spinning it in their hands and the like -to help determine what it might have been. Also, all but one of the participants with museum training followed correct object-handling protocols (P1, P14, P19, P20, P21) despite the fact that the object was a plastic replica. The physicality of the object implied a sense of reality that could amplify or influence the experience.

Manipulating Visuals
For many participants, touch was most impactful for changing the level of detail and angles from which the 3D prints or scans were experienced, such as P3's appreciation of 'moving them around to see the different angles'; or P22's desire 'to see the detail'. Two noted new thoughts arising in response to each new angle (P12, P11); another could imagine the object in use (P6); one described a greater 'level of response' (P8) and another 'a great benefit' (P18). For P9, even the 3D scan in the vitrine implied a sense of touch: it was something 'to look at and feel, not quite feel, but you know what I mean?' The engagement that visuals enabled -or disabled -is best seen in a mismatch between anticipated affordances of objects and the impossibility of physically manipulating a 3D scan in a vitrine. Three were disappointed when they found themselves unable to open the Pattern Maker's Toolbox (P3, P22, P24), though they may have been equally frustrated by a fixed 3D print. Future iterations must either manage expectations by choosing objects with no moving parts, or attempt to replicate movement. The vitrine seems to have acted on most participants in a similar way as the 3D prints did, in terms of enabling visual engagement and delivering a sense of solidity and reality, despite the touchable object being a box. Both the importance of visuals and the expectations of affordances arose through inductive analysis; neither had been foreseen by the designers.

Manipulating Scale
Several participants noted the changese in scale between the display cases in the static VR environment and the object they encountered. For some, it created a mental stumbling block that knowledge of the actual object's size would have prevented (P2, P5, P9, P19, P21). P6 objected to dramatically downscaling both the Engine and the Mask, even though the Mask in the vitrine was roughly the same size as the original. P6 wanted everything at the largest feasible scale 'because if you make it smaller, [it] is not going to be like the real object' regardless of the size of the real object. Of the 9 who disliked scale discrepancies, some lamented the less 'grandiose' (P2), 'magnificent' (P5), or 'immersive' (P19) hand-held versions, mostly of the Engine. Although 9 disliked discrepancies in scale, 8 did not notice or care, and 5 found it helpful or potentially helpful (P7, P8, P9, P10, P16). Of these, 3 (P7, P8, P9) concluded that whatever the drawbacks, the ability to have an overview of a large object has its benefits, too.
The roughly even split in responses leads us to conclude that we should clarify the actual size of each object, which will require rethinking their presentation in the VR environment. We would also like to offer people 3D printed sections of a large object alongside our handheld overviews to see whether a partial experience of its grandiosity has the effect they imagine. There are countless ways to approach these issues, both conceptually and practically, before drawing any solid conclusions from this analysis beyond the fact that in the moment, surprise and pleasure far outweighed concerns expressed in interviews. Those moments should not be lost.
To sum up our discussion of manipulations, 23 of our participants responded from neutrally to enthusiastically to their physical interactions with objects of different scales in VR. The exception was P5, whose deep-seated suspicions of technology VRtefacts could not overcome. Not one participant would have preferred object handling without the aim of telling a story, or storytelling without the object. We understand this to mean that our basic premise was sound, and that the boundaries of this design space are still a long way away.

Transitions
The headline for the design of the performative and performance-promoting experience through which participants transitioned is that it was rarely, if ever, commented upon unless in response to specific questions. This points to the possibility of latitude for more extreme transitions in future iterations. The framing of transitions through spaces and through storytelling that follows arose from inductive analysis of participant actions, reactions, and responses.

Transitioning between Spaces
Exploring how participants navigated between the physical and virtual environments sheds light on the importance of context and setting as well as on how coherent transitions can foster meaningful engagement between visitor and museum, potentially leading to long-term impact. Figure 4 shows the series of spatial transitions encountered by participants throughout VRtefacts. Transition 1 was the participant's move from the collection into the Art Room. This first transition was vital in order to enable the activation of the performance  context and provide time and space for telling personal stories. It was also vital not to make participants feel like they were being removed from the museum setting or distracted from their visit. For most, this transition was comparable to visiting other sections of the museum such as the 'stuffed animal part' (P4), though P17 and P22 felt isolated from the museum.
Transitions 2 and 3 were between the Art Room and the VR Donation Hall and then back. Several participants discussed these transitions in interviews. Those with less VR experience found them slightly more disconcerting (P5, P6, P11, P16, P17) than did those with more VR experience (P21, P22). Regardless of past experience, however, most expressed no discomfort beyond a 'weird' (P16) transition into the virtual environment, with only 2 (P5, P17) highlighting dissonance caused by the Donation Hall not being 'real' (P5 for their deepseated suspicions mentioned above, and P17 for the 'conflict between knowing you have your own space and there's still reality out there'). Nevertheless, all participants were able to quickly become immersed in the VR environment: I definitely felt myself disconnected from that kind of, sitting in the obvious chair in that room, which I knew was there, but I really kind of instantly got into being in the gallery space (P22).
We suggest that the museum-like aesthetic and explicit naming of the Donation Hall, as well as the narrative offered by the host, provides a consistency across spaces that eases the transition into and out of VR. The steady context across the museum visit acts as a means to smooth the 'seams' [9] and enable a coherent trajectory that affords creativity, engagement, and personal reflection [71,29]. Transition 4 was returning to the museum. Once again, the coherence of the context enabled participants to immediately apply their experience to the wider museum setting. For example, all 24 participants readily agreed to donate their stories to the museum, 18 expressed excitement or interest in the donated stories being interacted with by other visitors in the future, and 2 participants (P3, P7) expressed a desire to see their selected objects immediately after the experience. Almost all said that VRtefacts had positively affected their perception of Derby Museum, its objects, or both: I will probably notice [the objects] more now than I would have in the past, and I won't gloss over [them] (P12).
No one's perceptions of museum or object was negatively affected, and only 2 participants (P13, P19) expressed no change in their perception of either, both already having strong positive attitudes towards the museum. This overall positive impact also has implications for the final transition of leaving the museum setting, as the 22 participants who expressed a positive impact on their perception of the objects and/or the museum will, by their own accounts, see them differently now, or even '[spend] time looking at [other objects] more fully and in greater detail' (P18).
Exploring the navigation between physical and virtual spaces shows some scope for improvement of transitions. In order to address the concerns of the 2 participants who felt transition 1 was slightly isolating, the Art Room (and the corridor leading to it) could be treated, as P22 said, as a 'portal'. To achieve this, the 'separateness' of the Art Room must be reduced, and the museum context enhanced, for example through the use of formal signposting and consistent decoration. Transitions 2 and 3 would also benefit from the creation of a 'portal', whether in VR terms (e.g. [65]) or MR terms (e.g. [69]). This could be achieved through stronger environmental mirroring within the Art Room, or drawing on other elements of Substitutional Reality not explored in this study. Transitions 1 and 4 could be improved by introducing the donated stories to the general public within the exhibition space.

Transitions through Storytelling
Where Transitions through Space (see Figure 4) describes a canonical trajectory [10] for a VRtefacts participant moving through space, Transitions through Storytelling holds up a magnifying glass to their trajectory through Substitutional Reality (Stage 3 in Figure 4). Its transitions take place in the participant's own visual, haptic, mental, and emotional perceptions (see Figure 5).
The host's script is recapped here to signpost the transitions in this trajectory. They deviated from the script as necessary to guide each participant towards the canonical, or intended, trajectory. The host framed the VR experience by naming the space the 'Donation Hall', inviting participants to look around  at their leisure, and telling them that their thoughts were the high-value items in this collection. The host gave them time to situate themselves mentally and emotionally and then asked about their hobbies. The combination of solo exploration and self-oriented conversation was mirrored in Performative Substitutional Reality, where the room devoid of visible people was still shared by the host, whom the participant could hear and converse with. At this initial stage (see leftmost box in 5), formality and risk [73] were very low. Participants with a ready response to the host's questions probably also found consciousness and intensity levels to be low. The host gently nudged hesitant ones to consider everyday activities as hobbies. This invitation [73] to personal conversation set the participant on familiar conversational territory, where requests for personal information would be met with validation -a 'reward' [75] for the risk of sharing something about themselves.
The 1st transition (see 'I. Object Encounter' in 5) took place when the participant's encountered their chosen object as it materialised. 'Unconscious' exclamations of surprise predominated, and the host welcomed chatting. Formality stayed low or declined; risk and intensity stayed low or transferred to the object. The 2nd transition began once the initial surprise wore off. The host guided thoughtful exploration and story development. This shift of focus increased formality, intensity and risk. The host started extending the silences, giving participants time to think and preparing them for the long 'turn' of storytelling they were about to take. Consciousness of what they were planning to say was now required. Again, the host balanced these increases by encouraging and explicitly validating their responses. The 3rd transition occurred when the participant was ready to begin storytelling. The VR lights and camera snapped on, the host went silent, and participants performed at their own pace. Consciousness, formality, intensity, and risk were at their highest. Only when a participant clearly struggled did the host step in to prompt more of a contribution. The 4th transition happened when the participant stated or indi-cated through body language that they had finished. The lights switched off, the object dematerialised, and the host praised their story. The host either shifted back to the 1st stage for a second object and story, or invited them to take a last look around before leaving and confirming their wish to donate their story to the museum. The risk had passed; conversational norms returned.
As expected, participants had little if anything to say about the first 2 stages. The 3rd stage was the focal point for formulating stories. Touch was the most commonly named inspiration for story content (10), and 13 said that it helped them make and/or tell their stories. Touch influenced the length (5), depth (4), and/or detail (3) [75] of the storytelling they would soon be undertaking was higher earlier than we had anticipated.
In the 4th stage, participant trajectories tended to match the canonical trajectory with little intervention from the host. Participants nearly always risked divulging something of their personal life or ways of thinking (17 focused on personal memories, 9 on associations they made with the object, and 7 on imaginative or fictional scenarios made up on the spot). 'Personal' ranged from P14 mentioning a shirt they owned patterned with masks to a truly touching desire for P20's story to bring their partner to tears of joy about a happy memory

Emotions and Collaboration in VR
DIS '20, July 6-10, 2020, Eindhoven, Netherlands they share: 'I feel like I'd like him to cry at some point about this because he hasn't cried yet, so that might do it'.
Most either set the object on the table or held it nearly still, usually by the plinth, while telling their stories. Only P1 overtly gestured to the object during one story. However, almost all oriented themselves to the object while telling their stories, even when they sounded as though they were musing out loud to themselves. For some, speaking out loud seemed to trigger new associations in the moment, and participants transitioned from storytelling to thoughtful observation and back without loss of formality, risk, or intensity, though they may not have been fully conscious of their pauses.
While haptics generated opportunities for transition and engagement, the bright lights and camera in the VR did not increase consciousness, formality, intensity, or risk for 22 of our participants. The only one who mentioned it during the experience exclaimed 'How cute! I love it!' (P17), hardly the effect we desired. Many did not even wait for the lights and camera to turn on. However, P2 found that the visually increased 'pressure' made it 'quite hard to put it into a coherent story', while P21 said they would have told a less personal story to a group of co-located people. We conclude that our canonical journey was actually a series of micro-transitions strongly mediated by the object. The real scope for impact even in storytelling seems to be in relation to touch.

ON PERFORMATIVE SUBSTITUTIONAL REALITY
We are currently pursuing the immediate next step: scalable methods of sharing stories with the museum's visitors, whether in situ or online. We also wish to address some common complaints that can be solved by redesigning the VR environment. For example, we can show each object's true size and avoid the need to read grainy, distorted text in VR (known as the 'Screen Door' effect [25] and unfortunately inherent to the current generation VR HMDs). While the first implementation of VRtefacts employed several technologies near the forefront of digital innovation to create a cohesive, passive-haptics-enabled VR experience, future iterations could use technologies such as marker-less optical tracking and multi-material 3D printing to introduce options for weights, textures, and functionalities.
More compelling future design considerations for Performative Substitutional Reality centre on the role of the host and the use of Seamful Design [19]. First, having established the critical nature of the host in shaping the visitor experience and sometimes the nature, length, and depth of the stories, we now want to explore different manifestations of the host. For example, we felt that representing the host in VR would encourage two-way storytelling and distract the visitor from their tactile experience. If true, how would a more overtly conversational or interview-style approach affect the stories? This could be explored through performance analysis, content analysis, and interviews about the felt experience of telling and hearing stories, as well as any change in the percentage of visitors confirming their willingness to donate. How would different means to storytelling beyond simple exploration affect the experience and the stories? Could the host role be pre-recorded and activated by user interaction, not unlike a chatbot? Could the host's guidance be delivered purely through 'portals' to different areas within the VR environment that offer different types of interaction? Could small groups function as each other's hosts within the same VR space and co-create stories around the same object? There would undoubtedly be trade-offs, but perhaps benefits, in each of these approaches.
The prospect of a representation of the host within the VR environment leads seamlessly to the second most intriguing consideration for future work: Seamful Design [19]. A glitch in the representation of the object, like seeing it move out of arm's reach while holding it still (the most frequent glitch seen in VRtefacts), is easily overcome. But a glitch in the representation of the person you are speaking to might be detrimental to the visitor's ability to focus on their thoughts. We initially sought to avoid that seam altogether by not offering a visual representation of the host. As VR headsets in public spaces currently require at least some level of expert intervention to guide the uninitiated, the host also served to bridge the necessary seam of placing the HMD on the visitor and ensuring their comfort and safety. Perhaps anyone could be trained to 'perform' adequately in a minimised or altered host role, especially if a museum wishes to broaden the definition of 'storytelling' to incorporate a wider range of reactions. Future work could explore potential uses of such a seam.
Other key elements that could be understood and explored as seams include scale and materials. We intentionally changed the scale of most objects to make them available for complete tactile and visual examination. What would be the effect of allowing visitors to change scale in the VR while the object remained static? What other elements of the VR might then be rescaled, such as the potential representations of their hands? Might the room shrink to emphasise an increased object size? Where would the fidelity of the 3D model break the illusion? And how would such interactions enhance or interfere with the assumed 'seamlessness' of a typical curatorial presentation?
The materials used to create 3D prints also presented obvious 'seams' in the MR experience, as none of the original objects was made of, or felt like, plastic. Stronger, more expensive materials would broaden the scope of possible objects, such as the long-handled mower, and may better indicate relative weights, changes in density, textures, etc., that could then be used to create intentional seams within the experience. Details can be enhanced, reduced, or artistically modified to draw attention (or imagination) to specific features. Individual elements of a complex object could be visually magnified where the visitor touches an area of enhanced detail.
These are only the initial thoughts inspired by the VRtefacts design: VR, MR, and performance-led researchers can undoubtedly come up with hundreds more. Through the 'rehabilitation of touch' [38], VRtefacts inspired 'object' handling and visitor interpretation using Performative Substitutional Reality.