Design Fiction for Mixed-Reality Performances

Designing for mixed-reality performances is challenging both in terms of technology design, and in terms of understanding the interplay between technology, narration, and (the outcomes of) audience interactions. This complexity also stems from the variety of roles in the creative team often entailing technology designers, artists, directors, producers, set-designers and performers. In this multidisciplinary, one-day workshop, we seek to bring together HCI scholars, designers, artists, and curators to explore the potential provided by Design Fiction as a method to generate ideas for Mixed-Reality Performance (MRP) through various archetypes including scripts, programs, and posters. By drawing attention to novel interactive technologies, such as bio-sensors and environmental IoT, we seek to generate design fiction scenarios capturing the aesthetic and interactive potential for mixed-reality performances, as well as the challenges to gain access to audience members' data -- i.e. physiological states, daily routines, conversations, etc.


Introduction
Creating an engaging and innovative mixed reality performance (MRP) requires a shared understandings of the interplay between aesthetic, narrative, and technological possibilities amongst multi-disciplinary teams. If artists, directors, producers, set-designers and performers are to develop meaningful creative dialogues with novel, often experimental, technology, methods are required that enable them to interrogate the material properties of the technology at hand. Similarly, if interaction designers, programmers or hardware engineers are to meaningfully contribute to the development of performances, methods are needed that enable them to understand, and make concrete design judgements in response to the subtle issues and aspirations that drive creative processes and practices. In the words of John Lasseter [11], circumstances must be brought about wherein "The art challenges the technology, and the technology inspires the art".
A number of research projects have sought to address the tension between MRPs and the design of interactive technology [2,10,12]. Throughout this one-day workshop we intend to further discuss this tension by using Design Fiction to generate scenarios for MRPs. Design Fiction is based on a language of visual imagery and narrative (e.g. imaginary storyboards, scripts, vignettes) that is familiar to performance producers. This provides potential for this method to be successfully incorporated into existing creative processes used to author MRPs. Such design fictions can therefore act as boundary objects that enable technology designers to communicate creative possibilities in a form familiar to creative practitioners, and vice versa. Design fiction also encourages a holistic perspective on the fictional reality that would surround the artefact, providing a framework to explore scenarios involving Internet of Things (IoT) and biological sensors embedded in the environment for a variety of other purposes to be used in MRP.

Background
In the sections below, we introduce the three areas of research that set the background of this planned workshop;

Mixed-Reality Performance
Artists, designers and researchers within HCI have focused on creating MRPs, both for the value of the performance in itself, but also to better understand audience impact [1,14], technology infrastructure [5], and theoretical aspects of interactive theatre [9]. The focus on participation-through-technology in MRPs has also been seen as a way to understand other people and engage in a dialogue with them [8]. Humanaquarium [17] is an example of HCI research exploring the collaboration and creative agency in participatory musical encounters. As performer/researchers, they took part in the experience first-hand, inviting participants to 'jam' with them via a window interface that translated touch into audio-visual response. Several performances discussed in HCI literature include interactive experiences using (at the time) novel technologies such as location tracking [4,5], virtual reality [13] and mobile devices [3]. Current work includes performances that exchange the visual perspective of performers and audience members, and those that immerse participants in locative audio environments for the purpose of creating and exchanging gifts of digital music (see, for example, www.semanticaudio.ac.uk).

Bio-Sensing and IoT
While a lot of previous MRPs involve mobile, locationbased and other ubicomp technologies, more recently bio-sensing had entered the space of participatory experiences (e.g. see [15]). Bio-sensing is compelling for MRPs, making use of traditionally personal data (a racing heartbeat can only be felt by the individual) as material for performance [16]. This potentially uncomfortable experience, which makes users vulnerable during interaction, can be used to create deeply meaningful experiences [6].
Real-world uses of bio-sensors and environmental IoT sensors in interactive performances have been limited partly because the technology has not been readily/commercially available for real-world productions, often produced under time-and budget constraints. However, recently hardware prototyping toolkits have encompassed bio-sensing technologies and IoT [18,19] lowering both the cost and the time to develop.

Design Fiction
Imagining the social and cultural issues emerging from technology use, as well as the changes technology can being to our lives can be challenging. Design fiction [20] addresses such challenges by creating descriptions of fictional futures expressed as everyday artefacts, such product catalogues, newspaper articles, technical manuals and so forth. A recent example is the Internet of Things vision seen through an IKEA Catalogue produced at Mobile Life Centre [7]. In relation to designing MRPs, the fictional framing of design fiction can encourage playfulness, experimentation and the consideration of challenging, strange and otherwise unorthodox uses of technology -which might, in turn, facilitate more creative ideation. By framing ideation as an imaginary story, fictions also compel us to consider experience in a holistic and longitudinal fashion by e.g. considering long term consequences of a performance experience on an audience for example.

Workshop Goals
This workshop aims at bringing together an international, and cross-disciplinary working group for exploring the use of design fiction to create concepts for mixed-reality performances. Its goal is to explore the role of design fiction as a method for conceiving and designing MRPs through fictional scripts, programs, tickets, posters and other archetypes. The goal of the workshop is to enable a discussion with a wider community centred on the production of a number of fictional performance tropes, metaphors, and interaction modalities drawing on sensor-based interactions and IoT. The workshop themes will evolve from, but are not limited to: § Current research being conducted in the area of MRP; § The methodological challenges of deploying Design

Target audience
This one-day workshop invites researchers in MRP, designers, artists, curators but also to those who wish to understand the issues and current state-of-the art research and development. We plan to accept a maximum number of 25 participants.

Submission
The contributors are invited to submit to submit a 2-4 page position paper in CHI ACM Extended Abstract Format, a 3-5 minute video, or 2-4 page pictorial. This is designed to encourage artists, performers, and designers to attend the workshop. The organizers will review the submissions which will be selected based on relevance and innovativeness. The selection process will ensure the presence of different perspectives in order to foster discussion. Accepted submissions will be made available on the website for participants prior to attending the workshop.

Pre-Workshop Plans
A call for papers will be sent to relevant lists (i.e. CHIannounce), and we will personally contact researchers, designers and other practitioners we know could be interested in this event workshop. We will also advertise the workshop through social media platforms blogs and email, and contact artist groups (i.e. Bombina Bombast, Satan's Democracy, RATS Theater, Transmediale, Blast Theory, etc.). This workshop is being planned in parallel with one to be held over two days in Stockholm in March,2017. The results of this earlier workshop will be made available on the website to encourage and inspire later participants.

Workshop Structure
The agenda for this one-day workshop entails a number of activities meant to promote discussion and exchange of ideas.

Introduction (45 minutes)
The workshop will start by organizers with an overview of the workshop schedule and its goals. The participants then, will briefly introduce themselves and how they relate to this workshop.
Keynote (60 minute) The workshop will then continue with invited keynotes (two of the organizes) briefly reviewing current mixedreality performances and introducing the Design Fiction method to provoke participants' creativity and imagination. This will be followed by a coffee break.
Group activity (60 minutes) Participants will be placed into pre-arranged groups based on their submissions and backgrounds, in order to ensure a variety of points of views and expertise in each working group. The first group exercise will run until lunch and focus on working together to explore the possibilities afforded by the sensors. Each group will be given sample output from fictional sensors and data aggregators (from biometric, sociometric, and social media sources) and a selection of past mixedreality performances. Their goal here will be to brainstorm and discuss what potential archetypes they can design for a fictional performance based on the sensors.
Design fiction (120 minutes) After lunch the participants will start generating ideas for their selected archetype from the first session. The participants will focus on feeding the previous discussion back into the performative ideas generated in the morning. In doing so they will be encouraged to produce sketches or video recordings as documentation. Each team is asked to design at least one archetype based on the emerging topics at the workshop. The groups will also be asked to discuss and document their visions of the wider impact of the adoption of one or two of their ideas -on the individual audience member, the artist, and the wider society.
Presentation (45 minutes) After a coffee break, all participants will gather together, to present their designed archetype and its emerging issues.
Discussion (60 minutes) The workshop will conclude by group discussion, retelling what were the challenges, what has been achieved, with reflect on workshop goals.
Informal dinner The participants will be invited to have an informal dinner at nearby restaurant.

Post-Workshop Plans
The output of the groups will be collected, digitalized and polished by the organisers and made available on the workshop website. After the two planned workshops, the organizers will actively seek to publish an experience piece in Interactions. We also plan to edit a special issue on a selected journal.

Workshop Website
The website will be available at https://mixedreality.tech.blog. It will provide examples of design fiction, mixed reality performances, and sensing technologies. It will also host the results of the earlier workshop. Upon acceptance position papers will be published with an area for public discussion (seeded by the organisers). After the conference it will host the artefacts created during the workshop.

Call for Participation
This 1-day workshop brings together an international, and cross-disciplinary working group to explore the use of design fiction to create concepts for mixed-reality performances. By drawing attention to novel interactive technologies, such as bio-sensors and environmental IoT, we seek to generate design fiction scenarios capturing the aesthetic and interactive potential for mixed-reality performances, as well as the challenges to gain access to audience members' data -i.e. physiological states, daily routines, conversations, etc. During the workshop we will collaboratively generate and design fictional scripts, programs, tickets, posters and other archetypes of performances that take advantage of environmental IoT and personal biological sensor devices. This will involve the production and discussion of fictional performance tropes, metaphors, and interaction modalities drawing on sensor-based interactions and IoT.
We invite academics, performers, designers, and artists to submit a 2-4 page position paper in CHI ACM Extended Abstract Format, a 3-5 minute video, or 2-4 page pictorial. Submissions are welcome on related topics such as: mixed-reality performances, design fiction, or sensors data in interaction. The deadline for submissions will be 24th February 2017 and at least one author of each accepted position paper must attend the workshop and register for one day of the conference.
Accepted position papers and demos will be published on the workshop website prior to the start of the event for attendants to read and discuss. For more information please visit the workshop website at https://mixedreality.tech.blog. Please direct queries and paper submissions to (asreen@dsv.su.se).

Organizers
Asreen Rostami (primary contact) is a PhD candidate in HCI at Stockholm University. Her research focuses on technology for performance art and theatre and how interactive technologies can be designed for mixedreality performances. Her current research looks at immersive theatre, performers interaction and audience interaction in traditional and interactive settings with respect to technologies such as VR and IoT. Chiara Rossitto is a senior lecturer of HCI at Stockholm University. Her research is characterized by a combination of social theory and empirical investigations to understand the cultural and social impact of technology use. Her work on interactive performance has looked at critical qualities of audience participation, at the design of performances as means of civic engagement, as well as rehearsal practices for MRPs. Louise Barkhuus is an associate professor in user experience at the IT University of Copenhagen. Her research focuses on location-based systems and experiences with novel location technologies. Most related to the workshop, she recently followed a series of interactive theatre plays for the studying of issues such as rehearsal for mixed reality performances and civic participation as interactive factors in theater performances. Jarmo Laaksolahti is a senior researcher at the Mobile Life Centre and a Lecturer of Interaction Design at KTH Royal Institute of Technology. His research explores how we can learn about the properties of digital materials and successfully use them in creative design. He has also worked on several design fiction projects including a fictional IKEA catalogue envisioning an IoT future, a brochure envisioning how technology will change services provided by the city of Stockholm, and a future course catalogue for KTH envisioning how technology will change education. Donald McMillan is a senior researcher at the Mobile Life Centre and a senior lecturer of HCI at Stockholm University. His research explores the impact of mobile technology on social interaction. His current research project focuses on interactions with, and impacts of, personal sensor data. He has also published on the ethical challenges of deploying and researching such invasive tracking technologies. Julie Williamson is a Lecturer of Human Computer Interaction at the University of Glasgow. Her research explores how tangible performative interactions can be embedded into public places, focusing on ways of attracting users, encouraging playful behaviour, and evaluating user experience without intervening during users' interactions. Robyn Taylor is a Canadian researcher and musician working in Open Lab at Newcastle University. Robyn uses participatory performance as an investigatory medium, inviting members of the public to engage in improvisational performances. Through staging and evaluating playful collaborative experiences, she explores issues in human-computer interaction surrounding creative agency and social encounters in public spaces. Jonathan Hook is a Lecturer in Interactive Media at the University of York. His research uses humancentred design methods to explore how novel interactive technologies can support new forms of creative content and practice. Jocelyn Spence is internationally recognised in the fields of human-computer interaction and performance studies. Her framework of Performative Experience Design has formed the basis for projects discussed in the interaction design and intermedial performance academic communities. She has advocated the use of performance in interaction and experience design through her contributions to the CHI Digital Arts Community.