I just returned from our first Reality Remix workshop in Dundee, Scotland. The prompt that we gave ourselves afterwards was to write up things that came up for us, returning thoughts and think about what is next. I write now on the train journey back to London.

The background is that Reality Remix is part of an Arts & Humanities and Research Council AHRC grant around Immersive Experiences and I am one of the “collaborators” (artists) — others include Ruth Gibson and Bruno Martelli (Ruth is the Principal Investigator), Joe DeLappe, Darshana Jayemanne, Alexa Pollman and Dustin Freeman. We are also working with several “partners”, who are in academia, industry and government to act as advisors and contribute in various ways. These are Nicolas Lambert (Ravensbourne University), Lauren Wright (Siobhan Davies Dance), Alex Woolner (Ads Reality) and Paul Callaghan (British Council).

The short project description is:

Reality Remix will explore how we move in and around the new spaces that emergent technologies afford. Through the development and examination of a group of prototypes, initiated from notions of Memory, Place and Performance and with a team of artists, computer programmers, fashion and game designers, we aim to uncover the mysteries of these new encounters, focussing on human behaviour, modes of moving, and kinaesthetic response. Reality Remix brings a unique dance perspective in the form of somatic enquiry that questions concepts of embodiment, sensory awareness, performance strategy, choreographic patterning and the value of touch in virtual worlds.

Within this framework, each of us will be developing our own VR/AR projects and possible collaborations might arise in the process.

Some of the reasons that I was invited to be part of this project stem from core inquiries about what we call “Gibsonian” cyberspace versus a simulated cyberspace. I find it odd that when we often depict virtual reality — and for the purposes of simplicity, I will treat VR a subset of cyberspace — as a simulation, a weak reproduction of some sort of physical reality. VR has immense possibilities that most people don’t tap into. With the dominance of first-person shooter games, reproductions of museums, and non-spaces such as TiltBrush, I have often wondered about how we can conceptualize VR landscapes in new ways. And so, this was my starting point for our first session.

Improving the functionality of the headset

However, technical skills are not a precursor to producing compelling work and, in fact, this is part of my artistic practice. I quickly adapt. For example in 2014, I quickly dove into 3D printing without knowledge of any real 3D modeling package and in the space of a few months, produced some conceptually-driven 3D print work that drew strong responses. I will easily pick up Unity, Unreal, 3ds Max or whatever else is needed.

It is with this lack of technical knowledge, that I can approach concepts with a beginner’s mind, a core concept of Buddhist thinking where you approach a situation without preconceptions and harvest a disposition of openness. Without deep investment in the structures of discourse, it is here that you can ask questions about the effectiveness of the technology such the nature of immersive spaces, the bodiless hands of VR and hyperreality.

For this initial meeting, we each have our own project ideas that we will be researching and producing in various forms. Some of my own inquiries stem from these Gibsonian landscapes. On the train I re-read Neuromancer and was still inspired by this seminal quote from Neuromancer:

Cyberspace. A consensual hallucination experienced daily by billions of legitimate operators, in every nation, by children being taught mathematical concepts… A graphic representation of data abstracted from banks of every computer in the human system. Unthinkable complexity. Lines of light ranged in the nonspace of the mind, clusters and constellations of data. Like city lights, receding…

I arrived with this general framework and began to ponder about how to incorporate threads of previous work such as physical data-visualizations and network performance. What about the apparatus of the headset itself? How can we play with the fact that we are partially in real space and partially out. And as one of our partners (Alex) pointed out rather than being immersed in VR, we are absorbed by it. Like a fish in water, we are live in full reality immersion. And when we talked about this, I chuckled to myself, remembering this David Foster Wallace joke:

There are these two young fish swimming along and they happen to meet an older fish swimming the other way, who nods at them and says “Morning, boys. How’s the water?” And the two young fish swim on for a bit, and then eventually one of them looks over at the other and goes “What the hell is water?

The first day was a full-out setup day, installing our Windows work machines, getting the Oculus Rifts working, Google Pixel 2 phones, Unreal, Unity and anything else. We all centralized on some specific platforms so we can easily share work with each other and invite possible collaborations. Fighting with a slow wi-fi connection was the biggest challenge here but within a few hours I got my Alienware work machine making crude VR on the Oculus.

Goodies! This is the Oculus Rift hardware that we will all be using for project production

I yearned to visit Dundee despite the cold weather but all of our time was spent in this workspace, supported by the NEoN Digital Arts Festival (where I showed my Strewn Fields pieces last November) and in the evenings, we saw some art lectures by our own Joe DeLappe and my friend and former colleague Sarah Brin. We are food and drank in the local speakeasy bar, chitchatting about ideas.

Lecture by Joe DeLappe

Within the first couple of days, what was previously a mystery to me now became more clear. While some of the collaborators had well-developed projects (Dustin and Bruno/Ruth), others such as myself, Alexa and Joe were approaching it from a more conceptual angle with less technical aptitude.

I kept in mind that our goals for this project are to create compelling proof-of-concepts rather than finalized work, which makes it more of a research-oriented project with a forward-face rather than something that will compete in the already littered landscape of the good, bad and ugly of the Oculus Store.

We started each day with movement exercises led by Ruth, reminding us that we all live in “meatspace”. Our minds and bodies are not separate as we hunch over the machines and then stand with a headset on and wave our arms around. We began experimenting with the technology. I vacillated between diving into Unreal or Unity, each with their own advantages. While Unity has more generalized support and is easier to learn, Unreal undoubtedly has a better graphics engine and is the weapon-of-choice by Bruno. So, solving the early technical challenges began to help coalesce my ideas.

Ruth leading us on some Skinner Releasing exercises

We soon entered into a vortex of artistic energy — some of us from performance, wearables, immersive theater with various conceptual practices and the parters from other organizations who had a less artistic approach but loads of experience in the gaming worlds, university community and impact studies. I knew this on paper, but in reality, the various talents of our Reality Remix dream team soon became apparent.

Twice a day, each of the collaborators led workshops related to their practices. Alexa treated us to her performance-based clothing which registered AR markers and asked us to do an exercise where we tried to perceive something through someone else. Bruno and I made a drawing where he wore the VR headset and I sketched on his back which he replicated on paper. The process was fun and predictability the drawing was unimpressive. The process was fun and predictability the drawing was unimpressive.

Ruth is wearing fabric created by Alexa while she shows us some of her responsive AR augments

Our collaborative drawing in response to Alexa’s prompt

Dustin led us on a prototype for a sort of semi-immersive experience where actors jump into various avatars. With his deep background of improv, theater and role-playing, I began to shift, thinking about how to involve people not in a headset, which make up the majority of people in a VR experience as essential players.

Darshana in the VR headset while Dustin demonstrates some ways in which “non-players” can interact in VR space

I am not wondering about intimacy and vulnerability in VR. There is a certain amount of trust in this space. You are blind and often suffused in another audio dimension. Then, could you guide people through a VR experience like a child in a baby carriage? What can be done with multiple actors? So many questions. So many possibilities.

One thing that I was reminded of was the effectiveness of simple paper-prototyping and physical movement. Make things free from technology; keep it accessible. Stick to the concepts.

My own orientation began shifting more into virtual landscapes and thinking about data as the generator. I asked people to brainstorm various datasets and come up with some abstract representations based on that earlier quote from Neuromancer. I do want to get away from the sci-fi notion of cyberspace since it is limiting and enmeshed in VR 1.0, but will still claim this as the starting point to an antidote to the often mundane reality-simulation space.

This was useful for my own brainstorming. Alexa brought in an interesting point of view because she was thinking about time rather than landscape and brought in conversations around anticipation, reality and memory, which reminded me of the work by Bergson. Her movements were around personal data and captured her attention.

Meanwhile, Ruth made marks on the wall, translating gesture to 2D. Bruno worked with abstract visual forms. Despite being a poor draftsman, the question arose: how can we incorporate movement into a system? My own perception is highly visual and orientation towards abstract patterns. The success of my exercise was based in the fact that some useful (to me) renderings were produced while I also quickly learned that the a line-based VR landscape doesn’t resonate with everyone. How can we incorporate movement into a system.

Drawings by Bruno Martelli in response to my workshop prompt

As a manifesto bullet item: the scriptedness of VR is something we would all like to break. With all the possibilities of VR, why are the dominant forms assuming a feeling of immersion. Why don’t we consider what can be done before rushing to produce so much content.

Where is the element of surprise? VR is a solitary experience. I’m reminded of Joe’s work with the military and what one can do with a gaming space. Could we intervene or somehow interfere with VR space?

Darshana, the theorist amongst the group, did a beautiful summary of the session. My head was spinning with ideas at this point, so I can’t even recall everything he spoke about but certainly ideas around how to both be engaging and critical in this space surface. He envisioned a nexus around abstract spatialization, performance, role play and the body that tied our various projects together.

Bruno, Ruth and Alexa wearing some of Bruno + Ruth’s Dazzle Masks

There is much more to write and think about. I made progress on the technical side of things, such as getting an OSC pathway to Unreal working, so that can begin playing with electronic interfaces into a VR world.

More importantly, I feel like I’ve found my people with this Reality Remix team: one where we understand the history of new media, subversion of forms, aren’t dazzled by simplicity. We got along so very well with mutual respect and laughter. I’m excited about what comes next.

Reality Remix group phptp