Flagscape: Data-visualizing Global Economic Exchange in Virtual Reality

Overview

Scott Kildall is conducting research into data-navigation techniques in virtual reality with a project called Flagscape, which constructs a surreal world of economic exchange between nations, based on United Nations data.

The work deploys “data bodies,” which represent exports such as metal ores and fossil fuels that move through space and impart complexities of economic relations. Viewers move through the procedurally-generated datascape rather than acting upon the data elements, inverting the common paradigm of legible and controlled data access.

Economic exchange in VR

Details

The code constructs data from several databases at runtime including population, carbon emissions per capita, military personnel per capita and a United Nations database on resource extraction. All of these get combined to construct the Flagscape data bodies. Each one represents a single datum, linked to a specific country.

The only stationary data body is a population model for each country, which scales to the relative value for each country and resembles a 3D person using a revolve around a central axis. The code positions these forms at their appropriate 3D world location, such that China and India — the largest two population bodies — act as waypoints as their forms dwarf all others.

Population bodies of India and China

A moshed flag skins every data body, acting as a glitched representation that subverts its own national identity. Underneath the flag is a complex set of relations of exchange that exceeds nationhood. For example, resource-extraction machines are made in one country that then get purchased by another to extract the very resources that make those machines.

Brazil flag, moshed

Flagscape reminds us that our borders are imaginary and in this idealized 3D space, there are no delineations of territory, only lines that guide trade between countries, forms magically gliding along an invisible path. What the database cannot tell us is how exactly the complex power relations move resources from one nation to another. Meanwhile, carbon emissions, the only untethered data body in Flagscape, which affects the entire planet spin out of control into the distance only to get endlessly respawned.

Carbon emissions by Canada and Australia

The primary acoustic element triggers when you navigate close to a population body. That country’s national anthem plays, filling your ears with a wash of drums, horns and militaristic melodies that flow into a state of sameness.

Initial Inspiration

The project is inspired by early notions of cyberspace described by writers such as William Gibson, where virtual reality is a space of infinity and abstraction. In Neuromancer, published in 1984, he describes cyberspace as:

“Unthinkable complexity. Lines of light ranged in the nonspace of the mind, clusters and constellations of data. Like city lights, receding…”

Neuromancer

While this text entices, most VR content recreates physical spaces, such as the British Museum with the same artwork, floor tiles and walls as the real, or it builds militarized spaces in which “you” are a set of hands that trigger weapons as you walk through combat mazes. At some level, this is a consequence of linear-thinking embedded in our fast-paced capitalist economy, arcing towards functionality but ignoring artistic possibilities. This research project acts as an antidote to these constrained environments.

OverkillVR, a virtual reality game

It was with these initial conversations around virtual datascapes with Ruth Gibson and Bruno Martelli that I was invited to be part of the Reality Remix project and was included in the AHRC Reality Remix grant which is part of their Next Generation Immersive Experiences call. My role is a “collaborator” (aka artist) who is creating their own project under these auspices.

Spatialization and Materializing Data

Unlike the 2D screen, which has a flatness and everyday familiarity, VR offers full spatialization and a new form of non-materiality, which Flagscapes fully plays with. One concept that I have been working with is that since data has physical consequences, it should exist as a “real” object. This project will expand this idea but will also blur sensorial experiences, tricking the visitor into a boundary zone of the non-material.

At the same time, Flasgscapes is its own form of landscape, creating an entire universe of possibility. It refers to traditions of depicting landscapes as art objects as well as iconic Earthworks pieces such as Spiral Jetty, where the Earth itself acts as a canvas. However, this type of datascape will be entirely infinite, like the boundaries of the imagination.

Spiral Jetty

Finally, Flagscape continues the steam of instruction-based work by artists such as Sol LeWitt, where an algorithm rather than the artist creates the work. Here, it accomplishes a few things such as taking the artists hand away from creating the form itself but also recognizing the power of artificial intelligence to assist in creating new forms of artwork.

Alternate Conception of Space in Virtual Reality

VR offers many unique forms of interaction, perception and immersion, but one aspect that defines it is the alternate sense of space. Similar to the religious spaces before the dominance of science, as described by Margaret Wertheim in the Pearly Gates of Cyberspace, this “other” space has the potential to create a set of rules that transport us to a unique imagination space.

As technology progresses and culture responds, the linearity of engineering-thinking often confines creativity rather than enhances it. Capitalist spaces get replicated and modified to adapt to the technology, validating McLuhan’s predictions of instantaneous, group-like thinking. The swipe gestures we use on our phones get encoded in muscle memory. We slyly refer to Wikipedia as the “wonder-killer”. The flying car is often cited as the most desirable future invention.

Flying car from Blade Runner

At stake with technological progress is imagination itself. Will the content of the spaces that get opened up with new technologies be ones that enhance our creativity or dull it? Who has access to technology-inspired culture? How can we use, enhance and subvert online distribution channels? These are just some of the questions and conversations that this project will ask — in the context of virtual space.

I see VR in a similar place as Video Art was in the 1970s, which thrived with access to affordable camcorders. However, VR and this specific project has the ability to easily disseminate into homes and public spaces through various app stores. Ultimately, with this project I hope to direct conversations around access and imagination with art and technology.

Marshall McLuhan with many telephones

Work-in-progress Presentation

Our Reality Remix group will be presenting its research, proof-of-concepts and prototypes at two venues in London on July 27th and July 28th, 2018 at Ravensbourne and Siobhan Davies Studios. Both free events are open to the public.

Bibliography
Gibson, W. (1993). Neuromancer. London: Harper Collins Science Fiction & Fantasy.
McLuhan, M. (1967). The medium is the massage : an inventory of effects. Bantam Books.
Wertheim, M. (2010). The pearly gates of cyberspace. New York [u.a.]: Norton.

Revamping Moon v Earth

My artwork occupies the space between the digital and analog as I generate physical expressions of the virtual. In the last several years, most of my work with transforming data into sculptures and installations.

But sometimes I return to narratives themselves. It’s not so much a lack of focus but rather a continual inquiry into technology and its social expression. Imaginary narratives seem particularly relevant these days with the subjectivity of truth magnifying an already polarized political discourse.

I recently finished revamping a project called Moon v Earth, originally presented in 2012 at the Adler Planetary Museum. This augmented reality artwork installation depicts a future narrative where a moon colony run by elites declares its independence from Earth. It is now on display at the Institute of Contemporary Art in San Jose.

Here are a few augments from the 2012 exhibition that made it in the 2018 show. My favorite was this pair of newspapers, which showed two different ‘truths’. At the time, “fake news” meant nothing and the idea of seeding false stories into online outlets wasn’t a remarkable.

The last augment — the ridiculous wooden catapult about to launch rocks at Earth — refers to the Robert Heinlein novel, The Moon is a Harsh Mistress. This inspired the my project many years ago. In his plot line, the moon was a penal colony much like Australia 200 years ago and features an AI as one of the three heads of the revolution. The independence-seekers achieved victory by hurled asteroids at Earth as their most effective weapon.

I created this absurd 3D model in the imaginary world of Second Life as an amateur 3D assemblage. It was quick and dirty, like much digital artwork and as we see nowadays, like the fragility of truth.

The turn of Moon v Earth, at least the 2018 version is that the augments aren’t virtual at all, but instead are constructed as physical augments hanging from fishing line or hot-glued against a cardboard backing. At first, I tried working with AR technology, but soon discovered its compromises: a device-dependence and a distance between the viewer and the experience. Instead, the physical objects shows the fragile and fragmentary nature of the work in cheap cardboard facades and flimsy hanging structures distributed throughout the venue.

NextNewGames is at the San Jose ICA until September 16th, 2018

Farewell, Dinacon

I just spent 20 days on a sparsely-inhabited island in Thailand with about 80 artists, scientists and other imaginative people. Everyone worked on their own projects ranging from jungle-foraged dinners to plant-piloted drones to creating batteries from microbial energy. We had no AC for much of the day, got bitten by weaver ants, were surrounded by jungle cats and ate off each other’s plates. And, I absolutely loved the experience.

Microbial Battery Workshop

The gathering was Dinacon, the first Digital Naturalism conference and was co-organized by Tasneem Khan and Andrew Quitmeyer. I was a “node leader” which meant that I spent a bit of time reviewing the applications, organizing workshops and spending longer at the event than most.

Dinacon registration area

The site was Koh Lon, a small island that is just off the coast of Phuket. We stayed at a “resort”, which was actually fairly minimal and had small cabins, common house or options for tent camping. From the main work area, you walk a few minutes in one direction and you’re on the beach. In the other direction is jungle. There were no cars on the island, a handful of scooters, two hundred or so local residents and not a single dog. The soundscape felt entirely tropical with cicadas, birds and frogs filling the airwaves with their chatter. Our dinner was boated in each day and at the small restaurant we could get the three essentials: wifi (when the power was on), breakfast + lunch, and beer.

Selfie with Koh Lon in the background

The participants came from all over the world and arrived and left at random times such that there was constant inflow of new friends and outflow of sad goodbyes. Each day, we had about 40 people on the island. I could nerd out on my project, kayak in the water, take a break on the ship that we had access to (Diva Andaman), find myself sitting in a chair sharing ideas, play with hermit crabs or get away from everyone and walk in the jungle. Helping one another was something that effortlessly emerged in our temporary community.

Saying hi to the Diva Andaman

Questions that I asked myself upon arrival was will happen when you assemble a group of project-creating strangers in a natural environment, where you can take a break by putting on a pair of swim trunks and walk into the ocean? What does building things in on the island with its outdoor space and natural light do to your creative practice? How can I prototype an artwork that collaborates with this specific place?

I quickly became a lot less efficient and much more connected to people and place. I ended up creating better work and my body was utterly relaxed. Any shoulder pain I might have in an office space dissipated quickly. There were no Google calendar invites, no afternoon soy lattes and certainly no eating at my desk.

I found myself in daily arrhythmic patterns of production, often sitting on my neighbor’s the porch with headphones on and composing audio synth code, then stopping suddenly and reveling in nature. I would get interrupted to see a tree snake or find myself lost in conversation with someone’s project. In the evenings, we usually had self-organized small workshops or informal talks. I drank beer sometimes but also often went to bed early, worn out from the humidity and brain swell each day.

Arduino coding by susnet

I did make a thing! This experiment — a potential new artwork has a working title of DinaSynth Quartet. It is a live audio-synth performance between a plant, the soil, the air and the water, which is an electronics installation that is designed exist only outside. I connected each of these four “players” to sensors: plant with electrodes, ground to soil sensor, water to EC sensor and air to humidity.

Each one used a variation on my Sonaqua boards — a kit which I am actively using for workshops — to make a dynamic audio synth track, modulating bleeps and clicks to their sensor readings, creating a concert performance of sorts.

I’m not sure exactly where the work will go next, but I’m happy with the results. It was my first audio synth project and I’m far from being an expert, calling my approach “beginner’s mind”. However, most of the participants liked the idea and the specific composition that the jungle played.

I already miss everyone there: Jen, Tina, David, Rana, Pom, Sebastian and so, so many other delightful friends. And this is one thing I love about the life I’ve created for myself as an new media artist: after events like this, I now have friends who are doing inspiring work all over the world.

Jungle-foraged dinner party

 

Kira’s birthday party

Putting on a heart rate sensor on one of the local cats

Sonaqua Workshop in the common space

Local lotus flower

 

Millipedes were everywhere

 

Soldering work in the main space

Dani doing a lizard dissection on the beach at susnet

 

Andrew holding a snake

 

Little Niko, my favorite of the Dinacon Cats