Light Years Coast VR & 360

Software Art

In 2018 Anthony was appointed as an Immersion Fellow for the South West Creative Technology Network . The role involved exploring the possibilities of several forms of immersive media in order to better understand the potential and correct uses of such media. It was study to find out the differences between these different forms, by directly comparing the same project, output through different forms of 360/virtual reality.

To achieve the comparison, Anthony took his ongoing project, Light Years: Coast and updated it (previously not updated since 2014) in order to make three compatible versions, a large projected version, a 360 video version and a full stereo VR version. Light Years: Coast is a virtual painting, an semi-interactive exploration of the Dorset coast, featuring paintings by Jeremy Gardiner. It is a project that originally started in 2002, with several incarnations as an immersive, projected experience. As an experience, it involved a 15 minute boat-like trip up and down the coast, as a virtual gallery, seeing landscape paintings in the approximate location of the subjects they featured. In Light Years: Coast weather data affects the environment, causing tides and waves to change, rain to fall. Additionally sound recordings at further sensory input. All the landscape (gathered from LIDAR data) of the coast is covered in elements of painting, and the sky is also painting too, gradually fading from one sky to another. Light Years: Coast is about representation of time in a landscape, from current time, to geological time, to sped-up time and seasons.

Screen grab of the 360 degree Light Years: Coast

Light Years: Coast is a piece of software art created in a game engine, and it needed some updating and adaptation to work with the VR headset and updates in the development environment it was created in. In theory, converting a 3D experience into a virtual reality experience is straightforward, as VR is just a stereo virtual camera, that takes it position and rotation information from the headset, but in practice, working on a project that was initially programmed 10 years earlier was more complicated than at first appeared. Anthony worked with his assistant Emma Klasse and used an HTC Vive Pro headset to create a virtual reality version.

The 2010-2014 Light Years: Coast experience was a boat trip along the coast and as mentioned, the intention was to convert the project to a VR boat trip. The boat idea was a powerful experience in virtual reality, the motion of the boat as it rode up and down the waves was very physical. It tricked the brain into thinking that it’s rolling back and forth, side to side. Essentially a kind of nausea inducing motion sickness. It should be noted that this isn’t motion sickness as you would experience in a car, where the body feels its travelling but the brain doesn’t. In this case the brain thinks you are moving when the body doesn’t. The effect is similar but different. 

Either way, the nausea element of the project was not a desirable outcome. It would have been a distraction in being able to compare the different versions, and not a pleasant experience for a general public audience. Thus began the work to change the form of motion, whilst keeping nausea to a minimum. After some experiments with different types of navigating the space, a very slow acceleration and moving in slight arcs was settled upon. The viewer/participant would travel around from one featured painting (by Jeremy Gardiner) to the next. Slowly speeding up and then slowly decelerating. On testing, most people did not find this nauseating. To further update the project we took photographs of new paintings by Jeremy, and positioned them in specific locations around the coast, just west of Swanage, covering an approximately 3 mile stretch of coast.

The team started to experimenting with different settings how the virtual stereo camera behaved (which represents the user/viewer), it was realised that a sense of scale was more noticeable in VR than just looking at the screen. Whilst wearing a VR headset, the user ‘owns’ their scale, and this is dependent on the separation distance of the two virtual cameras that VR uses, which would usually match the separation distance between the left and right eye. However, making the virtual separation distance large (much larger than life), made the user feel like a giant, standing in the sea, up to one’s neck. It was a bit suffocating. It also had the effect of making the landscape feel like a model. It was not a feeling that was possible by just looking at the screen version, where your sense of scale is more imagined. Increasing the camera separation also implied adjusting the angles of the cameras, pointing them in towards each other, which has the effect of increasing the sense of 3D depth. This could also add a sense of nausea, particularly as one passed through the painting. Ultimately, these were interesting experiments but not the desired outcome. The aim was to try and give the person an accurate sense of scale to the coastal scape, and thus making the paintings in the environment appear super-sized.

Getting the virtual reality version correct led automatically to the projection version being correct. The only difference was that the projection version uses one camera (it’s a mono experience), and it didn’t include movement interactivity. This made it feel more like a film, although it wasn’t a recorded experience.

To create the 360 degree version that would be viewable online via YouTube, it was a case of screen grabbing the whole experience in equirectangular format, i.e. screen capturing all 360 degrees, in round, but mapping that to a rectangle image (as per the example above), at 8k resolution.  This involved thousands of screen grabs running overnight on the computer, then joined together to create one 360 movie. When viewed on a 360 platform, the user can look around in the space, but, as a recorded experience. Light Years: Coast 360 8k

Thus 3 versions of the new Light Years: Coast were created:

  1. A mono camera version for projection or computer screen, with real-time graphics.
  2. A mono camera, 360 recorded video version, for YouTube.
  3. A stereo camera, real-time graphics version for virtual reality

The results

The project has been shown at several events. Initially at one of the workshops for the SWCTN to other Immersion Fellows. It was also shown at a public lecture by Anthony at Bath Spa University, as well as being tested by students on the BA Digital Animation course.

Although, not an exhaustive study, there are early results that hint at the subtle differences between these virtual formats. As an experience, watching version 1 is like watching a film; it’s a passive viewing experience that’s comfortable to watch, and engaging as a result. The resolution is clear and expandable (i.e. could be shown on a 4k screen).

The 360 degree version is a passive viewing experience to an extent, but with the ability to move your head around. It gives the viewer the sense of being in the place, but somewhat disconnected. This is probably because its a mono viewing experience. There’s a feeling of being inside a ball, not actually being in the place. 

The virtual reality version, with stereo viewing, and full ability to move your head in 6 degrees of freedom, was the strongest in terms of feeling like you were really there, really flying. But as a result it made users want to have more control, to be able to steer around the space. In a sense, Light Years: Coast, as it was made, was a frustrating experience in VR, because one wanted to be active in the space.

 

Next steps

The aim is to make a brand new virtual reality version of Light Years, that is more interactive, giving the user more control of the experience, not just in movement, but in terms of pictorial composition.