Immersive visualisation with the Oculus Rift and gaming engines ... by Steve Richardson

BGS staff held a hackathon on 19-20 July at its Keyworth headquarters. Nearly 50 developers, scientists and judges worked in teams addressing scientific challenges and developing prototype solutions in a relaxed, informal, collaborative atmosphere. In the first of our reports from Challenge Leaders, Steve Richardson describes his team’s 14 hours tacking ‘Immersive visualisation with the Oculus Rift and gaming engines’...

The title of our hackathon challenge was ‘Immersive visualisation using the Oculus Rift and Gaming engines’. The idea behind this challenge was to give our developers and scientists a chance to experiment with emerging Virtual Reality (VR) technologies and see whether visualising 3D data could help make our data more accessible, visible and exciting. Ideally we wanted to show how such tools could be used and demonstrated at conferences, BGS open days or even downloaded by users around the world.

For the two-day challenge, our specific goal was to:
“Build an INTERACTIVE, IMMERSIVE model (using a gaming engine) of the BGS geological walkway for players to EXPLORE (wearing the Oculus Rift)”.

We chose to use the BGS geological walk because it was an ideal site to demonstrate the concepts of visualising real-world locations in virtual environments. We could have chosen anywhere in the world but being on-site was the most convenient and familiar.

We also had a lot of data already available that would help us to get on with testing the technology in a time-restricted hack environment, such as a LIDAR point cloud dataset of the concourse captured in 2012:

Figure 1 LIDAR model of the BGS concourse

Our team was a mixture of BGS developers (Steve Richardson, John Talbot) and 3D modellers (Bruce Napier, Luz Marina Ramos Cabrera) as well as a work experience student (Pierce).

After some hack-team planning we got stuck into the challenge at full speed, fuelled by bags of sweets and drinks provided by Patrick Bell and co.

The majority of our hack challenge tasks revolved around building a virtual world in the Unreal Engine 4 system.

First we used the concourse LIDAR model to use as a blueprint for the model, to make sure that all the buildings, trees, monoliths and paving were positioned correctly in relation to each other. This template allowed us to then start adding in the basic building blocks (cubes for buildings, spheres for the WSB atrium etc) of the built-up environment on and around the geological concourse.

Figure 2 Basic building-blocks of the concourse being added

With the rough outline of the concourse taking shape we then set about making it more realistic. To do this we took high-resolution photographs of the buildings, paving and monoliths, then using photo manipulation software we carefully stitched them together to form large digital wrapping paper to paint onto the bare shapes (buildings, pavement and monoliths). This is a slow and painstaking process which requires attention to detail and most importantly time – something which a hackathon is notoriously short on!

Figure 3 Creating seamless textures from high-resolution photographs

Once we had some textures the next stage was to paint them onto the models. This required us to have 3D models of each monolith, which requires LIDAR imaging or hand-building from scratch. As we were short on time we decided to use generic rock models which we downloading from the internet as placeholders.

Figure 4 Adding the new building textures onto the building model

At the same time as the modelling and texturing was taking place, other team members were searching for materials, models and text as the geological concourse content. As the walkway is in geological time order we wanted to place 3D models of fossils and (where possible) dinosaurs in their correct time sequence, to show how life evolved on the planet. We used the GB3D Type Fossils website to search for each time period, select an interesting range of fossils and then download their 3D model scan. This model was then imported into our concourse ‘world’ and placed into its correct location along the timeline. Virtual signposts were placed next to fossils and time periods to help players to understand how their geological time-travelling was going.

Figure 5 Showing a fossil and signpost on the Silurian section of the concourse

Once the concourse had started shape we regularly tested it in the Oculus Rift, which is a 3D stereoscopic headset that tracks the user’s head movement, creating an immersive experience for the user. 

Figure 6 Testing the model while wearing the Oculus Rift

Viewing the concourse in the Oculus Rift gives the user a sense of scale which is not possible when viewing a 2D screen – especially when we started adding the big dinosaurs into the model!

Figure 7 Release the dinosaurs!

All of this work was taking place under the loudly ticking hackathon-clock, constantly reminding us all of how little time was remaining!

Inevitably, we reached lunchtime of the second day with just enough time to whip up a short presentation for the judging panel.

Presenting our work to the rest of the hackathon judges gave us the chance to stop and review how far we had managed to come in such a short space of time. We had set out to produce an interactive, immersive and explorable model of the BGS concourse and in the short time our team had produced a substantial amount of work, with only the ‘interactive’ element of the task falling off the TODO list.

Figure 8 End-of-hack progress

 Figure 9 Demonstrating the Oculus Rift to a willing volunteer

Our team learned a great deal from the #BGSHack in terms of what worked well but also what didn’t. We hope that the Hack gave scientists and other developers an opportunity to see what these technologies can achieve and look forward to seeing where we can take these techniques in the future.
 
Thanks to the team for their sterling work and efforts over the Hack, and thanks to Patrick and co for all their hard work setting up and running the inaugural #BGSHack!