Project

Let's Explore


About

Released: Sept 2023
Platforms: iOS and Android

Let's Explore was a suite of three themed educational smartphone apps, each one including 360 video streaming, AR experiences and a quiz section.
The three apps were:

  • Let's Explore: Oceans
  • Let's Explore: Dinosaurs (ft. Wildlife)
  • Let's Explore: Space

AR Experiences
As well as VR, Let's Explore utilised AR adding an extra layer of interaction and immersion. Using the physical cube that came with the boxed project, users could scan this to hold various interesting items in the palm of their hands. 'Let's Explore: Oceans' and 'Let's Explore: Space' both had 16 AR models, however, because of the duel theme, 'Let's Explore: Dinosaurs' had 30 AR models (15 dinosaurs and 15 wildlife).

Video Streaming
Each app included 16 streamable 360 videos. Using the Google Cardboard Unity plugin, users could wacth rhe included videos on their phone in their hands, or place it into a VR headset and step into the action to dive with sharks, go on safari, or watch a rocket launch at NASA. Let's Explore allowed people to really experience and learn about, a host of activities that they wouldn't be able to otherwise whilst learning about the theme of the app.

Quizes
Utilising the knowledge gained from the videos and AR experiences, users could then challenge themselves in the quiz section. Choosing from 16 quizes, they would answer 10 multiple choices questions, as fast as they could, to earn the highest score they could.



Development

For Let's Explore I worked as the solo front end developer, using the app design from a contractor and working with the back-end web developer for any back-end API communication between app and server. I was solely responsible for building and testing on both platforms (iOS and Android), as well as store mananagement of the apps (app descriptions, screenshots, etc)

All three Let's Explore apps were developed over 8 months and released on the same day. The majority of those 8 months were spent developing a template project, exported as a package. Using the package, 'Let's Explore: Oceans' was develoved along side the template project, which allowed the template to be feature complete with a working project as a guide. Once the template had reached a state considered "finished" two new projects were created for Dinosaurs and Space, these were then up and running, ready for release, within 3 weeks.



Content Curation and Creation

Let's Explore had two types of content that required sourcing - 360 videos that fit the themes of each app, and 3d art to be used in the AR experiences.

AR Content
From the start it was decided to use models purchased from the Unity Asset Store to build interesting scenes. These would then be coupled with a voice over talking a little bit about the creature or object in the scene. It was my responsibility to decide what AR content each app was going to have, and to source their models. For 'Let's Explore: Oceans' and 'Let's Explore: Dinosaurs' We then used freelance artists to put together the environments for each creature. I chose the freelancers, and managed their work. Once they delivered the finished products I then imported them into the correct project, and made any necessary changes - post processing/lighting tweaks, positions, fog settings and (most importantly) optimisation. All of the 'Let's Explore: Space' AR content was dealt with by myself.

Video Content
From the start of development on Let's Explore we had a contractor tasked with sourcing 360 videos that would fit each theme, which I would give input on suitability and quality. Unfortunately, with a month left until launch I needed to step in and take over this task. Although many videos had been chosen, none had been processed, and there was still a lot to be found. Approximately a week after I took over, we had all the videos we needed, processed and handed over to the web developer for processing with ffmpeg to create various versions of each video in different bit rates, so that a user would be delivered a lower quality video rather than encounter buffering.

The apps launched on time, with all content ready and working.

After the licensing expire on some videos, they were replaced by content I created - using Unity to animate and render 360 videos. More information here.


Augmented Reality


Vuforia

The Vuforia SDK for Unity was used to add augmented reality to the projects. This plugin provides cross-platform smartphone AR, with functionality to easily start and stop the AR and most importantly, track multiple image targets easily, making the tracking of a physical cube easier to implement than other solutions.



Content Delivery - Addressables

We wanted to keep the size of the initial app download to a minimum, as well as be able to make updates the AR content without a new app build. The Unity Addressables met these needs. The app builds were approximately 90MB, with the additional 300MB to 500MB (dinosaurs was larger due to double the content) download required for the AR content.



Virtual Reality


Google Cardboard

The Google Cardboard XR Plugin for Unity was used to add virtual reality to the project. This plugin provides cross-platform smartphone VR, with functionality to easily start and stop the VR - allowing for the Let's Explore apps to have a non-VR menu and non-VR option for viewing the 360 videos.



VR UX

The implementation of the VR menu from Vodiac was reused in Let's Explore. By the time development on Let's Explore began, Vodiac had been released for a couple of years and had been updated and improved upon from user feedback. The only changes made where cosmetic, making the design match the rest of the app. This includes the VR on boarding, VR menu and media controls.



Quizes


Development

Each app required 16 quizes, with 20 multiple choice questions in each quiz. When playing a quiz 10 of the available 20 questions would be randomly selected. I wrote all the questions. There are two types of questions. A simple true/ false and a multiple choice with 4 possible answers.
Scoring was done based on time taken to answer, the faster an answer was given, the higher the score rewarded.

The quiz functionality was developed seperately from the template, and imported into each project as its own package.



Media