In 2021 Oculus released an update to the Quest 2 Virtual Reality (VR) headsets which gave developers access to an exciting new feature: Passthrough Mode.

Passthrough Mode uses the external cameras on the VR headset to display a live feed of the user’s real world surroundings and make them visible within the virtual environment displayed in the headset. This feature has existed within the Quest headsets since their launch, but until now this functionality was not accessible to developers and was exclusively used when a user steps outside the boundary box of their VR space.

In this blog post I will cover the steps needed to setup Passthrough Mode functionality for developers in a Unity project.

 

This post will cover the following sections:

 

Intro

Passthrough VR, in my opinion, is the future of mixed reality. Capitalizing on the benefits of both virtual reality (VR) and augmented reality (AR), this mixed reality (MR) environment will allow developers to create stable and consistent virtual content and interactions that both enhance and integrate with the real world in ways that neither medium could come close to previously.

Although Passthrough VR is just now starting to take its first steps and still has a long way to go before its as polished or stable as its VR/AR predecessors, these first steps are full of promising potential and are headed toward fantastic possibilities.

 

As exciting as the future of this technology is, the current state of Passthrough VR can be a little tricky to setup and get running correctly. In this blog post I will cover the steps needed to create a Unity Project capable of building applications using Passthrough VR on the Oculus Quest 2. Due to the rapidly changing nature of the XR landscape, these instructions may become outdates faster than I’d like. But even if that becomes the case, these steps will hopefully help others setup the baseline requirements for a Passthrough VR project.

 

 

New Project Setup

For this post, I am going to assume that any developers interested in starting a new Passthrough VR project in Unity are familiar with the baseline knowledge of how to install the Unity Engine and start a new project through the Unity Hub. To any readers who are not familiar with these steps, there are countless tutorial videos and web pages covering that material, so I won’t be going through those steps here.

 

The first step in this process will be to start a new Universal Render Pipeline project through Unity Hub using Unity version 2020.3.25f1.

 

  • Select “New Project”

 

  • Select “3D Sample Scene (URP)”

 

  • Enter the Project Name and select a location to create the project

 

  • Click Create Project

 

  • Once the project opens, create a folder in the Assets folder to store any passthrough related content you create for your project. This folder can be named whatever you’d like. I named mine “Passthrough_VR”. (This step is not required, but I find it helpful for keeping everything organized.)

 

  • Next create a Scenes folder inside the Passthrough_VR folder to hold any scenes your create.

 

  • Create and open a new scene. I named my scene TempScene01.

 

  • Now you can delete some of the unneeded bloat content that came included when the project started. In the Assets folder, delete the “ExampleAssets”, “Scenes”, “Scripts” and “TutorialInfo” folders.

 

 

Platform Setup

 

Now that the initial project is setup, its time to switch over to the correct platform to be able to build to the Oculus Quest.

First you will need to make sure that the Android Modules have been installed for Unity version 2020.3.25f1. This can be done through the Unity Hub.

  • To do this, open the Unity Hub program

 

  • On the left side of the Unity Hub window, click on the “Installs” tab

 

  • Find the 2020.3.25f1 section and click on the gear shaped icon on the right side of this section to open a small dropdown menu. Click the “Add Modules” option from this dropdown menu.

 

  • Select the Android Build Support section and each of its dropdown options. Make sure you have the Android Build Support, Android SDK & NDK Tools, and OpenJDK selected. Then click the install button on the bottom right corner of the window.

 

  • Once those Android modules have been installed, return to the Unity project and open the “Build Settings” Window. This can be accessed by clicking the File > Build Settings from the top menu in the Unity Editor.

 

  • In the Build Settings window, select Android in the Platform section of the window (lower left), then click the “Switch Platforms” button on the bottom right corner of the window.

 

 

Settings Setup

 

Now we will be changing some of the project setting to be able to build to the Quest headset.

  • Open the Project Settings window (Edit > Project Settings)

 

  • Open the Player Settings (left side of the Settings Window), then open the “Other Settings” section

 

  • Change the “Color Space” setting is set to “Linear” instead of “Gamma”

 

  • Change the “Scripting Backend” setting to “IL2CPP”

 

  • Change the “Target Achitecture” to exclusively “ARM64” (uncheck any other architecture boxes)

 

  • Change the “Graphic APIs” to only contain “OpenGLES3”

 

  • Change the “Minimum API Level” to “Android 6.0 ‘Marshmallow’ (API level 23)”

 

 

 

Plugin Setup

Next up, we will be setting up the plugins.

  • Open the Project Settings window (Edit > Project Settings)

 

  • Open the XR Plugin Management Settings (left side of the Settings Window)

 

  • Click the “Install XR Plugin Manager” button

 

  • Once installation is complete, click the “Oculus” checkbox in the Plug-in Providers section

 

 

Package Setup

 

Now we need to install the Oculus Integration package.

 

 

  • Import the package into Unity by clicking the “Assets > Import Package > Custom Package” before selecting the Oculus Integration Package.

 

  • Once the package imports, a few dialog windows will open, click “Yes”, “Restart”, and “Upgrade” respectively when these boxes appear.

 

 

Scene Setup

Now on to setting up the scene.

  • Open the “Augmented Objects” scene in the Assets/Oculus/SampleFramework/Usage/Passthrough/Scenes folder

  • In the Hierarchy, find and select the “OVRCameraRig” GameObject

 

 

  • Find the “OVRManager” component/script on the OVRCameraRig object, and in the “Experimental” section check the “Experimental Features Enabled” and “Passthrough Capability Enabled” checkboxes

 

 

  • Check that the OVRCameraRig has an “OVRPassthroughLayer” component attached to it

 

  • Open the Build Settings window (File > Build Settings)

 

  • Click the “Add Open Scene” button to add the current scene to the build.

 

 

Experimental Mode Setup

The final step in this setup process is to activate “Experimental Mode” on your Quest 2 headset. To do this you will need to install and setup the SideQuest desktop app.

SideQuest can be located here: https://sidequestvr.com/

 

Once the SideQuest app is setup, activating experimental more on your headset is quite easy.

  • In the SideQuest app, open the “Device settings & tools” tab by clicking the wrench icon in the top right corner of the window.

 

  • Scroll down to the section titled “Experimental Mode” and click the “On” button.

 

 

After completing these steps, you should be able to build a functioning PassthroughVR app to your Quest 2 headset.

 

The demo video below displays a few short demos of PassthroughVR in action.