This post will cover the setup steps taken to create a development demo for the Butterfly Experience Project. The goal of this was to create a basic playable demo which would demonstrate the core functionality requested for this application by our project collaborators.

 

 

 

This post will cover the following sections:

 

Intro

When starting this project, the development team here at the Center for Immersive Experiences (CIE) decided to pick the core functionality required for creating a basic working demo of this project. The goal for this initial greybox development demo was to display the basic functionality of the experience as requested by out project collaborators. Below I will cover the steps I took while developing the functionality for these core interactions.

 

Core Functions

The first step in creating this demo was to identify what core functionality the butterflies in the experience needed to exhibit. The primary functions and behaviors we decided on were related to movement and navigation. The butterflies needed to be able to fly around the environment in dynamically randomized paths, as opposed to having pre-determined paths set for each butterfly to follow. To do this, the butterflies needed to do be able to do the following:

  • Fly to specified targets
  • Update target locations dynamically within preset boundaries
  • Avoid collision with objects
  • Flap their wings as they move

I will cover the development for each of these functions in greater detail in the following sections.

 

Target Placement / Tracking

Moving a butterfly to a designated target and randomizing the new placement of that target once the target has been reached are both functions that I have previously worked on developing. While familiarizing myself with our team’s in-house VR framework (PuppetJump) and its physics driven interaction system (Forcibles), I had already worked with and/or developed some basics for these tasks. 

 

 

Each of the butterflies is equipped with a set of motors, one that moves the butterfly forward and one that controls which direction it’s facing by adding rotational force. By giving the butterfly a target to always attempt to look at and throttling the power of its forward motor based on distance from that target, we could get the butterflies to automatically move to a target object.

Once the butterfly reaches its target, the target gets randomly placed at a new location within pre-defined boundary parameters. This allows the butterfly to constantly create dynamically randomized flight patterns within a specified space.

 

 

 

Collision Avoidance

The next big obstacle to creating realistic looking butterfly flight was Collision Avoidance. When implemented properly, this would allow each butterfly to intelligently identify approaching obstacles in their path and dynamically alter their course to avoid colliding with the obstacles. This task took quite a bit of trial and error to create. Luckily, this functionality was also something I had previously developed while familiarizing myself with the Forcibles package. So, implementing it in this project only took some minor adjustments.

The video below displays clips of the reason collision avoidance is so necessary in an application like this. Without some form of collision avoidance, the flying objects can easily run into objects and get stuck.

 

 

When building the collision avoidance functionality, I started with the most basic situation of checking for objects directly in front of the butterfly within a given distance. As the butterfly moves, it casts a line (ray) directly forward and if this ray hits any objects that aren’t the butterflies target, that means an obstacle has been found. To avoid running into this obstacle, the butterfly initiates a dodge. During a dodge, the butterfly is assigned a temporary target to move toward which is placed in a direction which has been determined as being clear of obstacles. Verifying which directions are viable directions to dodge, and more so making those dodge options dynamically randomized is where things got tricky.

I knew I could give the code a set of predefined directions to check in order, but I wanted to avoid obvious repetitions in dodge functionality. I knew that if the butterflies always tried to dodge to the left when encountering an obstacle, they would appear too mechanical and less organic in their behavior.

So, I created a set of directions whose order gets randomized each time a dodge direction needs to be found. Once a viable dodge direction is found, the butterfly alters its course to move to the temporary dodge target’s location. When the butterfly has reached the dodge target or has cleared the obstacle, it’s original target is restored and it continues on its way.

The video below exhibits this dodge functionality in action. The second portion of the video below displays colored debugging lines along the rays used to check for obstacles. The green line displays the direct path to the target. The red line is drawn between the butterfly and the obstacle it is attempting to dodge. The yellow lines display failed checks for viable dodge directions, or directions which have been checked and found to not be viable dodge directions.

 

 

 

Wing Flapping

Now that I had butterflies that could organically move around the space without running into walls, the next behavior to be tackled was the flapping of their wings. This functionality was fairly easy to implement by writing some code that would rotate both wing objects in opposite directions simultaneously. I also added some code to use the butterfly’s velocity as a scalar which gets applied to the speed of each wing flap. This means that when the butterfly is moving quickly, the wings flap very fast. As the butterfly slows down to land, the wing beats slow down.

 

 

 

Freeze Functionality

Once those core functions were setup, it was now time to develop the functionality uniquely requested for this project specifically. The first of these features was a function that would freeze all butterflies in place during specific interactions throughout the experience. Implementing this would give the user a sense of time stopping while they were asked questions or given specific objects to inspect and examine.

When the user concludes these specific interactions and continues the experience, the butterflies would unfreeze and continue moving as normal.

It was important that this freeze function only effect butterflies in the scene. Otherwise if we froze everything in the scene, the user would not be able to continue interacting with the scene during the freeze. The second portion of the video below demonstrates this aspect of the freeze mechanic. In the second portion, when the butterflies get frozen, a flying cube continues it’s normal functionality without being effected by the freeze functionality.

 

 

 

Environment

At this point in development, I received the first model of the environment from our teams modeler Alex Fatemi. With a prototype of the environment in the program, I could now start working on building the experience itself.

 

 

 

Color Change

From the start of this project, I knew that we would be using two or three different types of butterflies. At this greybox stage of development, I decided to use solid colors and apply them to the wings of each butterfly to differentiate them by type. To accomplish this, I created a dynamic color changing system which allowed each butterfly to change color on the fly. This meant I didn’t have to create multiple versions of each butterfly in the scene and could streamline the development and testing of the experience.

 

 

 

Path Selection

With the color change functionality in place, it was time to build a path selection development tool. During the experience, various elements of the program’s progression will be randomized for each user. In order to test out each of the potential path variations individually, the path selection menu allowed me to select a specific path to take each time I tested the project. The path selection, in this case, determines the order in which I encounter each type of butterfly used.

 

 

Focused Interaction Stops

As the experience progresses, the user is meant to encounter Focused Interaction Stops where they will be asked to examine specific butterflies and answer questions about their observations. To build these stops, I needed to create functionality which would activate various elements and game objects at the correct times. As the user enters the designated area for a stop, small barriers are activated to prevent them from traveling past the interactions for the stop. A UI menu, which is just a placeholder at this stage, is activated which will eventually display the prompts and questions for the user. The butterflies around the scene will also all freeze at this moment, so the user does not have any distractions and can focus on the content for the stop. There will also be a single example butterfly at each stop which the user will be asked to examine and observe during the focused interaction.

 

 

 

Landing / Hold Position

When the user enters a focused interaction stop, the example butterfly designated for that stop needed to be able to move to a specific location and alter its movements while the user examines it. To create this behavior, I developed the landing and hold position functionality. The landing functionality, when activated, pauses the butterfly’s normal target tracking movements and instead directs it to move toward a pre-set stationary location. The hold position functionality alters the butterfly’s normal wing flap pattern to instead make the butterfly pause or slow its wing flaps to position them in a stance that makes the wings easy to observe by the user.

 

 

 

Swarm Color Change

While moving through the environment of the experience, a large swarm of butterflies can be seen flying around the ceiling and central tree. These swarm butterflies needed to match the color of the butterfly to be examines at the next focused interaction point. This means that when the user finishes one focused interaction and is heading toward the next, the swarm needs to transition from one color to another.

In order to avoid a visually jarring snap that instantly changed the color of all the butterflies in the scene, which would break the immersion experienced by any users, I decided to create the swarm color change mechanic.

This functionality works by changing the target placement boundaries for each butterfly in the swarm, updating these boundaries to a small hidden area above the central tree. As each individual swarm butterfly reaches its current target, the target gets reset to these new boundaries, and the butterfly makes its way to the hidden area. Once a swarm butterfly enters the hidden area, it’s color is changes to the correct new color and its target boundaries are reset to the original room sized boundary box. When implemented with the entire swarm, this functionality makes a gradual color transition happen among the swarm butterflies as each one navigates to and from this hidden color change area.

 

 

 

Wing Textures

The final touch added to this iteration of the Butterfly Experience project was the implementation of detailed butterfly wing textures. These textures, provided by our center’s designer Erica Krieger, are realistic renditions of butterfly wings. Importing and implementing these new wing textures added a new level of realism and immersion to the project.