The Virtual Frontier

Laying the Foundation for Prototype Park

by Leave a Comment

Earlier this year I began a focused exploration into creating an experience in VR for creating and manipulating objects with the purpose of developing virtual (paper) prototypes. Originally, the plan was to create an application heavily inspired by currently existing VR prototyping apps like ShapesXR. The logic being that using a market tested version of what we hope to create for Penn State would be a good springboard into tailoring our application to better fit our needs.

A look at ShapesXR
ShapesXR provides a wide array of options for creating testable prototypes.

And we weren’t wrong. I learned a lot from using ShapesXR and recreating some of the core functionality. Zac and I talked a lot about what I had been working on and what we liked and didn’t like about the functionality present in ShapesXR, and came to a pretty succinct conclusion.

We Want the Features, Not the Foundation

ShapesXR has a lot of good features we would be happy to adapt into our application, but its core systems aren’t precise enough for what we want people to be able to create. And this is something core to any VR experience. Humans are naturally fidgety creatures, we are not precise on our own or without experience. Add to that a headset and controllers whose weights we constantly have to counterbalance, and you’ve got an imprecise environment for creating what are supposed to be precise prototypes.

So for our prototyping application, how do we introduce this precision while maintaining the sense of scale and presence in VR? Well, after talking about it for a while Zac has two different systems in mind. And while he’ll tackle one way, he’s left the other way for me to think outside the box, or should I say, outside the block…

Minecraft Header
First released in 2012, Minecraft is one of the most popular games ever made, with an average of 3 million people playing it daily 10 years later.

It’s Time to Craft Some Mines!

Well, not exactly. See, Minecraft uses a 3D graphical concept known as Voxels to store data specific to its world. Voxels have a lot of interesting properties, but the one we’re most concerned with for our exploration is their precision. Similar to pixels, a system knows where a voxel is based on its neighboring voxels, so their position in space is always relative. So when you place a block in Minecraft, its always perfectly align with its neighboring voxels. This is a really helpful work around to the imprecise nature of VR. By sacrificing a bit of player autonomy in object placement, we’re able to give them much more control and precision during the prototyping process.

There was just one problem. I didn’t know how to make Minecraft in VR! Luckily, Zac had done a bit of research and found b3agz’s Code a Game Like Minecraft In Unity Series which I’ve been watching and learning a ton of techniques and concepts related to voxel generation and management. There’s going to be a lot of editing done on my part to make these concepts work for our team, but so far the results have been promising. And when I’ve got a more concrete demonstration I’ll post an update, but for now I’ll leave you with a video showing the first rough draft of this version of the prototyping application:

Tearables for PuppetJump

by Leave a Comment

Intro

So, at a recent meeting we discussed concepts for a new project. Everyone brought cool ideas and while we’re still working out exactly what idea we’re doing, in passing a comment was made about how tearing apart objects in VR is a cool idea with generic applications. And y’know what? That’s right! And so I spent the last week working on this “Tearing” mechanic and now I’ve got the first iteration of the mechanic to show off.

Abstract

Simply put, the system currently works by tracking the offset between a user’s hand in physical space vs the hand in virtual space. When that offset reaches a threshold set by Tearable.cs, the object “tears”.

Assets

Tearable.cs – Inherits from Grabbable.cs, since this is just a specialized Grabbable, it made sense to do this so we decrease the amount of components needed on an object and decrease bloat. And all the additional logic can be turned off with a boolean, so if you want to use just the base Grabbable system, you can just set isTearable to false and boom! You’re back to it just being a normal Grabbable.

DestroySelf.cs – An object destroys itself after a period of time that can be set in Editor. Used for Tear System.

Tear System – A particle system that activates on tear. Currently set to a cube mesh for the particle, however the color and scale of the particles scales with the object.

 

Breakdown

This system is based entirely around measuring the distance between the physical and virtual hand. This is great for a multitude of reasons:

  • Works with stationary objects, which all objects will be as you pull them in opposite directions with equal force
  • Because it doesn’t measure something tied to the physics system like object speed, there are no weird situations where holding an object with both hands and moving too fast causes the object to break unexpectedly
  • Because it’s reliant on the hands and not the object, it can be applied to any object and the base functionality works immediately

Although, the fact that it doesn’t rely on the physics interactions does limit the scope of this iteration, and introduce design complications like the tearing not being dynamically responsive, you’ve either torn the object fully or you haven’t, there’s no mesh tearing tricks to go half way.

The divisibility is user friendly, and only a click away. You can set an object to be your output, but if you don’t and still want divisibility, the object sets itself as the object, then divides in half indefinitely.

Future Considerations and Learning Applications

 

So where do we go from here? My next tasks will be incorporating the Rigidbody component into this project. Right now the object will visually split, but physical components like the mass stay the same, so next up would be that.

For learning applications I could see this being used in really any application. Viscerally the action of pulling an object apart fits a ton of educational genres. In biology, you could tear open objects to look inside them, like coconuts or eggs. In chemistry, you could tear apart molecules or atoms (and watch your virtual world explode from the fission).

 

Conclusion

So that’s my update! I’m super excited to further iterate on this and see where it goes. Stay tuned for the next update to this project when I attach the Rigidbody and Forcible component.

Matter Changer for PuppetJump

by Leave a Comment

Introduction

A couple weeks ago my coworker Bart posted about some experimentation he had done with Zac’s new VR development library: PuppetJump. It’s a fun experiment about magnets (how do they work?), and a real good read about his process (which you can check out here), but more importantly it’s given me some inspiration to get in to PuppetJump and do some fun experimenting with the system Zac has written specifically to create a demo called “The Matter Changer”!

Abstract

Alright so I’ve actually created 3 demos from this project: The Preset Matter Changer, “The Transfer” Changer, and the In Scene Editing Changer. Each one has their own purpose, which I’ll discuss more thoroughly when we get to them, but they all share a couple things in common. They all rely on the Forcibles and Rigidbody classes, and they all provided a chance for me to try out the PJ3DUI library. Here’s a list of the variables that can currently be changed using this system:

  • The object’s Mass
  • The object’s Physics Material
  • The object’s brake and angular brake (controls the drag and angular drag respectively)
  • The object’s max speed and max angular speed (how fast it can move and turn)
  • Whether or not the object is using Gravity

Assets

I’ve developed two classes for this project: MatterChanger.cs and MatterButton.cs. MatterChanger holds a reference to all of the information about the object placed on it, and MatterButton changes those values.

Version 1: Preset Matter Changer

The simplest form of the Matter Changer from a design perspective. The basic concept is that you assign each button a matter type using MatterButton.cs, and then they tell the Matter Changer what to change about the object you’ve placed on top of the Matter Changer. In its current iteration, this Matter Changer is great if you’ve got specific static material types and want a simpler method of interaction for the user to engage with. Since all you need to do is drop the object on the Changer, then press the button, it’s good for exploring the concept of changing materials, without bogging the user down with more complicated tasks.

Version 2: “The Transfer” Changer

This form of the Matter Changer takes advantage of the fact that the button is in of itself a type of object with all of these variables that can be changed, so if it simply reads in the data from one object on a special type of Matter Changer base, it can send that information to an object on a different Matter Changer base. This iteration could see use in simple puzzle exercises or explorations that wanted to focus on what happens when you give one object some of another object’s physical properties, but it still retains its innate shape and dimensional properties.

Version 3: In Scene Editing Changer

This iteration of the Matter Changer is the most malleable form of the Matter Changer. Theoretically you could create an enormous amount of materials to give objects with what is given to you in this final demo. Not all of them will be viable, not even close to all, but experimenting with what works and what you can create will be the main focus of this type of Changer. It also functions as a dynamic Matter Changer, and is adaptable in scene to whatever needs you have at that moment, so if you know what you’re doing, you can leverage some really interesting results with any objects you have in scene! The biggest issue with this is in order to understand what you’re doing, the user has to have some knowledge of how objects work physically and maybe a little bit of Unity knowledge to understand how things differ virtually. Or you could run in with no experience and just see what works, y’know: Science!

Future Considerations

So where can this project go from here? Well I see two lenses to view this question from, the first being educationally, the other being development. In terms of development my first thought is to make the Matter Changer mobile. For example, having a little instrument you could carry that allows you to change an object’s properties just by pointing at it, like a remote control, except instead of changing a channel, it changes the mass of an object. Maybe there’s a use case for a 2 way transfer, so both objects switch properties instead of the receiver just getting the sender’s properties. Heck, even something like a scanner that tells you the innate properties that an object comes into scene with would be cool. For example, Zac’s already added an abstraction for weight where an object moves slower when held because it’s offering more resistance because it has more mass, but if the user new what the mass of that object was going in, and a had a reference for how heavy that is, then it would help sell the illusion further. From an educational standpoint I could see this application being used in something as straightforward as a Physics 101 class as an aid for students to better understand the relationship of matter to its properties, to a more abstract philosophical discussion of the metaphysics. Y’know, like how much of a thing is a thing if you keep changing the parts that make it that thing. Could do a fun discussion referencing the ship of Theseus, or some aspects of metaphysics. I really believe there’s a lot you could do both inside of and around this tool.

Conclusion

And that’s what I’ve got to show! I think this application could have a lot of educational potential for the university. I mean, even the concept, like turning a barrel physically into a ball, could be taken in a million different directions. I hope with further iterations, or even collaborative work within my organization, this idea could be a good way to teach a concept that’s hard to visualize without this direct, virtual approach.

My Experiment with Machine Learning

by Leave a Comment

Over the last week I’ve taken Unity Learn’s ML-Agents learning course for a couple of reasons. The first reason was to explore the application and viability of Machine Learning in our VR projects, and the second was to familiarize myself with a new way of creating/developing AI. While I’m glad I saw this journey through ’til the end, the result of the experiment was marred by Unity’s disjointed and non-unified approach to Machine Learning. Lemme explain:

The Good!

HB
Giving this little guy a brain was more fun than I thought it’d be.

The course itself is well taught and well explained by someone who knows what they’re doing. Adam Kelly was a good teacher and I’m interested in learning more from him.

Watching the AI grow in its intelligence over millions (and I do mean millions) of runs was honestly breathtaking. Watching the little guy go from banging his head on the floor and ceiling to consistently executing the task he was given was a joy to watch, even if it took multiple hours to get there.

Inside of Unity, everything is fairly easy to understand. From writing the agent’s logic to attaching the Neural Network to the agent, as long as your inside of Unity, everything goes pretty smooth.

The Bad…

HB Profile
It’s really hard to know what kind of performance impact this has when AI is thrown into the “Other” category with 10 other things

Takes quite a while to train up the brain. Took my system over 2 hours and 3 million runs over 8 agents to get to a point where I felt comfortable using that Neural Network, whereas for AI that simple I could write up a state machine that I know would work as soon as I hit play.

If you want a new Neural Network (brain), you need to train the system all over again, so one little change means your computer is a render farm for AI for the next couple of hours.

You need to do a lot of prep for this project. ML-Agents is the easiest, at first, since all you need to do is access it through the Package Manager. But then, you find out it lives on its own repo, and you learn this because you need to download the corresponding Python package from there, because the actual machine learning is done in Python, so that means you need to download a Python interface (they recommend Anaconda). Then install the right version of pip via Anaconda. Then set up the environment in Anaconda for Machine Learning. Then  make sure your ML-Agents Python package’s version is compatible with your ML-Agents Unity package (this took me easily half an hour to debug and understand what was going on). Then you can start testing after you’ve setup a trainer configuration file so Python knows what parameters it needs to use to train the AI!

Now in all fairness, Adam does a pretty good job at walking you through all these steps. But as he doesn’t explain the Python side of things very well, it helps if you’re familiar with Python as a language and Anaconda as an interface.

The Conclusion!

I’m hard pressed to find a practical use for this in our line of work. On the one hand, machine learning systems and VR thematically fit together as two pieces of futurist technology. On the other hand, any AI system I could want to put in a VR game would be simple enough that writing a simple state machine or yarn spinner equivalent.

I think this is a system to put on the back burner for a while. At the very least I wouldn’t recommend using it until Unity has a more unified and simpler way to integrate the Python elements into the design of this tool.

The Showroom Exploration

by Leave a Comment

I recently made a few environmental concepts for a type of classroom I’ve been calling The ShowroomTM, and I figured it’d be good to take some time to further dive into exactly what’s going on with this conception and the types of interactions that would be possible in these different iterations. (All measurements are in Unity’s default units, which are reported to be meters)

The Small Stage

Current Object Size: 5x5x5

Current Stage Size(radius): 10

Uses for this stage?

  • Smaller classroom experiences
  • Smaller objects ( real size cars, or enlarged objects)
  • Simple goals/objectives

The Medium Stage

Current Object Size: 40x12x15

Current Stage Size(radius): 40

Uses for this stage?

  • A larger class size
  • Bigger objects (Dinosaurs, Ships, etc)

The Large Stage

Current Object Size: 384x192x18

Current Stage Size(radius): 212

Uses for this stage?

  • A huge class size
  • Buildings (like the St. Louis Arch, which the example is based on)

There may even be options for expanding this vertically instead of just horizontally. For example we could have like a whole rainforest and have it broken down by the different layers, with different activities in each layer. So like this:

And each layer is represented in the scene as a different level to be explored. And the professor could control the accessible layers and move the class as a whole or grant movement between layers to students individually so they could be on different layers if they wanted to do some interaction or activity on a specific layer to better reinforce what they’ve learned.

Unity to Unreal Development Part 1: Prep work

by Leave a Comment

If you have any interest in Game Development than there is no doubt you’ve heard of Unity and Unreal as the two titans of independent development. I myself have been using Unity professionally for over a year now, and have been using it for amateur development for nearly 4 years before that, but nothing exists in a vacuum, and with the eventual release of Unreal Engine 5 and Unity’s dichotomy of updates (really, it’s kinda staggering how their most recent stable release is currently two “years” behind the most recent beta release) I’m very interested in exploring new opportunities for development. Before I do any sort of move, or even open Unreal, I need to know what I’m getting myself into first, and that’s where I turn to tutorials.

Luckily, both Unity and Unreal have excellent tutorial sites, and wouldn’t ya know? Unreal has a whole tutorial series on moving over from Unity! They even got Craig Barr to do it, which means you’ll be hearing a familiar voice if you’ve spent any time watching Unity tutorials. That tutorial can be found here

After watching the course I kinda feel better about making the switch. I only say kinda because I figured that structurally Unity and Unreal are similar enough that I could eventually find my way around the editor, and I know that the struggle is gonna come once I start developing and need to start writing C++. At least, I though that would be a problem, until I found a neat little github repo called MonoUE which allows you to write code for Unreal using C#(yay!) or F#(cool? Does anyone know what this is?).  Another small gripe I have is every time you open a new window in Unreal it doesn’t attach itself like a tab in Unity, which is really a small thing, but something I noticed that’s probably going to frustrate me early on. On the other end of the spectrum, I think it’s really cool how UE occasionally releases assets from its games as free assets to be downloaded and used in your projects. That sort of thing can really help you either make your own game look better faster, or allow you to deconstruct what you like about the assets so you can replicate a more customizable version of it for your game!

So my next step is to actually get into a project in Unreal Engine and see what happens. Unfortunately I’ve run into a couple of issues with installing MonoUE (that’s on my end, not MonoUE) and to get it in my version of UE I needed to get UE through Github, which is fine, but it means I need to finish setting up UE through Github, and installing everything UE needs to work when accessed through that route. But once I do all that, I’ll get in and report exactly what it’s like to work in a UE editor as someone with Unity experience!

Basic Experience Comparison Project Post: What Space do I Use?

by Leave a Comment

The Intro

Hey y’all. I’ve been thinking about space again. This time I wanna focus on space as an extension of the play experience, rather than the current mechanical limitations placed on the play experience by our available space. More specifically I’ve been talking with my boss about creating an environment in which we can give people basic mechanical interactions/experiences within a well defined and easily understood space so we can have people test the mechanical viability of an interaction in a VR application vs. a desktop application. Part of this post is going to be a pros and cons discussion of the different environments we’ve been exploring, but another part I want to focus on is the idea of “functional space” and how it relates back to VR mainly.
[Read more…]

Half Life Alyx: Mechanical Analysis

by Leave a Comment

I recently finished Half Life: Alyx. It took me a little over 9 hours to complete the story, and while I’ll save an experiential review of the game for a later post, I did want to take some time to highlight the systems or mechanics that I enjoyed most, and may be the focus of a Mechanic of the Month later on (hell, one of them already is).
[Read more…]

Mechanic of the Month Review #1: The Frying Pan

by Leave a Comment

The Introduction

Welcome one and welcome all to my very first Mechanic of the Month review post! Mechanic of the Month is a somewhat laid back exploratory task where we take a game mechanic or idea that we think could hold some merit in being present in our future VR games and projects and work on a small prototype of that mechanic for a month. Then, at the end of the month we (or at least, I) will write a little report about what was accomplished, how, and how it can be improved on in the future.
[Read more…]

Skip to toolbar