Going to make a post about this to better explain the situation.
Volume Viewer Pro is a plugin for Unity to view volumetric data such as MRI scans. I’ve been exploring it as part of a potential project. Initially things were rather promising. Volume Viewer essentially does the reverse of a scan. It takes the volumetric data and creates cameras around a box. These cameras then do raycasts based on the data and seems to calculate the intersections of the rays with the box in order to shave away at the shape’s normal map and create a complex volume rendered by the camera. Due to this setup, the ability to view these volumes in VR is actually possible, at least with the Rift S and our higher end PCs. We have already tested this prior using some data we were given that was a scan of a mouse brain.
To start going forward, we asked for more complex data to try and accomplish the same result. We were given a scan of an aorta. Trying to simply plug it right into the plugin, however, resulted in a problem; Unity ran out of memory. Confused about this, I went ahead and examined the file and realized it was 25 times larger than the rat brain data and consisted of about 7 times more images in it’s stack. Considering Unity has to create multiple arrays of rays based on the number and size of the images, it’s no wonder this was causing the system to run out of memory.
So I went to meet with our partners and learn how to edit the data in the program Avizo, hoping to pare it down enough to fit in Unity’s limitations. Avizo is a rather interesting program, using various functions that are applied to the data and spit out modified instances of it. For example, if I apply the invert function, I get a new data set with the blacks and whites inverted (this was something I had to do with the given data for it to show up properly). I cropped the data and resampled it to the point where it finally showed up in Unity. Unfortunately, it showed up like this.
The shape of the aorta is clearly there, but there’s a TON of blobs distorting it. This was due to the noise within the scan. Any larger black spot would be included in the Volume Viewer calculations. There was also a circular pattern around the scan in question that resulted in the large chunks seen in the corners. Here’s a slice of how the data really looks:
Now, you may notice some other images in the shot above. Much smaller images than the aorta scan. These are slices of the example data the plugin came with (a human head scan) and the mouse brain that was above.
Both of these data sets are FAR smaller than the aorta in their resolution, and are composed of way fewer images. They’re also much less noisy, giving me the thought that maybe there was some cleaning process they were put through. The lack of noise, clear black-white seperation, and lower resolution are all factors that let these data sets come into the plugin cleanly while the aorta becomes cluttered. The flatter scan besides the human head scan is also from the plugin. This was a segmentation scan used as an overlay of data for the plugin, with the different gray values allowing the plugin to isolate various pieces and color them differently (i.e. removing half the outer head skin and highlighting a region of the brain). The aorta also came with a similar labels set, but it was in color so I’m not sure how well it will work with the plugin.
Overall, this plugin does seem to function well, but there’s clearly some processing that has to be done to the data prior to bringing it in to reduce the noise and increase the contrast. I’m hoping that Avizo has some form of masking feature, as that would at least make the process easier since we already were provided a nice label set that could potentially mask out the remainder of the other set and make such post-processing easier. I don’t know about this for sure though as this program and data structure is all very new to me. That said, if our partners were to figure out the proper processing methods to achieve scan similar to the mouse brain and example set, we could certainly be able to render them within Unity VR using the plugin and allow for exploration of said data.
Update 12/9: A peer looked into things some more. He found that it’s possible to use Avizo to convert the drawn Surface from the data into it’s own volume with a “ScanSurfaceToVolume” function. This gives us a cleaner looking volume to use in Unity, however the quality of this new volume was too low so we’d need to explore how high we can keep that quality before it becomes too large for Unity. The surface is formed from the labels set rather than the raw data, so it is at least clear of any large protrusions like I had mentioned above. It’s essentially the visualization of the data in Avizo being turned into it’s own volume set, which may be an option assuming this doesn’t result in a major loss of quality.
Leave a Reply