Demonstration

While not universal, there are demonstrable benefits from the use of immersive technologies in many geo-related areas, including agent-based modeling, geographic training, climate change, hazard control and management, human-environmental interaction, and geospatial simulations involving both the physical and human dimensions. As a spatial cognition researcher with GIScience background, I am broadly interested in immersive virtual reality (iVR) and its application in spatial cognition, wayfinding, earth science data visualization, and place-based education.

iVR Workbench for Visualization and Quantitative Exploration of 3D Earth Science Data Sets

I developed an immersive workbench as an iVR research platform delivering virtual fieldwork experiences of Iceland’s Thrihnukar volcano. I imported and visualized real-world earth science data in the virtual environment. The iVR workbench enables interactive visualization and quantitative observation of earth science data through immersive interfaces.

Video

Another Example – Immersive Visualization for Liwu River in Taroko National Park (Tianxiang)

I used Structure from Motion (SfM) mapping to construct a photorealistic point cloud of the sites (over 80 million points!). The prototype offers analytic functions currently not available in the field and is expected to help to make fieldwork experiences accessible, shareable, and available any time.

video

References 

Zhao, J., Wallgrün, J. O., LaFemina, P. C., Normandeau, J., & Klippel, A. (2019): Harnessing the power of immersive virtual reality – visualization and analysis of 3D earth science data sets. In Geo-spatial Information Science, pp. 1–14. DOI: 10.1080/10095020.2019.1621544    Full paper

Zhao, J., Lafemina, P., Wallgrun, J. O., Oprean, D., & Klippel, A. (2017). IVR for the geosciences. 2017 IEEE Virtual Reality Workshop on K-12 Embodied Learning through Virtual & Augmented Reality (KELVAR). doi:10.1109/kelvar.2017.7961557    Full paper

Immersive Virtual Field Trips for Geoscience Education

I integrated 360-degree images, audio scripts, SfM models, videos, and 2D texts and images into a consistent work frame as the first step to simulate field trip experiences for Penn State undergraduate students and beyond. A series of systematic evaluation has been conducted as part of an introductory geoscience laboratory course via comparisons between virtual and actual field trips and between immersive and desktop virtual field trips about the learning effectiveness and outcomes of virtual field trip experiences in place-based STEM education.

Video

Resources

The virtual field trip experience showed in this video is outdated, incomplete, and only for demo purposes.  I have developed virtual field trips for several field sites across different platforms, including HTC Vive, Oculus Quest, Oculus Go, standard desktop computer, and website. If you are interested in experiencing the complete version, here below are some resources for you to try them out!

References

Zhao, J., LaFemina, P., Carr, J., Sajjadi, P., Wallgrün, J. O., & Klippel, A. (2020, in print), Learning in the field: Comparison of desktop, immersive virtual reality, and actual field trips for place-based STEM education. 2020 IEEE Virtual Reality Conference. Full paper

Klippel, A., Zhao, J., Oprean, D. et al. The value of being there: toward a science of immersive virtual field trips. Virtual Reality (2019). https://doi.org/10.1007/s10055-019-00418-5

Zhao, J. & Klippel, A. (2019). Scale – Unexplored Opportunities for Immersive Technologies in Place-based Learning. 2019 IEEE Virtual Reality Conference. Full paper

Klippel, A., Zhao, J., Jackson, K. L., LaFemina, P., Stubbs, C., Oprean, D., Wetzel, R., Wallgrün, J. O., & Blair, J. (2019): Transforming Earth Science Education Through Immersive Experiences. Delivering on a Long Held Promise. In Journal of Educational Computing Research 10 (2). DOI: 10.1177/0735633119854025    Full paper

Klippel, A., Oprean, D., Zhao, J., Wallgrün, J. O., LaFemina, P., Jackson, K., Gowen, E. (2019) Immersive Learning in the Wild: A Progress Report. In: Beck D. et al. (eds) Immersive Learning Research Network. iLRN 2019. Communications in Computer and Information Science, vol 1044. Springer, Cham. https://doi.org/10.1007/978-3-030-23089-0_1    Full paper

Klippel, A., Zhao, J., Oprean, D., Wallgrün, J. O., & Chang, J. S. K (2019). Research framework for immersive virtual field trips. In KELVAR: The Fourth IEEE VR Workshop on K-12+ Embodied Learning through Virtual and Augmented Reality.    Full paper

iVR as an Effective Tool for Second Language Learning

In collaboration with the Brain, Language, and Computation Lab.

I developed virtual zoo and kitchen environments to position second language learning in an authentic context in order to decrease learning costs and foster the formation of long‐term memory. Learners can walk in the kitchen or virtually teleport along a predefined path in the zoo, interact with kitchen items or observe animated animals, and point at objects/animals to hear their names in Mandarin Chinese.

Videos

Reference

Legault, J., Zhao, J., Chi, Y.-A., Chen, W., Klippel, A., & Li, P. (2019). Immersive Virtual Reality as an Effective Tool for Second Language Vocabulary Learning. Languages, 4(1), 13. https://doi.org/10.3390/languages4010013    Full paper

iVR for Studying the Brain

In collaboration with the Brain, Language, and Computation Lab. 

I created an interactive learning environment where students learned the human brain’s anatomy and functions using immersive technologies. More information about this project can be found on Brain3M.

Video

Prototypes

Using hand gestures to manipulate a wood piece from Ishi Wildness, CA.

Embodied learning of Moon phases using a virtual Earth-Moon System in virtual reality.