Research

My Ph.D. dissertation research aims to verify the learning effectiveness of iVR-based VEs and to establish virtual-navigation interventions that improve spatial learning. Specifically, my research contribution includes two related areas: immersion in the sense of technical features and geographic scale or environment visibility from single locations. I will briefly introduce each of these two areas below. 

Effects of Immersion on Spatial Learning

One area of my work focuses on the effect of immersion on spatial learning. Immersion is a technical term describing the characteristics of VR systems such as the field of regard that measures what can be seen by physically rotating the eyes, head, and body. Although iVR is attractive to users such as inducing a higher sense of presence than low-immersion desktop computers, we know relatively little about whether higher immersion levels increase or decrease spatial learning outcomes. My study takes advantage of the flexible stimuli manipulation in VR systems by integrating VE navigation with well-established metrics measuring spatial knowledge acquisition, such as estimates of distance, direction judgments, and cognitive mapping, into a cross-platform spatial learning paradigm that can be assessed via, for example, HTC Vive, mobile VR, and desktop computers. In this area, I have one abstract (S55) presented in the International Conference on Spatial Cognition (2018) and one published paper comparing Oculus Go to HTC Vive in the International Conference on Immersive Learning (2019). Another paper on desktop computer versus HTC Vive has been submitted to Spatial Cognition & Computation and is currently in revision. In the future, I plan to extend my spatial learning paradigm to probe the role of immersion in many place-based disciplines (e.g., geosciences, geography, and biology). For instance, I recently published a full conference paper that probes the media effect of virtual field trips in learning about geology to the IEEE Virtual Reality 2020 Conference. 

The ground-level perspective of using teleportation on a desktop screen.
Teleportation using the hand controller in an HTC Vive HMD.
Onsite pointing task on the desktop screen.

 

 

Model-building task using the HTC Vive.
References
Zhao, J., Sensibaugh, T., Bodenheimer, B., McNamara, T. P., Nazareth, A., Newcombe, N., Minear, M., and Klippel, A. (2020). Desktop versus immersive virtual environments: Effects on spatial learning. Spatial Cognition & Computation, 3(3), 1–36. https://doi.org/10.1080/13875868.2020.1817925

Zhao, J., LaFemina, P., Carr, J., Sajjadi, P., Wallgrun, J. O., & Klippel, A. (2020). Learning in the Field: Comparison of Desktop, Immersive Virtual Reality, and Actual Field Trips for Place-Based STEM Education. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (pp. 893–902). Atlanta, GA: IEEE. https://doi.org/10.1109/VR46266.2020.1581091793502

Klippel, A., Oprean, D., Zhao, J., Wallgrün, J. O., LaFemina, P., Jackson, K., Gowen, E. (2019) Immersive Learning in the Wild: A Progress Report. In: Beck D. et al. (eds) Immersive Learning Research Network. iLRN 2019. Communications in Computer and Information Science, vol 1044. Springer, Cham. https://doi.org/10.1007/978-3-030-23089-0_1

Zhao, J., Klippel, A., Minear, M., Newcombe, N., Bodenheimer, B., McNamara, Bodenheimer, B., McNamara, T., Nazareth, A., & Sensibaugh, T. (2018). Walking and learning in a large-scale mediated space: Impacts of viewpoint transition and proprioceptive feedback on spatial learning in virtual reality [abstract]. In COGNITIVE PROCESSING (Vol. 19, S55‐S55). Retrieved from https://link.springer.com/content/pdf/10.1007%2Fs10339-018-0884-3.pdf

Effects of Geographic Scale on Spatial Learning

The second area of my research centers on geographic scale, defined as the spatial extent visually accessible from a single viewpoint, and its impact on spatial learning in an environmental space (i.e., which requires movement to apprehend). Investigating the relationship between the human body and its spatial environment is a critical component in understanding the process of acquiring spatial memories. However, few empirical evaluations have looked at how the relative visual accessibility of an environment affects spatial learning. My dissertation research aims to use immersive technologies to establish novel virtual-navigation interventions that can improve spatial learning. The geographic scale was manipulated by changing the perspective of the learner and changing the environment to increase visibility from single locations. My research in this area has produced one conference paper in the IEEE Virtual Reality 2019 Conference, one extended abstract in the IEEE Virtual Reality 2020 Conference, and one journal paper published in Cognitive Research: Principles and Implications. 

Change of geographic scale at a single position within a virtual maze. Left: ground perspective (4.5ft/1.4m above ground); the blue flag was the only visible landmark. Right: elevated perspective (17.5ft/5.3m above ground); both the blue flag and Big Ben could be seen from a single viewpoint. The spatial extent visually accessible from a single viewpoint was controlled by hedges along both sides of the path.
Left: six participants in the same session using Oculus Go headsets to learn the virtual maze. Right: participant’s view in the onsite pointing task.

 

References

Zhao, J., Ma, X., Simpson, M., Wallgrün, J. O., Sajjadi, P., & Klippel, A. (accepted). Reference Frames and Geographic Scale: Understanding Their Relationship in Environmental Learning. In Cartography and Geographic Information Science.

Zhao, J., Simpson, M., Wallgrün, J. O., Sajjadi, P., & Klippel, A. (2020, in print). Extended Realities – How Changing Scale Affects Spatial Learning. In IEEE VR Poster.

Zhao, J., Simpson, M., Wallgrün, J. O., Sajjadi, P., & Klippel, A. (2020). Exploring the Effects of Geographic Scale on Spatial Learning. In Cognitive Research: Principles and Implications, 5(14), 1–18. https://doi.org/10.1186/s41235-020-00214-9    Full paper

Zhao, J. & Klippel, A. (2019). Scale – Unexplored Opportunities for Immersive Technologies in Place-based Learning. 2019 IEEE Virtual Reality Conference. doi: 10.1109/VR.2019.8797867    Full paper

Graduate Research (2014-2016)

Visualizing Distant Landmarks on Mobile Phones

My interest in geospatial technology development had a specific focus during my master’s program: developing navigation systems that make a person more spatial oriented by visualizing off-screen landmarks on mobile screens. In this study, off-screen landmarks displayed on the edge of the mobile screen supported the user’s sense of direction during navigation by showing the concept of direction and distance to off-screen objects. The innovation point of this project is that this design embedded the concepts of both direction and distance that not only contributed to a person’s spatial orientation but also decreased the frequency of user interaction with the mobile screen such as zooming or panning. I implemented two approaches to visualize off-screen landmarks on a mobile screen, including using interval icons that changed in size continuously according to the distance and using ordinal icons that represented different ranges of distance. My research on this area produced one conference paper presented in AutoCarto 2016 and one journal paper published in KI – Künstliche Intelligenz (German Journal of Artificial Intelligence).

Approach of displaying distant landmarks on a small screen. Distant/Off-screen landmarks are re-projected onto the buffered area and resized in terms of their distance to the user (larger distance would reduce the size of edge proxies). The user location is indicated by a human icon in the center of the mobile screen.
References

Li, R., & Zhao, J. (2017). Off-Screen Landmarks on Mobile Devices: Levels of Measurement and the Perception of Distance on Resized Icons. KI – Künstliche Intelligenz, 7(27), 1–9. doi:10.1007/s13218-016-0471-7    Full paper

Zhao, J., & Li, R. (2016). Visualizing distance objects on mobile phones: Choice of resizable icons. In S. M. Freundschuh (Ed.), Conference Proceedings, AutoCarto2016. The 21st International Research Symposium on Computer-based Cartography and GIScience, Albuquerque, New Mexico, USA. September 14-16, 2016. (pp. 239–250). CaGIS.    Full paper

Zhao, J. (2016). Supporting spatial orientation: Using resizable icons to visualize distant landmarks on mobile phones (Master’s dissertation). University at Albany, State University of New York, Albany, N.Y.    Full paper