Strategic Plan seed grant advancing immersive learning experiences at Penn State

UNIVERSITY PARK, Pa. — Imagine a world where space and time do not matter, where it’s possible to witness critical events in the history of the Earth and humankind, or have a sneak peek into the future.

That’s what Penn State researchers, through the help of immersive technologies such as virtual reality (VR) and investments in the University's infrastructure, are hoping to accomplish with a Penn State Strategic Plan seed grant.

In two pilot experiences, Penn State students are traveling to Iceland to investigate the inner workings of a volcano and jaunting to a place to study its environment more than 425 million years ago, all without setting foot outside the classroom.

These virtual field trips are changing the way students are learning in geoscience and laying the blueprint for how others can embrace these new immersive learning experiences — while advancing the Strategic Plan's goal of transforming education. 

Virtual field trips can be shared across the Commonwealth, they provide accessibility to critical field sites across the globe, they are immune to bad weather, will be available for online learners, are cheaper, and have a very small carbon footprint. Additionally, they are safe, allow for temporal flexibility, and offer experiences not possible in the real world.

For example, students can instantly travel to both sides of the Atlantic Ocean to experience evidence of plate tectonics, looking at outcrops of the Appalachians and those formed during the same time period in western Europe, within the same lab session to witness the similarities in how each were created.

Peter La Femina, associate professor of geosciences, and Alexander Klippel, professor of geography and Gosnell Senior Faculty Scholar, are using the seed grant to lay a foundation for creating these trips and provide empirical evaluations to understand how immersive experiences enhance education.

“Virtual field trips have the potential to increase accessibility in geoscience education by providing field opportunities to students across the Commonwealth who normally would not have access to geologic field sites,” La Femina said. “Additionally, these experiences can be used to augment existing field labs and provide background for a field trip or field research project before the students or researchers go in the field.”

How virtual comes to life

To create virtual experiences, the researchers use a combination of 360-degree images, high-resolution photography and photo-based measurements, or photogrammetry, to develop virtual representations of the real regions. In addition, the researchers use high-resolution, 3D-imaging equipment to capture the terrain and drones for photogrammetry. Additional information, such as natural sounds and textures, also can be gathered and included to enhance the experience. However, the fastest way for someone to create an immersive experience of a particular location is by using 360-degree images. ChoroPhronesis, Klippel’s research team, uses high-end cameras with 36 lenses to collect images up to 108 megapixels.

These materials are then stitched together using Unity, a game engine, while augmented features, like measuring tools in the case of the volcano experience, narrations, and supplementary information such as charts, are added to the experience. Additional instruments, planned for future developments, will allow users to touch and feel the environment.

These virtual field trips are accessible using HTC Vive and Oculus Go at a growing number of locations at University Park, including ChoroPhronesis, the Pulse of the Earth lab, The Dreamery, and the Fletcher L. Byrom Earth and Mineral Sciences (EMS) and Pattee libraries. Penn State’s investment in immersive virtual reality technology across the Commonwealth Campuses means these resources can be shared far beyond University Park.

La Femina and Klippel, in collaboration with colleagues in Teaching and Learning with Technology (TLT), also are comparing the learning experiences for these virtual trips with data they gathered from students who took part in conventional field trips. In a series of recent experiments, researchers found knowledge gained during virtual field trips matched traditional experiences. The goal, they said, is not only to improve immersive experiences for students, but to improve our understanding of how effective immersive experiences are created.

The activities facilitated through the seed grant also organized an immersive community at Penn State, a group of more than 120 members and counting.

“Penn State has the potential to lead immersive learning efforts internationally,” Klippel said, and researchers are spearheading an effort to build a network of people interested in immersive technologies and learning experiences. To inspire undergraduate students and spark entrepreneurial ideas early on, Klippel, in collaboration with TLT, developed a general education course that teaches students how to design these experiences.

La Femina and Klippel are continuing to test and improve these experiences while forming guidelines to roll out the tools to faculty members University-wide. 

A learning tool with reach

Jiayan Zhao, a doctoral candidate in geography, has an academic interest in creating VR technology and sees its value as a learning tool. Zhao is a member of ChoroPhronesis and helps create much of its immersive learning technology.

In addition to the virtual field trip, Zhao worked on a project in collaboration with Ping Li, professor of psychology, and Li's doctoral student, Jennifer Legault. Zhao created a virtual zoo and a virtual kitchen to study whether immersive experiences foster second language learning such as Mandarin. In that study, the team found less successful learners benefited particularly from the immersive experiences, demonstrating the importance of adapting learning environments to learner characteristics.

Creating environments that lead to more efficient learning for those struggling with concepts is something that appeals to Zhao.

“The context that immersive learning provides can help to bridge the achievement gap between low- and high-ability learners,” he said.

Making immersive learning a reality

At Penn State, immersive learning is supported through TLT and University Libraries, which means tools for designing and using this technology are accessible to all.

Seeing the broad appeal, Klippel and La Femina designed their approach to conform with technology available at common locations such as The Dreamery, the libraries and the IMEX lab, an immersive learning facility with tools for mobile VR experiences.

At EMS and Pattee libraries, students can book VR rooms to complete assignments. Access to this technology is critical to its expansion in higher education, the researchers said.

Chris Stubbs, manager of emerging technology and media at TLT, said immersive technologies allow the viewer to participate in improbable experiences, such as conducting experiments on the surface of Mars, as well as more hands-on experiences, such as assembling the engine of a car.

“But whether or not you are doing something outlandish or mundane, these sorts of technologies also offer you a safe place to experiment, fail and work through ideas or concepts at your own pace,” Stubbs said. “Coupled with the level of engagement they afford, immersive experience technologies have tremendous potential as a learning tool.”

Kyle Bowen, director of innovation at TLT and a frequent collaborator with Klippel and La Femina, said Penn State is a great place for immersive learning to thrive because its broad appeal pairs nicely with the University’s broad research efforts. He sees the technology as a way of adding context to learning and research.

“One of the great opportunities is you can now provide context for how things happen and when they happen,” Bowen said. “That makes it possible to recreate environments that would otherwise be inaccessible or environments that would be impossible. It allows you to experiment, to engage in a world that isn't bound by reality. That allows you to tell stories, to demonstrate ideas in ways that wouldn’t have otherwise been possible.”