Music professor developing AI-driven virtual classroom

August 09, 2016

Ann Clements, associate professor and the graduate program chair for music education in Penn State's School of Music, is exploring how a virtual reality classroom featuring artificial intelligence (AI) "students" can be used as a training tool for pre-service student teachers. The AI-driven virtual classroom offers opportunities to practice various student engagement techniques that these pre-service teachers may not have had before.

"This technology provides students with an immersive experience where they can make meaningful mistakes in a low-stakes environment," Kyle Bowen, director of Teaching and Learning with Technology (TLT) at Penn State, said. "Through repetition and practice in a virtual classroom, future teachers can find their voice and make the most of time in the live classroom."

Clements worked with members of Penn State's Teaching and Learning with Technology (TLT) to develop this AI classroom prototype, known as First Class. Utilizing rows of virtual students in a classroom environment and Microsoft Kinect, a motion-sensing input device usually used with video game consoles like the Xbox, First Class allows preservice teachers to practice teaching skills and techniques as well as experience the classroom in terms of the varying abilities of their future students.

"Teaching is a complex craft because it involves human engagement," Clements said. "So we can talk to someone about how to teach and best practices and provide them with curricular content, but they need to engage with students in some way to have the skills to become solid teachers."

One of Clements' students, Brandon Buterbaugh, a senior music education major, was one of the initial users of First Class. He tested three modules, including those that focused on line-of-sight, student greetings, and proximity control. Buterbaugh said he would call virtual students by their names and they would make "eye contact" and raise their hand. Attention meters for each student gauged if students were paying attention. "When I saw a group of kids misbehaving or talking and their attention meter was down, I would move in that general direction to engage them more," Buterbaugh said.

"After completing the three modules, I was given a lesson to teach," Buterbaugh said. "While I was giving the lesson I had to keep in mind everything like saying the students names, making sure they could see me and my proximity to them.

First Class even evaluated Buterbaugh by enabling him to give the students a basic yes-or-no answer test. "Based on how well I did, that is how well the virtual students answered the questions," he said. "Their score on the test was really my score to how well I did teaching the lesson."

Clements stresses that First Class is not meant to be a replacement for in-person student teaching, but as a preparation tool and as a way to address issues pre-service teachers face getting access to the classroom. She notes that high-stakes testing in K-12 schools can last weeks, a time when student teachers are not allowed access to classrooms. "We're just trying to give pre-service teachers an early opportunity to practice teaching skills that they might not even have a chance to practice if we didn't use the virtual reality classroom," Clements said.

With an eye towards future possibilities, including addressing accessibility issues, work on the virtual reality classroom is continuing. "I really want to see virtual reality create a classroom environment that helps a pre-service teacher understand challenges various disabled students face, and how to address them," Clements said.

"I also see other uses for this, such as providing the student with extreme teaching conditions that pose ethical or moral dilemmas," Clements said. "Just things that a typical student teaching experience might never provide, so education students are prepared when they do happen in the future."

Beyond this, Bowen said he foresees other uses for FirstClass and hopes to explore them. "While designed with teacher education in mind, this platform has a wide variety of applications ranging from public speaking to intercultural awareness," he said. "We are actively seeking research partners to help expand and explore this new technology."

(Media Contacts)

Last Updated August 24, 2016