New computer program to examine collaborative online learning

UNIVERSITY PARK, Pa. — An effort to transform burgeoning online learning from being essentially individualistic to considerably more collaborative is gaining ground, according to a researcher who is breaking ground about the usefulness of online communities.

Marcela Borge, assistant professor in learning, design and technology in Penn State’s College of Education, earned a National Science Foundation grant for her work on "Fostering Ecologies of Online Learning Through Technology Augmented Human Facilitation."

The goal of the learning sciences project is to find ways to use technology to better meet the needs of online students by developing technological support for cognitive, metacognitive and social processes. The project has produced many interesting outcomes, including a new model for supporting the development of socio-metacognition, the ability to monitor and regulate collective thinking processes in online collaborative learning contexts. 

“We were looking at online learning because it’s becoming an increasingly important aspect of education,” Borge said. “Research in the learning sciences has shown that collaborative processes like discourse and collective sense-making are essential for learning. For this reason, we wanted to make sure that students who are learning in online contexts have equal access to meaningful learning experiences: collaborative learning, deep sense-making, building relationships with other students.”

Borge said the online format is usually the only way those students have to develop social relationships and engaging in cognitive forms of social discourse around course content.

“The problem is that really good discourse requires a certain amount of facilitation, because people are not generally good at collaboration or sense-making discourse, and there’s a great deal of research that supports that,” she said. 

Because there isn’t always an online facilitator to ask the “deeper” questions, Borge and her colleague, Carolyn Rosé from Carnegie-Mellon University in Pittsburgh, combined to examine how to use technology as a means to help student groups improve their own collaborative processes and prevent students from dropping out of massive online environments.

“We combined forces because we both recognized the importance of collaborative learning and the many problems that can prevent it,” Borge said. “My design expertise was needed to create and test a tool to help students learn how to monitor and regulate their own collaborative activity. Carolyn’s area of expertise, machine learning and natural language processing, was essential to the project because we needed to find a way to automate assessment of collaborative sense-making and give students feedback.” 

It was a potentially difficult feasibility project, Borge said, because there are few tools out there that actually show that they can improve students’ ability to monitor and regulate collaborative interactions. She stressed that higher-order thinking is becoming much more critical.

“Multiple iterations of our study have shown that we can help students learn to regulate collaborative processes and actually improve them,” she said. “We have looked at the quality of students’ conversations across an entire semester and we found that they significantly improve over time through use of the CREATE (Collective Regulation & Enhanced Analysis Thinking Environment) technology and related activities. However, the other thing we were trying to do is see if we could find a way of automatically assessing the quality of discourse.”

Borge said she and her students currently do that manually, which is time consuming. “We took our manual coding framework and worked with Carolyn to see if we could use machine-learning to develop a way to do it automatically,” she said. “We were partially successful; we could get the computer to automatically code the quality of discourse but only if we had a human in the loop.”

They were able to get the machine to automatically code discourse acts, whether students were adding information, making judgments, or requesting evidence or elaboration. Where it became more precarious was when these acts were aggregated for the entire session, and Borge and Rosé were looking for a combination of patterns that would allow them to score the quality of higher-level cognitive functions. 

The available options — even if some are unknown — are what makes the learning sciences exciting for Borge. “Part of the reason I love the learning sciences is because not only does it try and push the boundaries of new technologies to support learning and evaluating technologies that exist, but it’s extremely theoretically and methodologically rigorous,” she said.

“For those folks who are not familiar with the learning sciences, it is important to understand that analysis of learning occurs at multiple levels of scale in real-world contexts and there is a push to look at learning in social contexts — how do interventions, technology and tools impact discourse processes in the classroom?“ Borge said.

Part of the process of the system being studied is trying to determine what quality collaborative processes concretely look like. Borge explained the CREATE system as students being asked to conduct a one-hour synchronous conversation with other online students about their perspectives on difficult questions about course content.

“The system has tools to help students reflect on the quality of this discourse,” Borge said. “We operationalized what quality collaborative discourse looks like by drawing on theory. The system helps students monitor and regulate collaborative processes by having them assess and reflect on key processes.

“Students have to scroll through and look for certain patterns that are associated with higher quality sense-making and then evaluate where they are compared to where they would like to be and come up with strategies for them to improve next time around,” she said. The system then helps students make plans on how to correct problems in a future session and provides awareness features they can use to track their processes.

Borge said she is working with Teaching and Learning with Technology (TLT) because TLT would like to turn her prototype into a University-wide system. The team at TLT includes TK Lee, Audrey Romano, Robin Smail, Serena Epstein, Heather Harter, Jason Heffner, Kathy Jackson and Bart Pursel.

“There are a lot of folks excited about the CREATE extension, particularly those working with team-based analysis” including business schools and information sciences, Borge said.

Media Contacts: 
Last Updated October 04, 2017