UNIVERSITY PARK, Pa. -- When developing entertainment and multimedia products, companies must ensure that users will enjoy a positive experience by interacting with their services. Jack Carroll, a distinguished professor at Penn State’s College of Information Sciences and Technology (IST), and Mary Beth Rosson, a professor and associate dean for undergraduate studies at IST, have teamed up with Microsoft for a project that applies their research on self-efficacy — a measure of the belief in one's own ability to complete tasks and reach goals — to develop better evaluation techniques for Xbox, an entertainment and gaming system.
“We were contacted to help (Microsoft researchers) work on tools and techniques to assess what is called the ‘user experience,’” said Carroll, who directs the Center for Human-Computer Interaction at the College of IST.
Carroll and Rosson, who were awarded $50,000 by Microsoft to complete the project, are assisted by Jake Weidman, a doctoral candidate at the College of IST. The Microsoft team that they are collaborating with includes Umer Farooq, one of the team leaders in the Xbox division, who received his doctorate from the College of IST in 2007 and studied under Carroll.
“This (project) is also a great example of one of our students going out into the world, succeeding, and then coming back and working with us,” Carroll said.
The Xbox brand includes a series of video game consoles developed by Microsoft, which offer a myriad of applications, streaming services and a proprietary online service, Xbox Live. The brand was first introduced on Nov. 15, 2001, in the United States, with the launch of the original Xbox console.
According to Carroll, Microsoft has a “detailed taxonomy” of 49 experiences that users can have with Xbox services. To analyze these experiences, user experience researchers observe people interacting with Xbox services and score their experiences on a three-point scale — “good,” “neutral” or “not good.” Farooq, a user experience researcher for Microsoft, was familiar with Carroll’s research on self-efficacy, as he and Carroll had worked with the psychosocial construct in the past to study people’s beliefs on their ability be a capable member of a scientific research team. The Microsoft project operates under the assumption that experience outcomes could be studied by assessing a person’s belief that they can attain that outcome. The Microsoft researchers asked Carroll and Rosson to theorize user experience outcomes as self-efficacies.
“When someone buys an Xbox, they’re not looking to have an easy interaction, they’re looking to have a pleasant interaction,” Carroll said. “If the system enhances that, that’s a successful design.”
Over the summer, Carroll, Rosson and Weidman did preliminary work with Microsoft in which they designed self-efficacy scales for nine of the 49 experience outcomes developed by Microsoft. They asked subjects to perform a suite of Xbox tasks, and make self-efficacy judgments about their experiences in these interactions. A Microsoft expert user experience researcher also participated, observing the test subjects and judging their apparent self-efficacy. Carroll and Rosson “studied the initial psychometric properties of judgments to see if it’s feasible to make up a self-report scale using this self-efficacy construct to assess user experience outcomes,” Carroll said. They then wrote up a short report, which they sent back to Microsoft.
“They’re going to see if they can use what we developed,” Carroll said. “If that’s promising, we may go forward and do more.”
Carroll said that he and Rosson may expand their research to additional user experience outcomes besides the nine outcomes that they worked with over the summer and may also investigate how those outcomes are related.
“We can investigate many questions regarding user experience outcomes, if we have a measurement instrument that’s reliable and powerful,” Carroll said.