UNIVERSITY PARK, Pa. — When toddlers play, there’s more going on than meets the eye. Play and other natural activities parents engage in help developing children build social, emotional and locomotor skills, among others.
To help tease apart these processes, a team of researchers led by Penn State and New York University faculty will collaborate to build a digital library of videos of mothers going about their daily activities with their children. A $6.3 million grant from the National Institutes of Health will fund the project, called Playing & Learning Across a Year (PLAY).
Rick Gilmore, associate professor of psychology at Penn State and one of the researchers co-leading PLAY, said the project will provide more than 60 PLAY project researchers and thousands of others with a rich source of data for future studies.
“Between 12 and 24 months, children undergo an amazing period of rapid development where the human mind emerges and flourishes like never before,” Gilmore said. “With PLAY, we will use video to capture the complexities of this essential period of development. By sharing the data openly with researchers, we will seed new scholarship that uses and builds upon this data.”
The researchers will gather video data of more than 900 infants and their mothers from 30 diverse sites across the United States. The videos will then be transcribed, annotated and uploaded to Databrary, an open-source video library that Gilmore and colleagues at New York University have developed over the past several years.
Gilmore said the PLAY team includes experts from four different domains: speech, language and gesture; emotional expression; physical activity; and object interaction. Each expert will analyze and annotate the videos for behaviors related to their specialty.
“For example, the experts coding for speech and language will record, for each video, all utterances from mom and baby and the category of those utterances and gestures,” Gilmore said. “There will be codes for the behaviors from all of the domains. The codes serve as bookmarks or indices into the videos, which researchers can then use to search for exactly the kinds of behaviors they want to study.”
In the future, after the videos are uploaded to Databrary, researchers will be able to use the data in studies of their own. In addition to the videos, data from parent questionnaires and demographic information about the participants will also be available.
Gilmore said the project is an opportunity to apply big data, computer vision, and machine learning techniques to developmental psychology.
“How are researchers in the future going to find data they want? Are they going to have to manually go through 900 hours of video, or can we provide ways for them to search and filter the data to get exactly the subset they want for their particular question,” Gilmore said. “Thinking about how we build these kinds of tools is exciting to me, and I enjoy working with a strong group of researchers here at Penn State with similar interests.”
Karen Adolph, professor of psychology and neural science, and Catherine Tamis-Lemonda, professor of applied psychology, both from New York University, are co-leading PLAY with Gilmore.
Kristin Buss, professor of psychology and human development and family studies; Koraly Perez-Edgar, professor of psychology; Sheri Berenbaum, professor of psychology and pediatrics; and Guangqing Chi, associate professor of rural sociology and demography; and public health sciences, all from Penn State, are also working on PLAY alongside Gilmore. Kasey C. Soska, a 2005 honors graduate in psychology from Penn State, serves as PLAY's scientific director.