Online system offers photographers aesthetic feedback on their work

UNIVERSITY PARK, Pa. -- The advent of digital cameras opened up the field of photography to a new breed of amateurs. While the increased accessibility has been an advantage to many hobbyists, some may say that it has also led to an overall decrease in quality of photos. Consulting a professional photographer may not be an option for many amateurs, but researchers at Penn State are developing an online system that automatically analyzes the qualities of photographs and gives users immediate feedback based on highly-rated photos with similar properties.

“I think there is an untapped knowledge out there that photographers can leverage,” said Dr. James Z. Wang, professor of information sciences and technology, who leads the group that is developing the technology.

OSCAR (On-Site Composition and Aesthetics Feedback through Exemplars for Photographers), which is intended for general photos taken by digital cameras, provides a subjective evaluation of photo aesthetic quality and allows amateur photographers to "learn through examples.” Images similar in composition as well as content can be retrieved from a database of photos with high aesthetic ratings. The system responses include the most appealing color combination found in the photo, a general aesthetic rating and high-quality photo exemplars. The system was designed keeping the next generation photography needs in mind, Wang said, and is the first of its kind.

“We believe OSCAR is just the beginning of a photographic revolution,” he said. “It serves as a launching pad for a range of possibilities.”

OSCAR, which is patent pending, has been developed by the James Z. Wang Research Group at Penn State since 2010, although Wang said the system has been in the works for many years. The group’s research interests include automatic image tagging, biomedical informatics, art image analysis and retrieval, and image security. Doctoral students Lei Yao, Poonam Suryanarayan and Mu Qiao, funded by the National Science Foundation, were the main initial developers, working with their advisers, Wang and Jia Li, professor of statistics. The system was placed online for public use in June 2011, after the development of a real-time algorithm. Undergraduate student David Zhang worked on user interaction.

According to the researchers, interest in the research community on the plausibility of predicting the aesthetic quality of images has “increased dramatically over the past few years.” Amateur photographers or photo enthusiasts can potentially benefit from such a system by making use of the knowledge of professionals in taking photos, Wang said. Current camera phones or cameras do not have the capability to provide feedbacks on the aesthetics or composition of the photos.

“We believe with the increasing popularity of networked photographic devices, such as the camera phones, future-generation photographers, professional or amateur, have the potential to be much more efficient in their creative process,” Wang said. “They can employ the expertise of thousands of other professional photographers if proper information technologies can be developed to harness the vast amount of user-generated data already on the Internet.”

OSCAR can be utilized by uploading a photo from a local disk or providing a URL of a photo on the Web. The system can be ported onto a mobile device which travels with the user or can be seen as a cloud application through the wireless networks. As a photographer shoots, the photo is sent via the network to a server. OSCAR will provide immediate aesthetic quality assessment on the visual characteristics of the submitted photos, analyzing their composition and color properties, and send onsite feedback to the photographers.

“You can have a general sense of whether people will like your photo or not,” Yao said.

The researchers take a “data-driven approach” towards evaluating photographs, Wang said. The criteria on which the photos are scored were generated through a review of photography literature as well as online photo sharing sites. The ratings of the photographs that OSCAR generates are intended to reproduce the average scores users on the Internet may give.

After a photo is submitted for review, OSCAR retrieves composition relevant photographs from its database and displays the top nine ranked photos in a 3x3 grid. The ranking is mainly based on three clues: visual appearance, composition and aesthetics. The color combination feedback module finds the most aesthetically pleasing color combination from that image, with a score indicating the confidence of having high quality. A high color confidence indicates a larger chance that the color combination appears in high quality photos. OSCAR also reports a machine-generated rating, which is an estimate of photo aesthetics quality.

The image archive is used to store all the submitted images, which are labeled as “color images” and “monochromatic images.” Given an input image, the composition analyzer will analyze its composition properties from different perspectives. The composition analyzer performs spatial composition categorization, Yao said, and determines whether the image is textured, diagonal, horizontal, centered or vertical. For example, she added, pictures often appear more dynamic and interesting if there is a diagonal component in the composition.

At the same time, some aesthetics-related visual features are extracted by OSCAR. For instance, the system extracts from color images such features as light, colorfulness and size, which are provided to ACQUINE (an aesthetic quality inferencing system which the group developed earlier) for aesthetic quality assessment. The researchers have also developed a similar assessment module for monochromatic (black-and-white) images, which incorporates some other features, such as contrast, details, shapes and saliency.

In the retrieval module, a ranking scheme is designed to integrate the composition properties and aesthetic rating into SIMPLicity, an image retrieval system based on color, texture and shape features. Images with high aesthetic ratings, as well as similar composition properties and visual features, are retrieved.

OSCAR requires very little user input, the researchers said. Once the user takes a snapshot, he or she can choose to send the image to the system’s cloud server for feedback on composition and aesthetics. Based on the feedback, the user can go on to improve the composition of the image by taking new snaps of the subject and at the same time understand the ways in which they can improve the photograph. A demo is provided at http://acquine.alipr.com/oscar.

 

Media Contacts: 
Last Updated October 25, 2012