For pathologists, identifying damaged or diseased tissue is a time-consuming process of poring over samples under a microscope. But collaborative research between veterinarians and electrical engineers at Penn State could significantly speed up the process.
Animal Diagnostic Laboratory
Work by a team of Penn State electrical engineers and veterinarians may one day allow for computers to examine medical samples, such as these kidney cells to determine whether they are healthy, top, or inflamed, bottom.
Using image recognition technology similar to what powers photo editing software and social media to recognize faces, a team led by Vishal Monga, assistant professor of electrical engineering, with veterinarians at the University’s Animal Diagnostic Laboratory (ADL), has developed an automated method of classifying histopathological images.
“Image processing researchers routinely solve image classification problems,” Monga says. “The idea is if you have a collection of images, can you automatically put them in categories?”
Monga continues, “What was particularly interesting was [ADL veterinarians] were capturing tissue imagery—histopathological images.”
During the course of a year, ADL’s five pathologists examine more than 10,000 slides, according to Art Hattel, veterinary pathologist at ADL. To properly evaluate a slide, Hattel says, a pathologist usually takes seven to ten minutes, but the process can take as long as 20 to 25 minutes. “It’s time consuming. You have to look carefully or you’ll miss something,” he says.
At a chance meeting at a social gathering Monga was telling Bhushan Jayarao, ADL’s director, about his research into image processing and classification.
As Jayarao recalls, “I told Dr. Monga, why don’t we apply this to some of the work Dr. Hattel does? This is total virgin territory. No one has really ventured into histopathology.”
“Although image classification has been around for a while,” Monga says, “these images are nothing like the typical images you would see. They’re not photos of you and I or landscapes. The geometries and structures that they exhibit are entirely different.”
Vishal Monga
After initial tests showed it was feasible to use image classification on histopathology, the team began in January when Hattel received a $19,000 research grant from the Pennsylvania Department of Agriculture to initiate the project.
A series of training slides were sent to Monga’s team. In addition to the sample images, Jayarao says, ADL pathologists sat down with Monga and his graduate student, Umamahesh Srinivas, to help explain things.
Hattel says the engineers told him, “‘We want to see these images and why you’re calling this normal and why you’re calling that abnormal.’”
Starting with diseases that featured many lesions, Hattel says the ADL staff explained what pathologists look for in diagnosing the samples, including color changes, differences in shapes and sizes or an influx of white blood cells.
To accomplish the task, Monga’s team developed a set of custom tools which rely on a sparse encoding of image attributes to help with identification.
Srinivas explains that the electrical engineers designed the tools mimicking the same methods human pathologists use to classify samples. “They look for distinguishing features.”
Once the tools were developed, Monga says, the automated solution proved to be quite accurate.
“We have benchmarked against human judgment and we are already seeing 80- to 85 percent success in automatically categorizing into three areas—healthy, inflammation and necrosis,” Monga says.
“We were surprised with how well it picked things up,” Hattel says.
With the initial success using the training images, Jayarao says the system could potentially be scaled up to examine other types of abnormalities.
To do so, Monga says the software would require a “training phase” where it is taught what healthy tissue looks like and what diseased tissue looks like. But once the big step of pre-processing is finished, the software could process thousands of images in a second.
In addition to ADL’s large collection of images, the system could incorporate the massive disease databases maintained by the National Institutes of Health, the U.S. government and other academic and private institutions, Jayarao says.
Although the new method isn’t being used by ADL to diagnose any real samples yet, the researchers are applying for larger grants to continue the effort in the hopes that one day it might.
“This is only the tip of the iceberg,” Monga says. “Right now we’re just answering questions like whether it’s healthy or diseased. But we would like to answer more elaborate questions like what kind of disease and what stage of disease because then it can become a very powerful diagnostic tool.”
Vishal Monga, Ph.D., is Assistant Professor of Electrical Engineering, vmonga@engr.psu.edu. Umamahesh Srinivas, M.S., is a doctoral student in Electrical Engineering. Arthur L. Hattel, D.V.M., D.A.C.V.P., is Senior Research Associate in the Animal Diagnostic Laboratory. Bhushan M. Jayarao, Ph.D., is Professor of Veterinary and Biomedical Sciences and Director of the Animal Diagnostic Laboratory.