In a threat report published by U.S. Director of National Intelligence James Clapper last year, cyberattacks were listed first among global threats, above both terrorism and weapons of mass destruction. To combat those threats, companies have increasingly turned to "white hats" -- ethical hackers who expose vulnerabilities in computer systems to improve cybersecurity rather than compromise it. Researchers at Penn State's College of Information Sciences and Technology (IST) are investigating the dynamics of these "bug bounty" programs with the intention of helping organizations amp up their defenses against malevolent hackers.
"The major goal is to better understand this ecosystem," said Jens Grossklags, Haile Family Early Career Assistant Professor of Information Sciences and Technology.
Grossklags, along with Mingyi Zhao, a doctoral candidate at the College of IST, and Peng Liu, professor of IST, presented their paper "An Empirical Study of Web Vulnerability Discovery Ecosystems" last fall at the ACM Conference on Computer and Communications Security (CCS), the flagship annual conference of the Special Interest Group on Security, Audit and Control (SIGSAC) of the Association for Computing Machinery (ACM). In January, they also presented their results to a public policy audience at the Federal Trade Commission's PrivacyCon event to raise awareness about the challenges and opportunities of bug bounty programs.
According to the IST researchers, web security has become critically important for most organizations, and the prevention of security compromises enabled by web vulnerabilities is increasingly gaining the attention of company leadership and the broader security community. Web vulnerabilities are the likely causes of many recent security breaches leading to massive disclosure of user data, leakage of business information and other losses.
While the researchers had previously investigated white hat behaviors, Zhao said, their current study is more comprehensive and examines web vulnerability discovery ecosystems -- including businesses and organizations, white hats and third-party vulnerability disclosure reward/bounty programs.
"These ecosystems have been growing rapidly and are becoming more prominent in the battle against malicious actors on the Internet," Grossklags said.
"We are trying to determine how to shape these ecosystems in a way that diminishes potential disadvantages and provides the maximum benefit."
In their work, Grossklags, Zhao and Liu conducted the first empirical study of two major web vulnerability discovery ecosystems, basing their analyses on publicly available data. The first dataset stems from Wooyun, the predominant and likely the oldest web vulnerability discovery ecosystem in China. The second dataset was collected from HackerOne, a US-based start-up company that hosts bug bounty programs for hundreds of organizations, including Yahoo, Mail.ru and Twitter.
An important difference between Wooyun and HackerOne, Zhao said, is the number of organizations that are involved. While HackerOne has 99 organizations running public bounty programs, Wooyun affects 70 times more organizations.
"This drastic difference can be explained by the difference in organization participation models," he said. "For HackerOne and other similar US-based platforms such as BugCrowd and Cobalt, an organization has full control of its bounty program. It determines when to start, what are the rules and scope of vulnerability discovery, and whether to disclose the reports. We refer to this model as the white-hat-initiated model."
On the other hand, Zhao said, Wooyun and some other Chinese platforms give more control to the vulnerability researchers. White hats can submit vulnerability reports for almost any organization. The organization can claim the report and work with the white hat to fix the issue. However, if the organization fails to do so in 45 days, the report will go public.
Grossklags and Zhao said that the two types of ecosystems, HackerOne (organization-initiated), and Wooyun (white-hat-initiated) have different pros and cons. They found that the white-hat-initiated participation model covers a wider range of organizations, including government sites, and financial and educational institutions.
"To improve the coverage of bug bounty, we might want to give more control to the white hats," Zhao said.
Zhao and Grossklags also discovered that many organizations in the Wooyun ecosystem are not prepared to deal with the reported vulnerabilities, particularly smaller websites. The white-hat-initiated model may increase risk for unprepared organizations, Zhao said, since vulnerabilities with no response likely remain exploitable. Once a vulnerability becomes public knowledge, companies may feel mounting pressure to handle the situation.
"Eventually, it might build enough pressure that the companies are forced to react," Zhao said.
In contrast, the researchers found that the organizations that run bounty programs on HackerOne are able to resolve vulnerability reports in most cases in a reasonable amount of time. Further, a large portion of public bounty programs on HackerOne show a decreasing trend of vulnerability reports over time, which suggests continuous improvements of cybersecurity of these participating organizations.
While Zhao and Grossklags discovered that monetary compensation of white hats on HackerOne increases their productivity significantly, they still observed many contributions to programs without bounties. Contributing to vulnerability reports increases white hats' standing in their community, Grossklags and Zhao said, while also making the Internet more secure. In addition, the software engineering community and peer organizations can learn valuable lessons from vulnerability reports to avoid similar mistakes in the future.
One motivation of their study, Grossklags and Zhao said, is to inform the public policy process as it affects vulnerability discovery. According to Zhao, the proposed new treaty of the Wassenaar Arrangement -- a voluntary agreement among 41 countries that calls for regulating the knowledge of how to create "intrusion software" -- can significantly limit vulnerability research and impede the development of bug bounty programs.
"Our study effectively shows that white hats make important contributions to cybersecurity, but they can be significantly limited by such initiatives," Zhao said.
Moving forward, Grossklags said, he and Zhao will conduct more in-depth empirical and theoretical research on discovered vulnerabilities. A further key objective is to help consumers to make choices between website services based on their security. Vulnerability research can provide important input for such guidance.
At the upcoming TEDxPSU event on February 28, Grossklags will present additional results from the research project to a broader audience at Penn State and on the web.