By Dave Huseby — Hyperledger Security Maven
In October of 2017, the Hyperledger community launched our first bug bounty program through a partnership with HackerOne. This is in addition to the independent security audits we have initiated to vet our software. Now that the first year of the program is coming to a close, it is time for an update on the impact it has made so far.
We’ve found that increasing our bug bounty payouts has attracted more interest among analysts, and we’ll make even more changes in 2019 to boost bug reporting.
At the launch of the program, only Hyperledger Fabric was in a position to be included. Initially, the bug bounty program was kept private and only open to people in HackerOne’s pool of vetted security analysts because of the relatively new nature of the Fabric code base. By limiting the pool of analysts, we were hoping to better control the number of security bugs coming in.
HackerOne sent invitations to 174 analysts in their pool. Of those 174, only 72 accepted the invitation and joined our program. We were excited to finally have outside people trying to break our software. Unfortunately, at the end of the first six months of the program, we were not where we wanted to be.
To better understand why we are seeing a general lack of interest, HackerOne surveyed the analysts invited to the program to ask why they were not participating in the bug bounty. The results are listed in Table 1.
| Specialization | 26 |
| Uninteresting | 17 |
| Competitiveness | 11 |
| Small Scope | 10 |
| Onerous Setup | 9 |
| Skills Mismatch | 7 |
| Clarity | 6 |
| Unresponsive | 6 |
| Hardened | 5 |
| Objection | 5 |
| Unattractive | 4 |
| Access Criteria | 2 |
| Aggressive Policy | 2 |
| Low Payouts | 2 |
| Small Scope | 2 |
| Unclear Payout Structure | 1 |
| Total | 115 |
Table 1 — HackerOne Survey Results
We know that blockchain platforms are novel and different from what the HackerOne analysts are used to working with, but we had hoped that it wouldn’t matter. From the survey results, it appears that most analysts weren’t interested because of the high degree of specialization needed to mount effective attacks against the software. To offset the specialization required, we decided to increase our bounty payouts to move out of the “uninteresting” category.
The bug bounty program opened to the public this past April with increased bounty amounts. Immediately we had more interest, however we received a flood of bug reports from people who didn’t take the time to read our program rules and weren’t aware that we are an open source organization. However, we did receive three bugs worth paying attention to. Two were configuration issues with our infrastructure and one was a bug in Fabric. All were fixed and small bounties were paid.
Our bounty award levels are right near the industry median, but based on the required levels of specialization needed to effectively attack our networks, we may need to raise our bounty payouts even further to increase the level of interest. Table 2 lists our current payouts.
| Critical | $2000 |
| High | $1500 |
| Medium | $500 |
| Low | $200 |
Table 2 — Current Bounty Payment Schedule
Moving into 2019, we will reassess the efficacy of our bug bounty program and change our direction. One obvious thing is that we’re not getting the level of interest we had hoped for. Either Hyperledger Fabric has airtight security or we’re not doing enough to interest analysts and hackers. If I had to bet, my money would be on the latter. We’re exploring a number of ideas, from more marketing, using the available bounty budget better through limited time bonuses and offering other incentives like all-expenses-paid trips to Hyperledger events for anybody who reports a critical bug. The discussion is taking place on our Technical Security Mailing list right now: here and here.
Overall, we need to have a well-rounded security process that follows all of the best practices. We have a robust and public bug tracking system and development discussions. All of the teams have rules around code reviews, and they do a good job of managing risk when carving new releases. Over the past year, we have conducted outside security audits of Hyperledger Fabric, Sawtooth, Iroha, Composer, and Indy is underway. The bug bounty is underway and we’re discussing expanding it to include Hyperledger Sawtooth, Iroha and Indy. The idea of running test nets is also being considered.
You can stay connected and participate in that discussion to help us make even more effective use of our security resources in 2019. And as always, you can keep up with what’s new with Hyperledger on Twitter or email us with any questions: info@hyperledger.org.