Shortly after the Heartbleed bug had been publicized CloudFlare posted a public challenge to compromise the private key of a server specially commissioned for the task. It took about 9 hours before Fedor Indutny proved it was possible, and shortly after CloudFlare confirmed that Fedor had used the Heartbleed bug rather than any other means to access this private key. This effectively means that every site that is or has been vulnerable can no longer treat their private keys as private, meaning those responsible for SSL Certificates on affected systems must reissue and replace SSL certificates on all affected systems and any other systems that are not affected by the bug but share an SSL certificate. CloudFlare have done a superb job of croud-sourcing an answer to a question that many of us wanted an answer to, you can see a full explanation on their blog.
Unless you have been living under a rock you will have seen mainstream news items on the recently publicised “Heartbleed Bug” in OpenSSL. Depending on who you listen to it is either the end of the world, the end of the Internet or possibly just a hickup along the way. I personally subscribe to the latter; bugs and defects are inevitable and this defect is a tiny drop in the ocean compared to all of the great Internet things that OpenSSL has enabled.
When it comes down to it its another bug that was found and fixed, and probably one that will be forgotten in time, Netcraft estimates that 17% of websites are vulnerable and the individual vulnerability of each of the top 10,000 most popular sites is being tracked by users across the Internet. According to IsTheInternetFixedYet.com as of 12th April 2014 it appears that 83% of previouly vulnerable websites have now been fixed which is a huge testament to the hard work of Systems Administrators and Operations Teams across the globe.
As Systems Administrators, DevOps, Developers, Engineers and Users of OpenSSL either directly or indirectly we have a responsibility to manage the risks that Heartbleed poses. The remainder of this post will go into some of the ways this affects these groups of people.
GENERAL HEARTBLEED ADVICE
If you operate a website that uses OpenSSL to serve content over an encrypted channel and SSL Heartbeat is or has ever been enabled then the following steps would be prudent:
- Update OpenSSL or recompile with heartbeat disabled.
- Reissue and redeploy all SSL certificates.
- Revoke all old SSL certificates, consider them compromised.
- Communicate with your users, let them know what has happened, what you have done and what they must do to stay secure. For some websites this may be as simple as a mass mail explaining the problem and suggesting password resets. For other websites protecting more sensitive data it may be most appropriate to force all users to reset and validate their user accounts. The response must be proportionate to the degree of risk to the users personal data and the reputation of your website or company.
DEVOPS, DEVELOPERS AND SYSTEMS ADMINISTRATORS
If you are responsible for any infrastructure, now is a great time to start employing continuous infrastructure testing. You can start with a Extra Small virtual machine install Ubuntu and Gauntlt and start scanning your infrastructure for Heartbleed and many other vulnerabilities while you are at it.
Those of us fortunate enough to operate infrastructure that supports some degree of Software Defined Networking could go as far as inspecting HTTPS traffic for potential Heartbleed exploits and stop the packets in their tracks. This should not be a substitute for timely patching but may provide some breathing room until maintenance windows can be agreed with business units.
END USERS OF 3RD PARTY SERVICES
If you use any service that is or has been vulnerable:
- Wait until the service provider has confirmed they are no longer vulnerable.
- Reset your passwords and API keys.
- Reset the passwords of any other service that shared a password with the vulnerable service.
At this stage it is unclear if or to what degree client devices such as PCs, Phones and Tablets are vulnerable to the Heartbleed Bug, this will likely become evident over the course of the coming days. However it is always prudent to ensure your device is running the latest versions of software to protect against exactly this kind of issue.
REDUCING RISK THROUGH CONTRACTING
One of the key benefits of outsourcing services to cloud providers is the reduction in risk gained by removing responsibility for infrastructure and platform concerns. This has been achieved by thousands of enterprises by contracting out to Cloud Service Providers such as Shoppify, SalesForce and Office 365.
These Cloud Service providers have proven their ability to react quickly to a newly discovered vulnerability and patch their services quickly and effectively.