We probably can’t prevent the next Heartbleed. But that’s OK.

John Breeden writes the Technocrat column on FedScoop.com. 

Greetings to all my fellow techies. By now, you have all probably heard about the Heartbleed bug. FedScoop posted a great tutorial with advice on how to patch the flaw on servers and what users can do to protect themselves. Basically, from a user’s prospective, you need to change your passwords, but only after the sites you work with fix the problem on their end. It takes two to tango, and to fix Heartbleed.

Healthcare.gov, even with all its previous and widely reported problems, could actually become a model for eliminating Heartbleed. They fixed their servers over the weekend, and then promptly expired all user passwords, forcing everyone who uses that site to reset their passwords the next time they log in.

It may seem a bit draconian, but that’s probably the best way to handle things in this case, as a proper fix requires both site and user participation. If left to do it themselves, you know that quite a few users would drop the ball.

And it’s a good thing that government sites are acting quickly. Now that Heartbleed is common knowledge, hackers will begin to exploit that bug. A Canadian hacker was recently arrested after using Heartbleed to steal the proprietary information of about 900 taxpayers in that country. Given that Heartbleed affects about two-thirds of the Internet, there are a lot of juicy targets.

I, for one, am fascinated about how Heartbleed was created. Though, I have to admit, while I’m grateful as a journalist to have something to call this bug, I don’t really think it should have gotten a proper name. Viruses and worms that have disrupted the world get named, sort of like hurricanes, but this was a simple coding mistake. Unlike Code Red, I LOVE YOU, PoisonIvy and Melissa, there was no malice involved in Heartbleed. In fact, the bug did exactly what it was supposed to do. It just had unintended consequences.

OpenSSL, the encryption program that protects most of the Internet and which is vulnerable to Heartbleed, is open source. That means that hundreds of people work on the code, make improvements and add features. It’s organized of course, and the logic is that with so many people looking at the code, that it’s less prone to errors than commercial software that might have only a few people on a development team working under tight deadlines imposed on them by management or a financial situation.

What happened was that OpenSSL encryption was starting to run at an unacceptably slow pace, either because of the volume of transactions that were being processed or perhaps because of the complexity of them. A developer on the team was looking for ways to shave off a few microseconds here and there, and came across a feature called memory management that prevented memory leaks, especially during the transport layer security protocol’s heartbeat extension process. This was adding precious cycles to transactions, and was probably seen as unnecessary and redundant.

Thinking they had found a perfect way to speed up SSL without much effort, that function was disabled in version 1.0.1. I’m sure the entire team working on OpenSSL must have known this, but nobody considered the consequences of that action, which allowed the heartbeat process to bleed a little bit of information in 64 kilobyte chunks from both the server and the client. And it did speed things up.

Eventually in version 1.01f, the problem was fixed, but the slow nature of security upgrades means most sites were running the older version. Ironically, versions prior to 1.0.1 are also safe. So, if a site was really slow or really fast to upgrade, it was safe. Most fell into that middle ground in the danger zone.

By some saving grace, nobody in the hacker community apparently realized the heartbeat was bleeding information. Two years went by before researchers discovered the flaw, which is why the problem is so widespread today. Those computer scientists went public with their information, telling everyone they could about it, demonstrated how the hacking could take place and launched the Heartbleed website to explain the problem and how to fix it. They even came up with the scary bleeding heart logo,which has come to symbolize the flaw.

Had those researchers been hackers themselves, they would have had the keys to the kingdom. Imagine having the power of an unknown flaw in the palm of your hands, one that could be used to gain access to the majority of the Internet including government sites, banks, online stores, corporate email systems and almost everything else. They could have done literally anything they wanted online, gained any secret or stolen huge amounts of money, and nobody would have known. I can’t imagine it wasn’t temping. Instead, they went public and are helping to fix the problem. For that, I seriously believe they deserve a medal. A Nobel Prize wouldn’t be out of the question in my humble opinion.

Despite the fact that Heartbleed attacks are mostly untraceable, administrators can detect them now that they know what to look for in the log files. The exposed memory is bled out in 64 kilobyte chunks, so it takes a large number of repeated logins to gather anything useful, like the primary encryption key. That could be detected and scanned for, and there are tools now to make this possible. But the easiest way to fix the problem is simply to upgrade to the lasted version of OpenSSL, which has memory management turned back on, and then force all users to change their passwords.

Once the immediate threat is over, I’m sure there will be a debate on open-source versus commercial software. But the fact is that bugs can crop up in any medium. I don’t think we need a monster to slay. Our technology is invented by humans, and we are all wonderfully and tragically flawed creatures, so our creations are as well. Unlike a movie or an after school special, there may not be an overarching lesson here. We got lucky on this one. And perhaps that’s enough.