For several years, I had been a customer of CodeSpaces, a popular host for Subversion source control repositories. (If you’re not a techie – Subversion is a server application used to manage program code, allowing collaboration between teams of programmers.) I used CodeSpaces to host the code repositories for many of my consulting clients, until…
A few weeks ago, I was trying to check in some code and I got a strange error message from Subversion. Then I tried going to the CodeSpaces website to log into my account, and I was met with a page titled “CodeSpaces Is Down!” The page explained that someone compromised their Amazon EC2 hosting account, threatened to destroy their business if they didn’t pay a large sum of money, then deleted many of their virtual servers and backups when they didn’t comply. Because the damage to their business was so great (both financially and to their brand reputation), CodeSpaces decided to cease operations entirely.
I contacted CodeSpaces support, and thankfully they were able to provide me with backups of my customers’ Subversion repositories. Then I spent 2 days setting up my own Subversion server (with VisualSVN Server), importing all of the backups into SVN, setting up a Git server and migrating from SVN to Git.
Lesson #1 – Always Use Multi-Factor Authentication
If you have the option, always take advantage of multi-factor authentication, sometimes called “two-factor authentication” or “two-step verification.” Many websites and service providers offer this option. Basically, in addition to using your password, multi-factor authentication normally uses a physical device (typically a phone) to verify that you are really the account owner. While typing your password on a login page, you must also enter a verification code. This single-use code is retrieved via text message to your phone, or using an app on your Android/iOS device. Some services even support a physical device that plugs into your computer’s USB port. But most often, websites use Google Authenticator or Authy for verification.
Amazon EC2 offers multi-factor authentication. If CodeSpaces had used it, it is highly unlikely that the hacker could have compromised their account.
I personally use multi-factor authentication on a number of services, but I’ve been lazy about it myself. I plan to add it to more and more services when possible. There is a great list of websites that offer two factor authentication. I’ll be going through it and adding them to my Authy app.
Lesson #2 – Use a Unique Password for Each Website
People have recommended this for years, but nobody really does it. You should never use the same password for more than one website. You’re taking an awful risk if you’re using your banking website’s password for dozens of other sites.
Lately I’ve been migrating all of my passwords into 1Password. If you haven’t heard of it, 1Password is a fantastic security manager for desktops and mobile devices. It securely stores and syncs all of your passwords across all of your devices (if you so choose). It plugs into all of your web browsers to generate secure passwords, and it can automatically fill them in when you need to log into a website.
This means you can have a unique complex password for every website, but you only need to remember a single master password (“one password”). If one of your accounts gets compromised, hackers won’t be able to get into any of your other accounts. This is extremely important for any website that has access to your financial info (bank accounts, credit cards).
Lesson #3 – Keep Backups in Multiple Places
CodeSpaces had regular backups of their Amazon virtual machines, but they were doing all backups to Amazon S3. In other words, once the hacker had access to their Amazon web services account, they were free to delete the backups along with the virtual servers. Had CodeSpaces backed up to a different third party service, or even done regular FTP downloads to their local office, they could have recovered from the disaster.
I personally use Windows Backup to regularly image my desktop to an external hard drive. I also use Carbonite to back up all of my important files in the cloud. I also back up all of my customers’ source code and database images on third party servers. When it comes to backups, it’s good to be “redundantly redundant.”
Lesson #4 – Git is “Safer” than Subversion
The software development world has been moving toward Git source control in recent years, and I have been lagging behind. I first tried Git a few years ago and I found the tools for Windows and Visual Studio to be lacking. I missed having a simple GUI interface. I also found that the “distributed repository” model encouraged developers to do a lot of work on their own machines before checking their code into the central repository. This led to a lot of headaches with developers breaking each others’ code because they were sharing their work too infrequently.
I kept my opinion that “Subversion is better / easier” right up until CodeSpaces went out of business. A major drawback to Subversion and other “central repository” systems is that the server has the only complete copy. If the server breaks, or your Subversion host goes out of business, you’re out of luck. With Git, every computer has a complete copy of the entire repository. If the server breaks, or your Git host goes out of business, you just connect to a new server and push the repository.
I have found in recent months that the Git tools for Windows and Visual Studio have improved. There is now a TortoiseGit tool that is very similar to TortoiseSVN. There is also a decent Git Source Control Provider for Visual Studio, although it requires you to install 3 other tools first (msysGit, Git Extensions and TortoiseGit).
I also discovered that there is a very easy-to-install Git server (Bonobo Git Server) that works on Microsoft web servers. It works like a normal website inside IIS, so you don’t need to install Apache or any other Linux-derived software. I believe it even works on shared Windows hosts as long as it has the right privileges.
In Summary
I suggest that everyone should:
- Use multi-factor authentication whenever possible.
- Use distinct secure passwords for every website, especially those with financial info.
- Use a password manager to keep track of your secure passwords.
- Keep backups of important files in multiple places, including in your home/office and with cloud-based backup services. A single backup location is not enough.
- If you’re a software developer, move to a distributed repository system like Git.
People believe that “it could never happen to me…” right up until it happens to them. Learn from other people’s mistakes. It’s better to be safe than sorry.
Technimusings 