We muse about a lot of potentially nefarious scenarios at 4ARMED, after all our job is to understand and anticipate IT security threats and help our clients protect themselves.
One of the things I was musing about a while back was more sophisticated ransomware, something that didn’t just encrypt everything on your hard drive but something a bit more, well, sophisticated.
Being a Git to developers
The primary defence against ransomware is regular backups. Yes, yes, I know people shouldn’t click on things too but we’ve got less control over that and, as Wannacry has recently shown, it doesn’t always start with an attachment.
4ARMED’s primary focus is application security so I was considering the risk from ransomware targeting software companies and their source code. In the vast majority of cases our clients are hosting their code on GitHub or Bitbucket. Some use Gitlab or in Microsoft shops they use Team Foundation Server but the point is, in many cases there is already a kind of backup offsite. You get ransomware on your dev machine, yes, you might lose your unpushed changes but the loss is probably not massively business impacting - rebuild your machine and go checkout the code and carry on.
The problem is, these source repositories are easy to identify, there is local configuration that an attacker could check for. Credentials are often stored locally or, in the case of Git, often relies on SSH keys that are currently loaded into a running agent. An attacker running code in the context of your user account can read these very easily. More sophisticated malware could read this configuration, send a copy of the source code to the attacker and then delete the copy in the repo too.
Purely Theoretical?
Unfortunately it looks like it’s not just theoretical. The Register reported recently that a US software development company, (ironically?) named Panic, was breached and had their source code held to ransom.
Now this wasn’t the full on, automated ransomware of my musing but it’s close enough. So the big question is, what would you do if it happened to you? As we’re all about preparation at 4ARMED, here are a few steps you can take to prepare your organisation against this kind of attack.
Caveat: These steps are very Github biased. I’ve tried to point out equivalent functionality on Bitbucket where I can and if someone knows different please let me know, but generally I was a bit disappointed with Bitbucket’s capabilities to help with this.
1. Backup your Git repository
Gee, no prizes for guessing this one. Github Enterprise has some backup features but, even without this it doesn’t take much to write a short backup job that will clone the repo, encrypt it and store it somewhere safe.
There are a few utilities out there which can help you clone or backup a repo, just make sure you add encryption to whichever one you choose. Putting it somewhere like Amazon S3 is definitely a decent idea. (Make sure you configure the bucket policy appropriately - but this is why the encryption is a must, just in case an error is made).
You could of course use a different cloud provider. Anything will do to be honest, just make sure all that’s being synced is a big encrypted blob.
Make sure you don’t unnecessarily expose credentials to the repo through this backup process. Create a read-only user dedicated to the purpose.
2. Set an SSH key passphrase
It may sound obvious but it’s amazing how often SSH keys are created without a passphrase. At least if you’ve put a passphrase on it, provided you haven’t got that key loaded into your SSH agent, the attacker is going to have to try a lot harder to access your repo. By SSH at least.
Wondering if your key has a passphrase? You may have remembered it in your Keychain and now you’re not sure. Have a look at the private key file and look for something similar to this at the top:
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
3. Enforce Two-Factor Authentication on Github/Bitbucket
If an attacker compromises your machine it’s fair to assume they can capture keystrokes. This means there’s a good chance they can grab your Github/Bitbucket password as you type it. Using a password manager that pastes it? That’s great (genuinely) however, an attacker with control of code execution in your user context can subvert a password manager in all sorts of ways. This is the big downside to a password manager.
Two-factor authentication gives you a fighting chance of a local compromise of your machine not resulting in a compromise of remote services such as Github. Assuming that your second factor of auth is not on the compromised machine so don’t go installing the Authy Chrome Extension - just about the worst idea I’ve seen when it comes to this kind of thing - or anything else that does TOTP on the same machine. The idea is to keep the second factor separate to the first. Use your phone or tablet.
4. Prevent Organisation members from deleting a repository
By default members of your Github organisation with admin privileges can delete repositories. Depending on your specific requirements there’s a very good chance this is unnecessary. If you turn this off the ransomware (or attackers in general) will need to compromise the organisation owner to delete the repo reducing the likelihood.
You can change this setting on Github at https://github.com/organizations/[:organisation]/settings/member_privileges under Repository delete.
This is important as, even if an attacker doesn’t manage to get access to your Github account they may grab a personal API token from somewhere. For example, if you use Github’s command line tool Hub it will generate a personal access token for you and store it in plaintext in ~/.config/hub.
Fortunately, this is generated with repo access by default so can’t be used to delete repositories but, nevertheless, hopefully you get the idea.
What else is lying around your machine that would be very useful to an attacker? I’m betting ~/.aws/credentials is pretty interesting. :-)
5. Audit your Git repo
Try to detect trouble as early as possible.
Github has a security log at https://github.com/settings/security or, if you are looking at an organisation, there is a comprehensive audit log at https://github.com/organizations/[org name]/settings/audit-log.
Bitbucket also has an audit log at https://bitbucket.org/account/user/[user]/auditlog/.
Curiously, from my relatively cursory review I couldn’t see a way to access the audit log via the API on either Github or Bitbucket which kinda sucks. The closest I can see you could get is to write something like a Mechanize script that logs in to your account and does a JSON (or CSV) export of the activity log on Github. Bitbucket doesn’t obviously support an export function.
It might be worth it, we love automation, however, the big problem there is you have to provide a username/password to the script and this is fraught with danger. I’d prefer a proper API integration but it might be worth it.
There’s a tonne of options but I was thinking of a Lambda job to export the audit log into Sumologic and then have an alert set up to tell me if there’s anything I need to know about. You could just as easily use any number of other combinations of modern tech, such is the beauty of the IT industry these days.
Azure Function -> PagerDuty
Google Cloud Platform Function -> Logz.io alert
The list goes on…..
Bonus 6. Third parties
It was meant to be five, it could easily be ten or more but here’s a bonus number six for you to consider. Don’t forget about third parties. You can do all of the above but if you’ve got a third-party developer, agency or any other organisation with access to your Github/Bitbucket repo you have to make sure they are playing by the same rules or they become the weakest link.
We do a lot of PCI DSS work and one of the things that comes up a lot is e-commerce sites hosted by third party web development agencies getting breached and companies getting into legal debates over who was responsible. Unless you’ve spelt out very clearly what you expect of a third party - and preferably asked them to sign a contract to that end - they are not likely to comply with your security requirements. Getting owned by ransomware sucks. Getting owned by ransomware through a third party sucks even harder.
Summary
This list is not designed to be definitive, it’s designed to get you thinking. Think of something that has value in your organisation, in this case your source code. Think of ways it could be taken from you then think of ways you could get it back. For each way you think of to get it back, think of a way that might not work then think of way to make it work.
It snowballs quickly but welcome to threat modelling.
The key message is, we can’t stop every attack. At some point we have to consider how to react to a breach. That is what separates the companies that recover from the companies that don’t.
If you would like some help getting your company ready to defend against IT threats (I think the cool kids call this cyber now) contact us today and let us help you be forearmed.
