Security Training with a Capture the Flag Exercise in a Virtual Machine
Security always has been an important part of software development, but in recent years has become an ever more vital topic. Software infrastructure is increasing in complexity and interdependence, and so is the business of keeping that infrastructure secure. Unfortunately, security is also sadly neglected in our industry. The average developer doesn't have a good grasp of all of the basics, if by basics we mean the OWASP list of cardinal sins. Consider that it has taken a lot of time, effort, advocacy, and very public breaches to get to the point at which most developers in the industry understand that SQL injection is a risk, and go on to both (a) incorporate that understanding into their work and (b) informally educate their up and coming peers. Yet there are a good dozen other classes of vulnerability in modern web applications alone that are just as bad in practice, and of which most developers are at best only academically aware. Thus vulnerable applications keep emerging, and at a much greater rate than has to be the case.
The current state of the field means that in practice, for near all software engineering organizations, any effective form of security training for developers adds considerable value over time. Given that good knowledge of security practices in development, just like devops, is both a complex topic and pretty thin on the ground in the engineering community, you can't expect it to spread via cultural osmosis. Training is an important part of getting to the point at which knowledge and good practice becomes more self-sustaining within an organization.
Most Security Training is not Particularly Effective
Unfortunately, most security training isn't all that good when it comes to achieving practical outcomes. You'll see organizations putting bad slideware presentations in front of employees as a part of their regulatory compliance, checking that box, and never following up. Unfortunately, even good static explanations of security issues and best practices tend to do little to help: however interesting it might be to read at the time, there is a big difference between reading up on a topic and actually putting that information into practice. Developers, like everyone else, internalize and assimilate new knowledge by working with it in a practical context, not by reading about it - either in the abstract, or specific examples.
Capture the Flag Exercises as a Basis for Practical Training
A number of companies have run and open sourced the code for public capture the flag (CTF) competitive exercises, as recruiting, publicity, and training exercises. A series of flawed web applications are deployed, and contestants must break into each in turn to find the keys to the next level. The goal is worth through the chain of ever harder challenges to gain the final prize in the final level. Of these the second Stripe CTF from 2012 is one of the most accessible, and its challenges are still very relevant today. When carried out as a group exercise, running through the first few levels of a CTF contest in a friendly in-house competition is a very good way to introduce a level of practical experience with vulnerabilities to a development organization. Get a few dozen people into the same room for a few hours, form small teams, offer modest prizes for the best contributions, and walk through each level and its vulnerability after it has been solved. Short of identifying and fixing vulnerabilities in the wild, this is perhaps the best form of training in this portion of the software engineering space.
Packaging the Stripe CTF 2.0 into a Virtual Machine
While the code for the Stripe CTF 2.0 challenges is published, the original deployment infrastructure is not. There are a few partial how-to documents out there, but no complete recipe. For the purposes of an internal training exercise a full deployment isn't necessary, however. All of the various servers and web applications can be packaged into a single virtual machine image that developers fire up and run locally. The only fiddly part of the process is restricting developer access inside the VM sufficiently well to run a training session as a friendly competition - ensuring that it isn't trivial to cheat by breaking the VM. This is, obviously, a pretty low bar for the existing Stripe CTF 2.0, since answers and analyses are out there if you want to find them, but will be useful when constructing your own flawed web applications and challenge levels.
The requirements for the VM:
- Run all of the challenges, but only when unlocked with the key for that level.
- When initially created, randomly generate the keys for each challenge level.
- Provide an administrative user with full sudo access for troubleshooting.
- Provide a participant user that can only run and unlock challenge levels.
- Lock down access so that the participant user cannot obviously cheat.
An outline for management of permissions in the VM is much as follows:
- The participant logs in as the
ctfuser with very limited permissions.
- A series of scripts such as
ctf-unlockemploy limited sudo rights granted to the
ctfuser in order to run and unlock the challenge applications.
- Application code is stored in directories under
/var/ctf, inaccessible to the
- Applications run as
rootwhere it is believed that they cannot be used to break the challenge due to their inherent flaws. This is all the more incentive for people to find other issues.
- Those applications that must have their file system privileges restricted, in order to limit their vulnerability to that level only, run under separate users such as
ctf-2. Each of these users can only access its own application directory.
When I first set up a CTF VM as a training exercise, the security group at that organization offered a small prize to anyone who could break the intent of the VM - claiming keys without solving the challenge. That was awarded a couple of times. It is very easy to overlook small things, and permissions management in Linux becomes very complex very quickly for anything other than the simplest use cases, especially when sudo is involved.
Another item worth noting is that the published CTF code is actually broken as written in a couple of places. It probably doesn't represent the final version that was used in 2012. Building the VM required a few fixes, and some of the more complex code was tidied up a little in other ways as well. That might be overly generous on my part, given that reading badly written code is an important skill, and one likely to be practiced frequently by anyone who specializes in security fixes. It is, however, hard to let poorly written code go by.
The Stripe CTF 2.0 VM at GitHub
A Veewee Virtualbox VM builder for the Stripe CTF 2.0 is open sourced at GitHub. It should provide a good basis for efforts to build CTF training exercises, and it is certainly possible to swap in different web applications within the overall structure, more suited to the work of a given organization.