Introduction

One of the tools we are using on the operations side of things here at Gowalla is the configuration management tool Puppet. Many of you are likely already familiar with the project, but if not, I definitely recommend that you check it out over at Puppet Labs. It has matured quite a bit since its beginnings, and is now fairly easy to get up and running, especially with all of the open source modules being shared on PuppetForge and GitHub.

After using Puppet for a bit, you will likely come to know the tedious task of adding and removing node definitions. This is especially true if your infrastructure is hosted on a cloud provider such as EC2, as ours is. We bring down and spin up new instances fairly often, and having to manually add and remove these definitions becomes time-consuming and hampers automation.

Puppet ENC and EC2 Security Groups

This is where Puppet's External Node Classifier feature comes to the rescue. It allows you to delegate node classification, and therefore which classes are applied to each node, to an external script or application. As you can imagine, this provides all kinds of possibilities for integrating with other systems that may already know about your nodes.

But what if you don't already have this information in another system? This is where we found ourselves. After doing a bit of research, we stumbled across a presentation that suggested using EC2 security groups to define nodes. Bingo! This seemed like a great solution. We already had our nodes (mostly) separated into different security groups based on their purpose, as this allowed us to easily define firewall rules between our different types of instances. Why not re-use information that's already there?

The implementation was fairly straightforward. We hacked together a quick Ruby script (ec2_enc.rb) that would query the EC2 API with a node's hostname to determine its security group. With that information in hand, the script loads a YAML file (node_groups.yml) containing all of the security groups mapped to the classes and parameters to be applied, and passes the proper data back to the puppetmaster. After creating the YAML file with the required mappings, we were up and running.

It's a simple solution that has worked quite well for us so far. Of course there's always room for improvement—please feel free to use, modify and send us feedback at will!

Files

• ec2_enc.rb
• node_groups.yml