Recent Posts

Eight months have passed I'm overdue for an update on my freelancing business. A lot has transpired since my last update in October so I'll get right to it.

It hasn't even been a full year yet since my last post on continuous deployment and I've already reimplemented a more robust system. This post covers the challenges I had with that previous iteration, then the design and implementation of the new deployment platform. I hope you'll find it useful.

User Data is that feature that allows you to configure new instances at boot using parameters or arbitrary scripts. It's an essential feature for building automated systems, often used to run a management daemon or to configure services.

Commonly, instances need access to secrets as part of the initialization process. A database connection string, keys to an API, or other sensitive data need to be distributed to the instance, and user data is a handy way to distribute those secrets.

But user data is not encrypted. It is exposed via the web console, in a file on disk, and to any process running on the instance via the instance metadata. The docs even warn against using it for anything sensitive. Some web app attacks have been known to seek access to user data, hoping to reveal secrets and access info.

Here's one method to secure those secrets and still preserve the usefulness of user data.

I signed up for access to ECS when it was announced at reInvent, but didn't get in to the preview until December 18. I spent the past week kicking back and watching Anki's data ingest pipeline handle the insane Christmas traffic, and now that we're stabilized I've had a chance to take ECS for a spin. Here are some notes and observations about my experience using it for the first time.

"Let's design this to be agnostic in case we want to switch cloud vendors later." -- Person focused on the wrong things