Generate Secure Pillar

Anyone using Salt Stack should be familiar with using secure pillar files. If not, here is a basic intro I created for setting it up.

Using the secure pillar can prove a challenge for folks who aren't familiar with GnuPG or PGP in general. This is a shame, as many times secrets that should never be stored in a repository end up being placed in plain text, where any bad actors can find them. Using secure pillar makes keeping secrets (API keys, passwords, SSH keys, etc) much more secure. Only people with access to the private PGP key created for your Salt Master can decrypt the content.

Much of the problem with people not wanting to use secure pillar files comes down to two things:

1) Setting up secure pillar can be confusing, as the docs don't have a basic example. The link above should help there.

2) Using GnuPG can be a pain in the ass if you aren't familiar with cryptographic nomenclature.

3) Even if you can use GnuPG, you need to make sure your pillar files are formatted correctly. A misplaced space can be hard to notice and the slightest typo can cause no end of problems.

For the past year and half I have been working on a large project, which needed to have secure pillar configured. I did most of the work on that and in doing so I realized that working with secure pillar files kinda sucks if you just want to add or update a secret quickly. And if you have to rotate your secret keys? A giant pain in the ass. I once worked for a company whose salt master PGP key expired, breaking everything. It took a long time to rotate out the key, decrypting all the files and re-encrypting them with the new key.

Seeing that we would need a tool to make working with secure pillar easier I set out in my spare time to create one.

This also gave me a good excuse to learn Go. I have used many languages over the years, but I started as a C programmer. Part of me misses it. (Part of me is, apparently, a masochist.)

I liked the familiar syntax of Go, but I also liked how the language is very strict about things like unused variables, etc. I once wrote some C using GCC on a linux box, then tried to port that code to an SGI machine. The SGI compiler barfed all over it, as it was stricter. I came to love that, as it made my code cleaner and more correct. Sloppy code can be a nightmare for whoever comes along and has to work with your code. That someone might be you, so be nice to future you!

The end result of this side project was a Go command line tool, called generate-secure-pillar. It has all the features I could imaging needing, and as I was developing it I was also using it. This made me focus on the right features to make things as easy as I could. I freely admit that I am very lazy, so being easy to use was a large requirement. When we needed to rotate the PGP key and convert all the files to use the new key I spent some time and added the ability to rotate the key used for all files with one command. When the time it took annoyed me (about a minute) I made it concurrent (now about 6 seconds). Any pain point I found I tried to kill.

When I was happy with it, I talked to my boss and he agreed it would be a good thing to make open source. I know a lot of folks that could have used a tool like this at other companies (I was one of them). I'm glad to say that this was the first official open source project at my current employer, Everbridge, Inc. It feels good to give something back to a community that has given me so much over the years.

I had a great time writing it and I hope folks find it as useful as I have. Pull requests are always welcome!

--Ed Silva