Mark Henke has spent over 10 years architecting systems that talk to other systems, doing DevOps before it was cool, and matching software to its business function. Every developer is a leader of something on their team, and he wants to help them see that.
Continuous deployment may sound like a magical, mystical thing. No manual deployments? Production code with no pain? Sign me up! But then the reality hits you: what tools do I use? What if they’re hard to learn? Will they be too expensive? What if they don’t work together? Which ones are the best for me?
Let’s take a look at some of the best tools out there. I want to give you a fighting chance to choose the right ones for your team.
Let me be clear: there is no one best tool, but there are tools that are the best fit for certain continuous deployment environments. For each tool I mention, I’ll flesh out what those situations are and why that tool is so well suited for them.
A Brief Overview of Continuous Deployment Tools
I will assume you’re familiar with the concepts of continuous integration, continuous delivery, and continuous deployment. If not, I strongly recommend reading about them first. And then spend the night dreaming about them. Let it all sink in deep. Mmm…that felt good, didn’t it? Excited yet? Good!
The key to using any of these tools effectively is to have a DevOps mindset. Oftentimes, teams treat the title “DevOps” as a specialist role, like “database administrator”. But it’s not a separate thing you do on a team. DevOps means taking full ownership of a feature from code commit all the way to the users’ hands. This includes testing, getting dependencies in place, and deploying all the way to production.
Here’s a brief overview of a general continuous deployment pipeline:
The infrastructure setup stage is what many people are referring to when they say DevOps. Again, I want to emphasize that DevOps is a mindset, not a selection of tools.
We’ll break down the must-have tools in each of these sections. And for each tool, we’ll include a brief description, an explanation as to why you would use it, and the reasons why you wouldn’t use it.
Build servers are also known as continuous integration tools since their focus is, well, continuous integration. There are more of these than you can shake a stick at. This will probably be the most overwhelming section we cover today, just because there are so many options. Fortunately, most tools have a fairly standard set of features, and almost any one of them will get you up and running.
TeamCity has been around the block. It’s used by a myriad of OSS projects. TeamCity offers both hosted and on-premise solutions. It’s a reasonably priced, reliable solution to running your builds. If you’re not sure how to use it, don’t worry—you get a free trial to try it out. And it supports many different platforms, .NET and Java included.
Compared to most tools, I find it easy to get builds up and running with TeamCity. It doesn’t require files to be stored in your source control, unlike something like Travis CI. And it’s owned by one of my favorite companies, JetBrains. These guys make good, intuitive tools.
When you should use it: If you have a budget and you want to get something up and running quickly, TeamCity is great—but only when you have a fairly standard build process that requires little custom scripting.
When you shouldn’t use it: If you’re not running an open-source project and you don’t have a budget for your server, Travis CI or Jenkins may be a better bet.
Circle CI: This tool is very popular with the Ruby crowd.
Jenkins: This is the preferred on-premise solution for many enterprise shops. It has a lot of plugins.
Deployment servers are less common. In fact, at many shops, I’ve seen them try to shoehorn CI servers into this role. However, a good continuous deployment server has very different needs. Environments need to be first-class citizens, for example. There should be a clear dashboard of what versions are in what environments.
If you are using .NET, this tool is hands-down the queen of deployment servers. It’s one of the most intuitive pieces of software I’ve ever used. Getting Octopus up and running is fast and easy. Their server makes it simple to have consistent deployments across environments. There’s support for multiple languages, but it shines with .NET. Plus, it supports Docker and a myriad of cloud providers. You can put the server on-premise, and they recently released a cloud version.
Octopus also supports storing your configurations in a way that is trackable over time. It hides sensitive data and is easy to maintain.
When you should use it: If you’re using .NET in any capacity, this is hands down the best tool for you. Even in non-.NET environments, it’s hard for me to see reasons to choose another tool.
When you shouldn’t use it: If you’re in an environment where you can’t provision a Windows box and you can only use on-premise technologies and you have no budget, only then would I consider the open-source alternatives below.
Concourse and Go: Both of these tools are similar in that they delegate most of the heavy lifting to scripts you provide and they coordinate those scripts across environments. They’re both open-source. As a result, they’re harder to use. Still, it’s worth a look.
You can get far by just using the tools described above. However, as your needs scale, you’ll want a way to handle servers and prerequisites that scales with you. You may also see these tools under the configuration management heading.
Docker is a hybrid of a VM and an application. For teams that have to deal with dozens or even hundreds of servers, it fills a much-needed gap of getting these servers to behave consistently. As applications scale, they commonly build up this list of needed settings. This could include stored certificate keys, proxy configs, and script execution policies. Applications also tend to curate a growing list of prerequisite software. They could require embedded caches, command-line runners, or a specific framework version. The more servers you have, the more painful this gets. Docker can let you code and template these prerequisites. Then you can spin up the template as a set of containers on a server, isolated from each other and consistent. These containers then make for a great place to drop in your application.
Docker works by destroying and recreating containers as necessary, as opposed to changing their configuration over time. It also has cross-platform support.
When you should use it: If you have to maintain dozens or more stateless servers in your environments, you should consider Docker. It’s also a good choice if your app has many little dependencies that keep it running.
When you shouldn’t use it: Don’t use this for your persistent stores, such as database servers unless you’re using it for testing.
Ansible: This tool is a leader in simple configuration management. It accomplishes similar things to Docker but in a different way. Ansible focuses on keeping server configurations consistent and up to date, as opposed to creating disposable containers that can be recreated as necessary. It may be a good choice if you have many persistence servers, like a database server.
Puppet or Chef: Like Ansible, these tools are older and focused on keeping persistent server configurations up to date. The main differences between these and Ansible lie in their plugin ecosystems and formats. They focus on creating a “desired state” for your server then diffing the current state to create a set of commands to make it happen.
Kubernetes: Kubernetes can be used alongside a tool like Docker when dealing with a large number of instances of similar containers. Think of it as an orchestrator for your highly-scaled environments; the Docker containers are the musicians.
A Little Bit of Everything
One fairly unique tool actually runs the gamut of both build and deployment—and does it well.
This little gem is built to handle the whole pipeline, from build to production deployment. And it does it all with one little YAML file! This makes it fast and convenient to get a deployment pipeline up and running with a new project. It also makes it easy to play with the different options you have at your disposal. Even better, the tool is mostly free for a variety of use cases, though your jobs may need to wait in a queue if you don’t pay. It supports projects up on GitHub, both private and public. GitLab also can host your codebase itself.
When you should use it: Check out GitLab when you want to play around with your first deployment pipeline. It makes it easy to get started with little to no cash. It’s also great to use when your continuous deployment pipeline is fairly standard.
When you shouldn’t use it: If you need to host something yourself or if you already have your own build or deployment server and just need to fill in the gaps, you’ll want to pass on GitLab.
Travis CI: This is essentially the on-premise version of GitLabs, using the same YAML format. It supports a hosted version that’s more limited than GitLab.
A Note on Databases
Many of us use relational databases. This means we need to keep our schemas up to date across all environments. The most common way to do this is with tools called migrators. You can run these migrations as a pre-deployment step in each environment so your database is ready for the app upgrade. Here’s one of my favorite tools:
Flyway by Boxfuse keeps database migrations as simple as possible by using templated SQL files. It runs the migrations through a console application, and it lets you use placeholder replacements for certain things that can evolve in different environments—for example, if you have an H2 database in dev but SQL server in production. It’s cross-platform and simple. There’s not much more that you could want.
Choose and Go
I’ve listed many tools here today. I hope this lets you pursue a continuous deployment pipeline that is simple and cheap and makes your team very happy.