Gruntwork Infrastructure Packages
World-class infrastructure, all of the control, none of the work.
Let’s say you just started a new project — maybe it’s the infrastructure for your new startup or maybe it’s a new, more scalable microservice architecture at your existing company — and after weeks of coding, it comes time to launch. That’s when you realize how painful software can be.
There are just. So. Many. Moving. Parts. You need to deploy app servers, databases, caches, and load balancers; you have to configure routing, health checks, and CDNs; you have to fiddle with your build system, write automated tests, fight with dependencies, and set up a continuous integration server; you need to write automated deployment scripts and hook up monitoring and alerting; you have to figure out how you’re going to authenticate users, store secrets and credentials, and how you’ll manage secure SSH access; you need to set up a staging and production environment, manage log rotation and aggregation, install an SSL certificate, buy a domain name, figure out an automated backup and restore policy for your databases, write a system to build, minify, concatenate, and fingerprint your static assets, set up database replication and failover, protect yourself against spam, phishing, cross-site scripting, and denial of service attacks, all while keeping up with the security updates and ensuring that everything you build is scalable, reliable, maintainable, and fault tolerant.
Or, you just do what most developers do, and you skip all of this, and build a bunch of hacky, kludgey, half-assed solutions that are held together with duct tape and bubble gum, and then drown in tech debt for the next 10 years.
It shouldn’t be this hard. In fact, no one of those tasks is particularly difficult. Launching a satellite that uses earth’s gravity as a slingshot to enter Jupiter’s orbit is a hard problem. Running a few servers in the cloud shouldn’t be. Just about any developer can become proficient in any of the long list of tasks above in a week or two. But when there are ~100 such tasks, 100 times a week or two is an astonishingly large number.
This is the problem Josh and I set out to solve when we launched Gruntwork. Our mission is to make it an order of magnitude easier to understand, develop, and deploy software. The first step in that direction is to take the thousands of hours we’ve spent building infrastructure and to condense all that experience and code into pre-assembled Infrastructure Packages. Each Infrastructure Package is a best-practices definition of a single piece of infrastructure (e.g. a Docker cluster or a relational database) that is reusable, battle-tested, documented, configurable, and supported.
We think it’s a better way to build and manage infrastructure. And today, we’re unveiling the full list of Gruntwork Infrastructure Packages and releasing the documentation as open source so previous customers can find info from a Google search and future customers can explore our packages directly.
Gruntwork Infrastructure Packages
Here are a few of our most popular packages:
- Network Topology: Set up a best practices network topology in your AWS account, including isolated VPCs, isolated private and public subnets, network ACLs, security groups, NAT gateways, and VPC peering connections.
- Monitoring and Alerting: Set up monitoring and alerting using AWS CloudWatch, including custom metrics, Route 53 health checks, a default set of alerts (e.g. for CPU usage, disk space, 5xx errors), and log aggregation.
- Docker Cluster: Set up an AWS EC2 Container Service (ECS) Cluster to run your Docker containers, including zero-downtime, rolling deployment, load balancing, health checks, service, auto scaling in response to traffic, and automatic recovery of failed nodes.
- Continuous Delivery: Set up an automated pipeline to build, run tests, package your code, and automatically deploy to production using a CI server such as Jenkins, CircleCI, or Travis.
- Relational Database: Set up Amazon’s Relational Database Service (RDS) to run MySQL, Postgres, MariaDB, or Aurora, including replicas, automatic failover, and automatic backup.
- And many others: Check out the list of Gruntwork Infrastructure Packages for all the details.
When I worked at LinkedIn, we had to solve all the problems above from scratch. When Josh was building the infrastructure for his previous company, Omedix, he had to solve these problems from scratch. Every developer I interviewed while writing Hello, Startup, including early employees at Google, Facebook, Twitter, GitHub, Stripe, Instagram, Pinterest, and many other successful startups, also reported going through the pain of solving these problems from scratch. There is no reason every company should have to reinvent the wheel—and in the case of many companies, to reinvent the wheel poorly because they do not have the time, budget, or expertise to do it well. At Gruntwork, our aim is to solve these problems once, to do it well, and to make the solutions available to everyone else in the form of reusable Infrastructure Packages.
In some ways, we are trying to do for DevOps and infrastructure what Ruby on Rails did for web development. Instead of building your own HTTP server, attaching a templating system, coming up with a custom routing system, writing all the database code from scratch, rolling your own configuration system, wedging in an asset pipeline, and so on, Rails lets you leverage a coherent set of tools that have already been assembled for you and are well documented, tested, proven in production, and represent thousands of hours of work — a far better package than any one company could do on it’s own.
Gruntwork Infrastructure Packages are similar. We’ve put in a huge amount of time and work into those Packages — far more than most companies could afford to do on their own. And as we keep working with more customers, we’re only going to make those Packages better. Of course, our Packages won’t be a perfect fit for every company, just as Ruby on Rails isn’t a perfect fit for every project. But for the ~80% of common use cases, we believe that our Packages can save you a huge amount of time and money, and allow you to start your project on top of a robust infrastructure instead of resorting to duct tape and glue because you lack the time to do it right.
Why not PaaS?
At this point, you might be wondering how are Gruntwork Infrastructure Packages different than Platform as a Service (PaaS) offerings such as Heroku or Docker Cloud? PaaS providers take care of many of the same concerns and can be a great way to launch a project quickly. But if the project starts growing, you may run into problems. The inherent nature of PaaS providers is that they hide all the underlying details from you, which is great at first, but it makes it much harder to customize things, to scale your product, and to keep costs from spiraling out of control. Eventually, you may hit the problem that you just cannot debug because the PaaS provider doesn’t give you the visibility and control you need, and then your only option is to migrate to something else.
Josh and I are both developers and we’ve designed the Gruntwork Infrastructure Packages in the way a developer would want: we give you the code. You can read it, debug it, customize it, and just about anything else you want. Or, you can just ignore it, assume it works, and take advantage of the fact that it solves all the common problems for you, a bit like a PaaS. We even offer a subscription where you get access to all the updates, fixes, security patches, and new features we develop for the Gruntwork Infrastructure Packages to keep your maintenance costs low. But there is no lock-in. If you cancel your subscription, or even if Gruntwork goes out of business, the Packages are still yours, the code is in your codebase, and your infrastructure will continue working.
The power of infrastructure-as-code
This focus on code, rather than a SaaS or PaaS platform, is one of the key advantages of Gruntwork Infrastructure Packages. Instead of manually clicking around to deploy and configure software, we define infrastructure as code. All the details of how to provision servers and load balancers, how to build the code, how to run tests, how to backup and restore data, how to do automated deployment, and so on, are defined in code. That means you can put your infrastructure in version control; you can code review every change; you can see a full revision history of all changes that have ever happened; you can roll back to any previous version; you can use the code as documentation of how your infrastructure works instead of finding out that this information is locked in the head of a developer who left the company months ago; you can write automated tests for your infrastructure; you can create reusable pieces of infrastructure and even launch a replica of a production environment in minutes.
The main infrastructure-as-code tools we use are Terraform, Docker, Packer, and a collection of shell scripts and cross-platform binaries written in Go. Currently, our Infrastructure Packages are designed for use with Amazon Web Services (AWS), and we can get most customers up and running on AWS with DevOps best practices and world-class infrastructure in about 2 weeks. We will add support for other cloud providers in the future.
For more info or to schedule a chat with us, check out gruntwork.io and the full list of Gruntwork Infrastructure Packages. And if you want to follow along with the latest news on DevOps, AWS, software infrastructure, and Gruntwork itself, subscribe to the Gruntwork Newsletter.
