Gruntwork Newsletter, August, 2023

Published in

Gruntwork

9 min readAug 21, 2023

Every few months, we send out a newsletter to all Gruntwork customers that describes all the updates we’ve made since the last newsletter and news from the DevOps industry. Note that many of the links below go to private repos in the Gruntwork Infrastructure as Code Library and Reference Architecture that are only accessible to customers.

Hello Grunts,

A lot has happened over the last several months! The biggest news is that HashiCorp decided to adopt the Business Source License (BSL), a non open source license, for all of its products, and in response, we’ve joined forces with dozens of other companies to create OpenTF, a foundation to keep Terraform open source — forever. In other news, we released a new product called Gruntwork DevOps Foundations to help companies set up fast, secure, & reliable account foundations, code foundations, CI / CD foundations, and maintenance foundations with AWS & Terraform; added Promotion Workflows support to Patcher; created an official GitHub Action for Terragrunt; added an inspect command to Gruntwork Pipelines; published a blog post on using secrets securely on the CLI with 1Password; fixed an S3 ACL Bucket issue; started work on the AWS provider v5 and CIS 2.0 upgrades; and a whole bunch more. Read on for all the details.

As always, if you have any questions or need help, email us at support@gruntwork.io!

Gruntwork Updates

The future of Terraform must be open

On August 10, 2023, HashiCorp announced that after ~9 years of Terraform being open source under the MPL v2 license, they were suddenly switching it to a non open source BSL v1.1 license. We believe the BSL license is a poison pill for Terraform which threatens the entire community and ecosystem. As a result, we’ve joined forces with dozens of other companies to create OpenTF, a foundation to keep Terraform open source—forever.

As Gruntwork customer, here’s the short version of what this means for you:

As long as you use Terraform 1.5.5 or older, you can keep using all our commercial and open source products with no changes. Terraform 1.5.5 and all older versions are still MPL licensed, so you can keep using that version of Terraform with Terragrunt, Terratest, the IaC Library, the Reference Architecture, Gruntwork Pipelines, etc., with no changes of any kind.
For future versions of Terraform, Gruntwork will use open source Terraform. For versions of Terraform that come out after 1.5.5, we will switch all our commercial and open source products to work only with open source Terraform: that is, if HashiCorp chooses to switch Terraform back to an open source license, we will use that, and if they don’t, then we will use our open source fork from the OpenTF Foundation.

For the full details, read on: The future of Terraform must be open—our plan and pledge to keep Terraform open source. If you’ve got any questions or concerns, please email us at support@gruntwork.io.

Introducing: Gruntwork DevOps Foundations!

We’ve had something new in the works: a product focused specifically on setting up your basic foundations for DevOps. This blog post will not only tell you, but show you, through text and short videos, what it’s like to have fast, secure, & reliable account foundations, code foundations, CI / CD foundations, and maintenance foundations set up with AWS & Terraform.

This includes:

Account factory: spin up new AWS accounts with secure baselines, SSO, and Control Tower as a single pane of glass.
Infrastructure deploys: self-service, GitOps, everything managed as code.
CI / CD pipelines: approval workflows, promotion workflows, testing workflows, security-first pipelines, etc.
Maintenance: auto update, auto patch, etc.

To see it all in action, read on: DevOps Foundations—fast, secure, reliable AWS environments using Terraform. If you’re interested in getting DevOps Foundations like these set up at your own company, reach out to us!

Promotion workflows with Patcher

A couple of months ago, we announced the beta release of Gruntwork Patcher, a tool to automatically keep your infrastructure code up-to-date, even with breaking changes. We’re now happy to announce the beta release of Patcher promotion workflows. This gives you a way to automatically promote changes from environment to environment using a GitOps-driven, immutable infrastructure workflow: you can use it to roll out a new module version from dev to stage to prod, all automatically. And we’ve even released example workflows for GitHub Actions as open source!

For the full details, read on: Promotion Workflows with Terraform—How to configure GitOps-driven, immutable infrastructure workflows for Terraform using Gruntwork Patcher.

Official Terragrunt GitHub Action

More and more companies are using Terragrunt, and more and more companies are using GitHub Actions, so we decided to put the two together, and have released the official, open source GitHub Action for Terragrunt!

This makes it super easy to run Terragrunt commands in your CI / CD builds on GitHub Actions. For example, here’s how you can run terragrunt apply in the dev/eks-cluster folder of a repo:

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: 'Checkout'
        uses: actions/checkout@master

      - name: Deploy
        uses: gruntwork-io/terragrunt-action@v1
        with:
          tf_version: 1.5.5
          tg_version: v0.50.3
          tg_dir: dev/eks-cluster
          tg_command: 'apply'

Check out the Terragrunt GitHub Action in the GitHub Actions Marketplace and on GitHub and let us know what you think!

AWS Provider v5 and CIS v2.0

In the last few months, there have been two major updates released:

We have started the process of going through the IaC Library and updating all modules to be compatible with these new releases. We will post an update when we’re done. In the meantime, sit tight!

Gruntwork '`inspect'` command with support for Pipelines

We’ve added a new command to the Gruntwork command line interface (CLI) to make working with Pipelines easier.

If you run gruntwork inspect pipelines , we’ll search for any deployed Pipelines installations and show you the AWS resources they involve.

You can also delete your Pipelines installation from within the same interface to make cleaning up deployed Pipelines examples a snap.

To try it out, please install the gruntwork CLI at version v0.4.13 or later.

How to securely store secrets in 1Password CLI

We published a blog post on how to securely store secrets in 1Password CLI and load them into your ZSH shell when needed. This details a convenient and secure workflow for using secrets with command line tools such as Terraform, Packer, and the AWS CLI. Take a look and let us know how it works for you!

S3 ACL Bucket Issues resulting from AWS’s S3 security change

AWS rolled out a new policy for S3 in April 2023 to disable ACLs on bucket creation. The change was rolled out incrementally across AWS’ global infrastructure. This change affected modules that create S3 buckets, leading to errors similar to this one.

To resolve this issue, we patched all of our modules as well as our service catalog to be compatible with this change. If you are seeing ACL errors with S3, you’ll need to update your module versions.

For instructions, see https://github.com/orgs/gruntwork-io/discussions/701.

All other Gruntwork releases and updates

You can find details on every single release and update we do in the Gruntwork Releases page of our docs site. And now, you can use Patcher to update your dependencies automatically! If you’re a Gruntwork customer and don’t have access to Patcher yet, please email support@gruntwork.io. If you’re not a Gruntwork customer, please contact our sales team.

Here are the dedicated pages for new Gruntwork releases since the last newsletter:

DevOps News

HashiCorp adopts the Business Source License (BSL)

After more than a decade of being known as the open source company, HashiCorp announced that they will be adopting the Business Source License (BSL) for all of its core products, including Terraform, Consul, Vault, Nomad, Packer, Waypoint, and Boundary. The BLS is not an open source license, so this is a shocking move. In response, we’ve joined forces with dozens of other companies to create OpenTF, a foundation to keep Terraform open source — forever. For more info, see The future of Terraform must be open.

Terraform 1.5 improvements

Terraform 1.5 has been released with several powerful new features:

Config-driven import
Instead of running a bunch of terraform import commands, you can now add import blocks to your code:

import {
  # ID of the cloud resource
  id = "i-abcd1234"
 
  # Resource address
  to = aws_instance.example
}

Now, when you run plan, Terraformwill tell you what you’re about to import, and if you run apply, it’ll do the actual import.

Code generation on import
In addition to import blocks, you can now add the -generate-config-out flag to have Terraform generate code for whatever resources it is importing. That’ll create a generated.tf file:

# __generated__ by Terraform from "371e85d406a937b359d5cc3a49a423b736ec6e9367abe705dcdf6673ec1fb4c4"
resource "xxx" "yyy" {
  zzz  = "..."
}

Check blocks
This is a way to have your modules self-test themselves (“smoke tests”) at the end of an apply. If the checks fail, you get a warning in your logs, but the apply does not exit with an error. You can even embed a data source within a check block that will run at the end of apply and if it fails, again, you get a warning in the logs, but apply doesn’t fail:

check "health_check" {
  data "http" "example" {
    url = "https://${aws_lb.example.dns_name}"
  }
 
  assert {
    condition     = data.http.example.status_code == 200
    error_message = "${data.http.example.url} returned an unhealthy status code"
  }
}

AWS Identity Center (SSO) now supports automatic user provisioning from Google

AWS has announced that you can now connect Google Workspace to AWS IAM Identity Center (successor to AWS Single Sign-On) and it will sync users automatically. In the past, user syncing wasn’t supported, other than with external tools (e.g., a Lambda function that did the user syncing), so you didn’t really get Single Sign-On with Google. But now, they support user auto-provisioning via SAML.

New default S3 security settings

AWS has announced that they now apply two new security settings to all S3 buckets:

Public access will be blocked by default.
ACLs will be disabled by default.

This may cause some issues with Terraform modules; see our announcement earlier in this newsletter for how to resolve this.

AWS Provider v5

Version 5.0 of the Terraform AWS Provider has been released. You can find the details on what changed here. We’ll be updating the IaC Library to be compatible with this new version and will share another update when that is ready.

CIS Benchmark 2.0

Version 2.0 of the CIS AWS Foundations Benchmark has been released. You can find the details on what changed here. We’ll be updating our Gruntwork Compliance offering to meet the requirements of this new version and will share another update when that is ready.