Gruntwork Newsletter, December, 2021

Once a month, we send out a newsletter to all Gruntwork customers that describes all the updates we’ve made in the last month, news in the DevOps industry, and important security updates. Note that many of the links below go to private repos in the Gruntwork Infrastructure as Code Library and Reference Architecture that are only accessible to customers.

Hello Grunts,

In the last month, we added new features to Terragrunt which allowed us to minimize code duplication in the Reference Architecture, reducing its size by nearly 50%! We also added a new module for AWS Backup, launched a new knowledge base as a centralized way to capture questions, feedback, and all other types of discussions, and made many other fixes and improvements. On the security side, the entire software industry was hit with a severe vulnerability in log4j, so if you run anything on the JVM, make sure to patch this vulnerability ASAP. Finally, a reminder that we’ll be taking a holiday break for two weeks in December, so please plan accordingly!

As always, if you have any questions or need help, email us at support@gruntwork.io!

• Gruntwork Updates
• DevOps News
• Security updates

Gruntwork Updates

DRY Terragrunt —reduce the amount of code in your Reference Architecture by nearly 50%!

Motivation: Last month we announced new Terragrunt features that allow you to further DRY your infrastructure projects. Although the features were available in Terragrunt, we haven’t updated the Reference Architecture and example code to leverage the feature to provide a real world use case of the new features.

Solution: This month we are happy to announce that we have updated the Reference Architecture to leverage multiple includes! This greatly reduced the footprint of the Reference Architecture code base, making it much easier to maintain and replicate across new accounts and environments being added to the Reference Architecture.

Customers who purchase the Reference Architecture starting now will automatically receive the updated version. Existing customers who received an older version can follow the step by step guide we published to incrementally refactor each component in your Reference Architecture.

What to do about it: Download the latest terragrunt version and follow the guide to update your Reference Architecture to be more DRY!

[NEW MODULES] AWS Backup

Motivation: AWS Backup is a fully-managed, automated and configurable backup service designed to work with your existing AWS resources. It allows you to backup RDS and Aurora databases, EBS volumes, Dynamo DB tables, EFS, EC2 instances as AMIs and, recently, S3 buckets, to name just a few of the supported services.

Solution: We created two new modules, backup-vault and backup-plan, in our terraform-aws-data-storage repo. These modules enable you to create and configure custom vaults, notifications, locks, backup plans and resource selections so you can ensure your critical data is backed up regularly and that your resulting recovery points are optionally protected against deletion. Meanwhile, we’re currently at work integrating AWS Backup support to our service catalog and to our Reference Architecture.

What to do about it: Clone our new modules and take them for a spin! You can either use your default vault or create custom vaults with their own separate notification and lock configurations to support your use cases. Check out our new examples for more information:

• default-vault-plan-and-selection
• vault-locks
• vault-notifications
• vault-plan-and-selection
• vault-recovery-points

Knowledge Base

Motivation: We’ve found that many of the same questions are repeated again and again in support. Slack doesn’t have great support for search, tagging, categories, etc, so we were looking for a better way to capture and expose common questions and answers.

Solution: We have created a Gruntwork Knowledge Base! This Knowledge Base allows us to capture questions and answers in a more reusable form, with support for organizing content using categories and labels, upvoting discussions to bring important topics to the top, marking answers as “official,” and vastly better search. In fact, the Knowledge Base is public, so it’ll start turning up answers in your Google searches. As it’s powered by GitHub Discussions under the hood, you don’t need any new login.

What to do about it: You can continue to use Slack for chat, but we encourage you to give the Knowledge Base a shot for asking questions and finding answers, and our team will start moving some of the questions in Slack to the Knowledge Base on your behalf so we can better capture the answers. Try it out and let us know how it works for you!

Winter break, 2021

Motivation: At Gruntwork, we are a human-friendly company, and we believe employees should be able to take time off to spend time with their friends and families, away from work.

Solution: The entire Gruntwork team will be on vacation December 20th — January 2nd. During this time, there may not be anyone around to respond to support inquiries, so please plan accordingly.

What to do about it: We hope you’re able to relax and enjoy some time off as well. Happy holidays!

Service Catalog Updates

terraform-aws-service-catalog

• v0.67.0: Exposed additional_security_groups from terraform-aws-eks control-plane; updated dependencies. Refer to the release notes for more details.
• v0.68.0: Exposed ability to specify additional security groups for the EKS cluster workers (using the new input var additional_security_groups_for_workers). As a part of this change, the input var additional_security_groups on eks-cluster module has been renamed to additional_security_groups_for_control_plane.
• v0.68.1: Exposed the ID of the common security group created for Managed Node Groups in the eks-cluster and eks-workers module.
• v0.68.2: Added bucket_kms_key_arn variable for SSE-KMS in s3-bucket module.
• v0.68.3: Fixed bug where allow_ssh_from_cidr was hardcoded to true in openvpn-server module. This will now be set to false if the allow_ssh_from_cidr_list list is empty; Added iam_role_id and iam_role_name outputs to ec2-instance module; Added the ability to configure S3 lifecycle rules to the private-s3-bucket module; Added the ability to configure cross region replication in the ecr-repos module.
• v0.68.4: Added the ability to configure ECR lifecycle polices.
• v0.68.5: Updated dependency terraform-aws-vpc version to 0.18.4.
• v0.68.6: Updated terraform-aws-vpc to v0.18.5 and exposed ability to disable binding of default NACLs with subnets.

Open Source Updates

Terragrunt

• v0.35.8: Fixed bug where deep dependency merge did not properly merge the config_path when the dependency block was redefined in the child.
• v0.35.9: Added new option --terragrunt-modules-that-include for run-all command which will restrict the run-all stack set to only those modules that include the given configuration file. This is useful for driving CI/CD workloads based on updates to common files that are included in child configurations.
• v0.35.10: Fixed bug where terragrunt was not treating apply -destroy as a destroy operation, so it was not taking into account the prevent_destroy flag.
• v0.35.11: Fixed bug where terragrunt could not handle args passed in as --terragrunt-option=value.
• v0.35.12: Fixed bug where terragrunt would log a “Failed to detect where module is used” warning unnecessarily.
• v0.35.13: Updated aws-sdk-go to v1.41.7; Fixed bug where Terragrunt would prefer local state files even if Terraform was configured with remote state.

Terratest

• v0.38.4: Updated RulePath of opa.EvalOptions to support any go-getter URL.

Kubergrunt

• v0.7.11: Updated the sync command to work correctly across all AWS regions.

helm-kubernetes-services

• v0.2.7: Updated the chart to use capabilities introspection to ensure the right API version is used for Ingress and PodDisruptionBudget resources based on the target kubernetes cluster version.
• v0.2.8: Updated the chart to allow specifying a sub path for the volume mount for ConfigMap.

Other updates

terraform-aws-ci

• v0.39.4: Fixed bug where terraform-update-variable included output from terraform wrappers like tfenv; Added support for passing through --terragrunt-modules-that-include.
• v0.39.5: Introduced a new script git-updated-files which will return the list of files that were modified between two refs, with filter functionality.
• v0.39.6: Fixed minor style issues in git-updated-files; fixed quoting in one of the error messages in infrastructure-deploy-script.

terraform-aws-lambda

• v0.14.2: Added the ability to attach additional security groups to the lambda function (using the new input variable additional_security_group_ids).
• v0.14.3: Exposed the ability to set custom tags and name on the IAM role created for the Lambda function.

terraform-aws-data-storage

• v0.22.2: Added support for managing AWS Backup service; Exposed ability to attach additional security groups to the RDS instance.
• v0.22.3: Fixed a bug where major version upgrades were broken for Postgres Aurora clusters.
• v0.22.4: Added support for restoring an Aurora cluster using a Point-in-Time restore. Refer to the variable documentation for restore_source_cluster_identifier for more details.

terraform-aws-eks

• v0.46.6: Upgraded helm chart version for cluster-autoscaler to include updated permissions; Exposed the ability to set force_update_version on the managed node group in the eks-cluster-managed-workers module.
• v0.46.7: New module to setup and manage CloudWatch Container Insights on EKS cluster. Refer to the docs for the new eks-cloudwatch-agent module for more information.

DevOps News

New serverless data stores: Redshift and MSK

What happened: AWS now offers Redshift Serverless and MSK Serverless, both in preview mode.

Why it matters: You can now make use of a data warehouse (Redshift) and Kafka without having to configure or manage clusters: no worrying about node types, node counts, capacity planning, scaling, etc.

What to do about it: Give the early preview a try and let us know what you think!

Feature flags using CloudWatch Evidently

What happened: AWS has launched a new service called CloudWatch Evidently that makes it easier to use feature flags in your code.

Why it matters: Feature flags are a way to separate deployment (i.e., rolling out a new version of your code onto servers) from release (i.e., making new functionality visible to users). Any time you are building a new feature, you wrap the newcode with an if-statement that checks the value of a feature flag. By default, the flag is always off, which means you can check the code in and deploy it to production without it having any impact on the user. You can then turn the feature on using a web UI, perhaps for employees only, then for 1% of users, then 10%, and so on, up to 100% (“ramping”). If anything goes wrong, you turn the feature back off. You can also use the same strategy to compare how different features perform: e.g., turn one feature on for 50% of users and another feature for the other 50% of users, and see which performs better (“A/B testing” or “bucket testing”). In the past, you’d either have to build your own feature flag service (which is much harder than it seems!) or use a 3rd party; now, AWS provides a feature flag service natively.

What to do about it: Give CloudWatch Evidently a shot and let us know what you think!

HashiCorp IPO

What happened: HashiCorp had their initial public offering (IPO), raising $1.2B.

Why it matters: An IPO can bring many benefits to a company: more money, more public awareness and brand recognition, more stability, etc.

What to do about it: Congratulate to everyone at HashiCorp on this huge milestone!

Security Updates

Below is a list of critical security updates that may impact your services. We notify Gruntwork customers of these vulnerabilities as soon as we know of them via the Gruntwork Security Alerts mailing list. It is up to you to scan this list and decide which of these apply and what to do about them, but most of these are severe vulnerabilities, and we recommend patching them ASAP.

Java/Log4J

CVE-2021–44228: 0-day Remote Code Execution vulnerability in log4j versions < 2.15.0. On December 10th we were alerted of a 0-day vulnerability in the log4j library, where a malicious log entry can open a remote shell into your servers. This can be exploited if any user controlled value is logged on your servers, which is very common on web server logs (e.g., the user agent string).

We advise that you update the log4j library in your applications to the latest version ASAP.

Note that for certain AWS Services such as OpenSearch Service, you will want to roll out the automatic updates if AWS has not rolled it out for you yet.

Also note the following for Gruntwork modules that manage Java based data stores:

• terraform-aws-elk: Rotate your Elasticsearch clusters to a patched version of Elasticsearch. If you are using the default settings from the module, we recommend upgrading to Elasticsearch version 6.8.21.
• terraform-aws-kafka: Kafka uses log4j v1, so is not vulnerable to this particular vulnerability.
• terraform-aws-zookeeper: Zookeeper uses log4j v1, so is not vulnerable to this particular vulnerability.
• terraform-aws-ci/modules/jenkins: Jenkins core does not include log4j.

For more information, refer to the following security disclosure, as well as the AWS service based disclosure.

We alerted the Gruntwork Security mailing list with this information on December 10th, 2021.