If you're a full-stack software developer, you probably want to focus your time delivering core business value. Dealing with infrastructure, environment inconsistencies, manual testing, and manual deployment steps is time consuming, frustrating, and requires a different skill set, especially if you're part of a small team or are the solo developer at a budding startup. Provisioning infrastructure manually for an environment only happens so often, for instance for new projects or when a change needs to be made to a virtual machine or database configuration, but as you grow, your infrastructure needs to scale and so do your processes. Inconsistencies between environments can create problems down the pipeline, costing your small team time and money. And if your build and deploy process is also manual, then so is your rollback process. If you even have one. Because these processes are manual and time-consuming, they are done infrequently. Which means mistakes are made. When bugs are eventually found, they are often more severe, more difficult, and expensive to fix than if they had been found earlier.
Sound familiar? In my last post, we learned that Infrastructure as Code (IaC) is the management of your infrastructure resources and their dependencies with code. IaC allows you to scale quicker and easier, improve your quality, control your costs and risks, and know your infrastructure better.
With IaC, you can start solving the problems above and begin to free up your time so you can get back to building quality web apps quickly and delivering core business value. Today, we'll dive a little deeper with a look at AWS CloudFormation, an Infrastructure as Code framework. We'll cover templates, stacks, and change sets, and then take a look at five examples to demonstrate some of the foundational features to get started.
Prerequisites
To work through these examples, you'll need an AWS account and the AWS CLI installed. You can create your account here and find instructions for installing the AWS CLI here.
What is CloudFormation?
AWS CloudFormation is a framework for provisioning your cloud resources with infrastructure as code. You can manage and provision your AWS resources predictably and repeatedly with code. With a CloudFormation template, you define your AWS resources and manage that collection of resources together as one stack. By treating your infrastructure as code, you can manage the CloudFormation template in version control, just like you do your application code. With CloudFormation you can automate your best practices, scale your infrastructure across the globe, and integrate with other AWS services to control access, improve automation, testing, and controls.
Templates
A CloudFormation template is a JSON or YAML (skip the JSON and use YAML!) formatted text file where you will define your cloud resources. For example, your template could define an Amazon S3 bucket, give it a name, and configure it to have encryption enabled by default.
This is the basic anatomy of a CloudFormation template. The only required top-level object is Resources. In this post, we'll cover Parameters, Mappings, Resources, and Outputs.
AWSTemplateFormatVersion: 2010-09-09 # The only allowable value is 2010-09-09
Description: # Describes the template
Metadata: # Objects providing additional information about the template
Parameters: # Input parameters to pass to your template at runtime
Rules: # A set of rules to validate the parameters provided during creation or update
Mappings: # Key/value pairs used for lookup at runtime
Conditions: # Conditions that control whether certain resources are created or whether properties are assigned values
Transform: # Customizations for serverless applications
Resources: # Defines resources and their properties, the only required top-level object
Outputs: # Values that are returned whenever you view information about the resources created from this template
When you apply a template, in the AWS Console, using the APIs, or with the AWS CLI, you create a stack.
Stacks
A CloudFormation stack is an implementation of the resources defined in your template. If you're familiar with object-oriented programming, you can think of a template as a class and the stack as the implementation of that class, the object or objects.
A stack allows you to manage a collection of related resources as a single unit. When you want to make changes to resources in a stack, you'll update the template and create a change set.
Change Sets
A change set allows you to preview what will change when you apply the template to update the running stack. Each resource will behave differently depending on what type of resource it is and what you are changing about that resource or it's dependencies. For instance, if you are changing the BucketEncryption of an S3 Bucket, your bucket will be changed in place. However, for resources like an RDS instance, if you change anything that requires replacement, like DBName or AvailabilityZone, the database will be deleted and recreated, and you'll lose your data and all automated snapshots. There are ways of handling these scenarios with stack policies, but we won't be covering that today. Just know that it exists and change sets can help you see what will be changed in place or replaced altogether.
Templates in Action
Let's take a look at some CloudFormation templates in action. For readability and so that you can take these and run with them, these examples will all be in YAML format. If you need to convert them to JSON, you can use the handy tool, AWS CloudFormation Template Flip, to flip it from YAML to JSON.
Over the course of these next five examples, we will iterate on each template to extend it with new or changed configuration. This is how you will build your own templates. Even if you have one small configuration change to make, you re-apply the entire template to make that change.
Example 1: Resources
In this first example, we create an Amazon S3 bucket.
# file: 01_example_s3.yaml
Resources:
ExampleBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: 'intro-to-cloudformation-example-1'
Let's walk through the parts that make up this template.
- Resources: Top-level object where we define all resources in this stack
- ExampleBucket: The Logical ID for this resource. We'll use this ID to refer to this resource in other parts of our template.
- Type: This is the type of resource to create. You can find out more about the allowed properties for an Amazon S3 Bucket resource here and see all the supported resource types here.
- Properties: For each resource, you'll be able to configure it with properties.
- BucketName: The only property we're configuring is the bucket name, which is a string.
I like using the command-line, so that's what we'll use today, but you can also apply your template to create or update a stack using the AWS Console.
From the directory where your template file resides, run this command:
$ aws cloudformation create-stack --stack-name intro-to-cloudformation \
--template-body file://01_example_s3.yaml --region us-east-1
If everything goes as planned, you'll see the stack ID displayed. You can navigate to the CloudFormation stack in the AWS Console to view the status of the stack you just created. When it's complete, your bucket named bucket-example-1 will be created!
Example 2: Parameters and Intrinsic Functions
In the second example, we use parameters and intrinsic functions in our template.
# file: 02_example_s3.yaml
Parameters:
BucketName:
Type: String
Description: The name of the S3 bucket
Environment:
Description: This stack's environment.
Type: String
Default: dev
AllowedValues:
- dev
- test
- prod
Resources:
ExampleBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Ref BucketName
Tags:
- Key: Env
Value: !Join [ '-', [ !Ref Environment, environment ] ]
First, we've added the Parameters top-level object with a BucketName parameter that accepts a string. We've also added an Environment parameter with three allowed values and a default value of dev. When we apply this template to create or update a stack, we'll be prompted (in the AWS Console) or have to pass (at the AWS CLI) in these parameters.
Second, we're referencing the BucketName parameter when specifying the BucketName property of the resource. To do that, we're using the Fn:Ref intrinsic function. !Ref is a shortcut for Fn:Ref.
BucketName: !Ref BucketName
Third, we've also added a Tags property and we combine the Environment parameter value with a string by using Fn:Ref and the intrinsic function, Fn:Join. Fn:Join let's us join two strings together. We're joining the value of the Environment parameter with the string 'environment' to create "dev-environment" for the tag value.
Value: !Join [ '-', [ !Ref Environment, environment ] ]
Let's update our stack now!
$ aws cloudformation update-stack --stack-name intro-to-cloudformation \
--template-body file://02_example_s3.yaml --region us-east-1 \
--parameters ParameterKey=BucketName,ParameterValue=intro-to-cloudformation-example-2 \
ParameterKey=Environment,ParameterValue=dev
Notice we're using the update-stack command. Be sure to update the file name in the command if you're working from a new file.
Now, your bucket has been renamed based on what you input as a parameter and a tag dev-environment has been added to the bucket.
Example 3: Mappings
In this example, we'll use the Mappings top-level object to find a key/value pair to use later in our template. Mappings allow us to create resources with properties based on these key/value pairs at runtime.
# file: 03_example_s3.yaml
Parameters:
BucketName:
Type: String
Description: The name of the S3 bucket
Environment:
Description: This stack's environment.
Type: String
Default: dev
AllowedValues:
- dev
- test
- prod
Mappings:
EnvironmentToKeyARN:
dev:
KMSKeyARN: YOUR_DEV_KEY_ARN
test:
KMSKeyARN: YOUR_TEST_KEY_ARN
prod:
KMSKeyARN: YOUR_PROD_KEY_ARN
Resources:
ExampleBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Ref BucketName
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: 'aws:kms'
KMSMasterKeyID: !FindInMap [EnvironmentToKeyARN, !Ref Environment, KMSKeyARN]
Tags:
- Key: Name
Value: !Join [ '-', [ !Ref Environment, environment ] ]
Here, we've added the Mappings top-level object to map environments to key ARNs. To get the value for the environment we specify in the parameters, we'll use the Fn:FindInMap intrinsic function.
KMSMasterKeyID: !FindInMap [EnvironmentToKeyARN, !Ref Environment, KMSKeyARN]
Before applying this template, be sure to update the YOUR_DEV_KEY_ARN, YOUR_TEST_KEY_ARN, YOUR_PROD_KEY_ARN values to KMS key ARNs in your own account. Then, to update your stack, run:
$ aws cloudformation update-stack --stack-name intro-to-cloudformation \
--template-body file://03_example_s3.yaml --region us-east-1 \
--parameters ParameterKey=BucketName,ParameterValue=intro-to-cloudformation-example-3 \
ParameterKey=Environment,ParameterValue=dev
Example 4: Outputs
In our last example, we'll show how to specify output values so that you can see information about the resources you created. You can use outputs to import into other stacks and to see information like resource IP addresses, URLs, and IDs.
# file: 04_example_s3.yaml
Parameters:
BucketName:
Type: String
Description: The name of the S3 bucket
Environment:
Description: This stack's environment.
Type: String
Default: dev
AllowedValues:
- dev
- test
- prod
Mappings:
EnvironmentToKeyARN:
dev:
KMSKeyARN: YOUR_DEV_KEY_ARN
test:
KMSKeyARN: YOUR_TEST_KEY_ARN
prod:
KMSKeyARN: YOUR_PROD_KEY_ARN
Resources:
ExampleBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Ref BucketName
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: 'aws:kms'
KMSMasterKeyID: !FindInMap [EnvironmentToKeyARN, !Ref Environment, KMSKeyARN]
Tags:
- Key: Name
Value: !Join [ '-', [ !Ref Environment, environment ] ]
Outputs:
BucketARN:
Description: 'The ARN of the bucket'
Value: !GetAtt ExampleBucket.Arn
We've added the Outputs top-level object with one output value named BucketARN. We use the Fn:GetAtt intrinsic function to access the Amazon Resource Name (ARN) of the bucket.
You can apply the updated template to your stack:
$ aws cloudformation update-stack --stack-name intro-to-cloudformation \
--template-body file://04_example_s3.yaml --region us-east-1 \
--parameters ParameterKey=BucketName,ParameterValue=intro-to-cloudformation-example-4 \
ParameterKey=Environment,ParameterValue=dev
To see the output values, you can navigate to your stack in the AWS Console and view the Outputs tab for the stack or use the describe-stacks command at the CLI:
$ aws cloudformation describe-stacks --stack-name intro-to-cloudformation
This command will produce information about your stack similar to this, where you can see the output, BucketARN:
{
"Stacks": [
{
"StackId": "...",
"StackName": "intro-to-cfn",
...
"Outputs": [
{
"OutputKey": "BucketARN",
"OutputValue": "arn:aws:s3:::intro-to-cloudformation-example-4",
"Description": "The ARN of the bucket"
}
],
...
}
]
}
Example 5: Delete Your Stack
The last step here is to clean up the resources we created. You can delete your stack and all the resources along with it with this command:
$ aws cloudformation delete-stack --stack-name intro-to-cloudformation
Note: If you've added any objects to your Amazon S3 Bucket, this delete-stack command will fail because the bucket must be empty.
Without CloudFormation, it takes us a number of manual steps to configure this one bucket in the AWS Console. Imagine if we had to create a bunch of different buckets and configure more than just the bucket name. That would be tedious, time-consuming, and error-prone.
Our CloudFormation template to do the same is about 40 lines of YAML. However by moving this into code, we are able to automate the process of creating cloud resources, adhere to company and security best practices through reuse of templates, manage this code in version control, just like we would our application code.
Wrapping up
In this post, we looked at some foundational examples for writing your infrastructure code with AWS CloudFormation. We learned about:
- Templates, stacks, and change sets
- How to define resources in a template and where to find more information about them
- Using template parameters and intrinsic functions to make your template more dynamic
- Using mappings to dynamically map key/value pairs to change configuration at runtime
- Using outputs to get information about the resources you created in your stack and to feed them into other templates
- How to create, update, and delete your stack from the command line.
In the next post, we'll dive a little deeper and use each of these concepts to build a CloudFormation template to host your web app backed by a database.
Like what you read? Follow me on Twitter to stay updated!