AWS CloudFormation Essentials — Concise Notes from the Field

A summary of CloudFormation concepts and experimental learnings from the field to help you quickly start building basic, nested or cross-reference stacks.

Published in

FAUN — Developer Community 🐾

12 min readOct 29, 2019

This article will walk you through the basic building blocks of AWS CloudFormation to simplify your AWS infrastructure provisioning and management requirements by writing meaningful templates.

It will also highlight some key concepts and provide important guidelines to help you write robust, reusable and error-free CloudFormation code.

CloudFormation Template Basics:

AWS CloudFormation template lets you declare the infrastructure resources and their settings in a file to achieve your Infrastructure as code goals. A good understanding of templates structure is the first important step towards creating a meaningful template. You can write the templates in either JSON or YAML format. Template samples presented in this article will use the YAML format for its cleanliness and readability reasons.

A basic skeleton of a CloudFormation template is shown in the below image. If you have noticed, the Resources field is the only mandatory field of a CloudFormation template, but in practice, most of the other optional fields mentioned below will find a place in your template. Let’s look at them one by one.

AWSTemplateFormatVersion: 2010-09-09  # OptionalDescription: # OptionalParameters: # OptionalMappings: # OptionalConditions: # OptionalResources:  # RequiredOutputs:  # Optional

➡️AWSTemplateFormatVersion: 2010–09–09: This is the default template format version supported by AWS at the time of this writing.

➡️Description: A basic introduction describing the purpose of the template.

➡️Parameters: Parameters allow you to pass user inputs at the time of stack creation or update. The below image shows a sample parameters section in a CloudFormation template.

A parameter name is a user-defined name and should be unique. The above example defines three parameters i.e. KeyName, VpcName, InstanceType.
Every defined parameter must have a parameter Type. Some commonly used types are, ‘Strings’, ‘Numbers’, ‘Lists’, ‘AWS defined specific types’. In the above example, AWS::EC2::KeyPair:: KeyName is an AWS specific type, which allows users to choose from a drop-down list of values at the time of stack creation. Please refer to AWS official documentation to see a list of supported AWS-Specific Parameter Types.
You can define a default parameter value in the template. The users can choose to either use the default value or provide a new value at the time of launching a stack.
When you create a stack using the CloudFormation console, parameters are listed in alphabetical order. If you have a long list of parameters and you want them to be listed in a specific order, or bundle them in multiple groups of parameters, you can use AWS::CloudFormation::Interface meta-data key. The sample code below groups the parameters into two categories, Network configurations, and Instance configurations.

Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
        Parameters
    ParameterLabels:

The following image displays a comparison of the CloudFormation console view when you create stacks using the two parameter examples shown in this article. Notice the labels (highlighted in green color) used to group the parameters using AWS::CloudFormation::Interface

How to access parameters? The intrinsic function !Ref allows you to access the value of your parameter in Resoures and Outputs sections of your template. see an example below.

Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 192.x.x.x
      Tags:
        - Key: Name
          Value: !Ref VpcName  # VpcName from parameters section

➡️Mappings: Allows you to create a key: value map of literal strings. A common use case of Mappings in CloudFormation is to declare a map of AWS region to AMI Ids. This mapping allows the use of the same template across multiple regions by referring to the different AMI Ids to region pairs defined in Mappings. see an example below.

Mappings:
  RegionMap:
    us-east-2:
      AMI: ami-08b4a0f6e106c1dba
      availabilityZone1: us-east-2a
      availabilityZone2: us-east-2b
      availabilityZone3: us-east-2c
    us-east-1:
      AMI: ami-0de53d8956e8dcf80
      availabilityZone1: us-east-1a
      availabilityZone2: us-east-1b
      availabilityZone3: us-east-1c
      availabilityZone3: us-east-1d

How to access Mappings? The intrinsic function Fn::FindInMap or its short form, !FindInMap is used to return the value associated with the key. The sample code below shows an EC2 Instance resource referring to the Mappings created in the above code.

Resources:
  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: !FindInMap
        - RegionMap
        - us-east-2
        - AMI
      AvailabilityZone: !FindInMap
        - RegionMap
        - us-east-2
        - availabilityZone2

The intrinsic function !Ref can also be used to refer to input parameters inside the FindInMap function instead of hardcoding values. You will also see the use of AWS predefined parameters AWS::Region (aka Psuedo parameters) which returns the string value of the AWS region where the resource is being created.

👉 Above mentioned ‘AMI Id mapping to a region’ use case has a drawback. It would require us to manually replace the AMI Id of the image in the template to keep the AMI at the latest version released by AWS. There is a better way to query the latest AMI’s during stack deployment or update by using the type AWS::SSM::Parameter::Value<AWS::EC2::Image::Id> in your templates parameter section. Refer AWS Documentation on how this works for Linux/windows AMI’s. Below image shows an example code.

Fetch latest Windows Server 2016 AMI during stack creation

➡️Conditions: This section of the template allows you to pre-condition the creation of a resource or generate the output if the condition statement is true or false. A condition is evaluated based on the input parameter values supplied by the user at stack launch.

Following intrinsic functions can be used to create conditions in the template. Most of these intrinsic functions are self-explanatory. Fn::And , Fn::Equals , Fn::If , Fn::Not , Fn::Or
Conditions are useful for reusing a single template to create resources based upon the True or False status of a condition.
A very common example you will find for conditions is deploying a different set of resources for ‘prod’ and ‘dev’ environments by passing these two values via parameters and evaluating them inside Conditions and then attaching the condition to the required resource.

Below example showcases the use of Conditions for deploying different editions of MS SQL Server RDS.

➡️Resources: This section of the template must be present in a cloud formation template for stack creation. You can define each AWS infrastructure resource separately in this section. During stack creation/update these resources will be deployed as per their definition in this section.

Each resource will have a logical id (resource logical name) and a physical id ( id assigned by AWS after resource creation). You can use !Ref to refer the resource. The physical id of the resource will only be available after resource creation is successfully completed. For example, subnet-af532dd5 is a physical id of a subnet after it is successfully created.
The below example shows the definition of an EC2 Instance resource and its properties. Each resource will have an AWS defined resource Type and properties. Refer AWS Official documentation for the list of resources and their properties.

➡️Outputs: This is one of the very important section of your template when you plan to return resource values from your current stack to be used by other stacks in a cross-stack reference or nested stack scenario.

For example, if you have a stack which creates a new VPC and Subnets, you would probably like to return the Id’s of the created VPC and Subnets, so that these values can be consumed by other stacks which are creating EC2 Instances or deploying an AWS service.

In the above example Export key would make the outputs available upon stack completion by their respective names as mentioned in ‘Name’ key. You can refer to these resources in another stack by the value of ‘Name’ field for a cross-stack reference purpose. The value field would return the physical id of the resource like `vpc-017689f9594

This is all you need to know about the basics of CloudFormation to start writing your templates. Next sections of the article will cover ‘cross stack references’ and ‘nested stacks’.

Cross-Stack References:

Consider a case where you are using one CloudFormation template to deploy a set of infrastructure components such as VPC, Subnets, SecurityGroups, etc and there are other templates that you are using for deploying applications or services like EC2 Instances.
In this scenario, you can export the values of resources created in the first stack and refer them inside the application template. You can use the intrinsic function !ImportValue to import the values which were exported in the first stack.

Below example imports the VPC id which was created in another stack. This VPC id was exported in the other stack by the name ‘TestVpc’.

PublicSubnet2:
Type: 'AWS::EC2::Subnet'
Properties:
  VpcId: !ImportValue TestVpc

👉 The exported name must be unique for the AWS account within a region. Its recommended to use a unique identifier ( a Tag) for each stack and append this tag to the exported values. This tag can be passed as a parameter during stack launch and dynamically referred in output sections export name by using !Sub. See the below example for tagging your stack.

PublicSubnet1:
Value: !Ref Subnet1
Export:
  Name: !Sub ${myunique_id}-myvpc //myunique_id is a Tag prefixed before the actual name 'myvpc' and will be replaced with user supplied parameter value.

Nested Stacks:

A nested stack lets you stitch together multiple individual CloudFormation templates into a single template for the deployment of resources using AWS::CloudFormation::Stack resource.

Nested stack template acts as a parent template for the deployment of all other templates which are referred inside the parent template.

A nested stack is declared as of Type AWS::CloudFormation::Stack
All the child templates referred in parent template should be accessible to the parent template via the Amazon S3 bucket.
You provide input parameters at the time of launching the parent template. These input parameters will be passed on to child templates as per the declaration.
You can refer output values of a child stack as an input parameter for the subsequent child stacks.
When you launch a nested stack, AWS CloudFormation console will highlight NESTED keyword in front of each child stack.

Let’s look at a basic nested stack example. Please note that this example is only for representation purpose to highlight some key concepts in the nested stack creation and is not a complete example.

This example has two templates vpcw.yml and SG.yml, which are accessed in the parent.yml template. All these three files are uploaded to an S3 bucket named bkt7 and parent.yml refers to the child templates using S3 URL.

File: vpcw.yml

AWSTemplateFormatVersion: 2010-09-09
Description: This template deploys a VPCParameters:
  VpcCIDR:
  Description: IP range (CIDR notation) for this VPC
  Type: String
Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VpcCIDR
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: VPC1
  PrivateSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: !Select [ 0, !GetAZs '' ]
      CidrBlock: ....
Outputs:
  VPC:
    Description: A reference to the created VPC
    Value: !Ref VPC

File: SG.yml

AWSTemplateFormatVersion: 2010-09-09
Description: This template creates securitygroups and access rulesParameters:
  VpcId:
  Description: Id of new vpc created using nested stack
  Type: AWS::EC2::VPC::Id
Resources:
  SecurityGroup1:
  Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: "Service Members"
      VpcId: !Ref VpcId
      SecurityGroupIngress: 
        - { IpProtocol: '-1', FromPort: 0, ToPort: 65535, CidrIp:         0.0.0.0/0 }

File: parent.yml

AWSTemplateFormatVersion: 2010-09-09
Description: This is a parent template to deploy child templates.Parameters:
  SourceCidrForRDP:
    Description: CIDR range for remote access
    Type: String
  VpcCIDR:
    Description: CIDR range for VPC. Example 192.168.0.0/16
    Type: StringResources:
  vpcw:
    Type: AWS::CloudFormation::Stack   #VPC creation
    Properties:
      TemplateURL: https://s3.us-east-2.amazonaws.com/bkt7/vpcw.yml
      Parameters:
        CIDR: !Ref VpcCIDR  SecurityGroups:
    DependsOn: vpcw
    Type: AWS::CloudFormation::Stack  #Security groups creation
    Properties:
      TemplateURL: https://s3.us-east-2.amazonaws.com/bkt7/SG.yml
      Parameters:
        VpcId: !GetAtt vpcw.Outputs.VPC  # output reference
        SourceCidrForRDP: !Ref SourceCidrForRDP

👉 Notice VpcId: !GetAtt vpcw.Outputs.VPC in the above example. vpcw is the name of the previous child stack. Outputs.VPC refers to the Output values declared in the vpcw stack. GetAtt is used to refer an attribute ‘VPC’ which is created when vpcw stack creation completes successfully.

When you launch the stack using parent.yml, the AWS CloudFormation console view will list the child stack status as ‘NESTED’.

AWS CloudFormation console view of a nested stack

AWS Intrinsic Functions:

This article has already mentioned or used a few intrinsic functions in sample examples. These are built-in functions provided by AWS to manage your stacks.

👉 Yaml can use the short form of these functions which does not include Fn:: and instead uses ! and function name. Consecutive nesting of the short form of intrinsic functions is not allowed. Refer AWS documentation to know about the correct syntax when you are using intrinsic functions in their short form.

Let’s look at some of the frequently used intrinsic functions.

Fn::GetAtt Returns the value of an attribute after a resource is created. A complete list of such attributes is available at AWS Documentation. Its short Yaml form is !GetAtt
Fn::GetAZs Returns a list of availability zones for a region. Its short Yaml form is !GetAzs
Fn::Select Returns a single object from the list of objects. Its short Yaml form is !Select

AvailabilityZone: !Select # short form of Fn::Select
  - 0                     # index to be returned from the list
  - Fn::GetAZs: !Ref 'AWS::Region'

Ref Returns the value of a parameter or resource. Short Yaml form !Ref
Fn::ImportValue Returns the value of output exported from another stack. Short Yaml form !ImportValue
Fn::Join Joins a set of values in a single value based upon the delimiter provided. In the below example, a blank delimiter is provided as “”, thus it will return a value by joining VPC id, string -mydepartmentname- and availability zone name. Short Yaml form !Join

Tags:
  - Key: Name
    Value: !Join [ "", [ !ImportValue VPC, "-mydepartmentname", !Sub "${AvailabilityZone}" ]]

Fn:Sub Substitutes values in an input string with the values specified at run time. Refer AWS Documentation for further details.

!Sub ${variable_name}# Example
!Sub ${AWS::StackName} # stack name given by the user

Fn:Cidr Returns an array of CIDR blocks. In the below example, ipBlock is your CIDR range i.e. 10.0.0.0/16, the count is the number of subnets, and cidrBits are host bits.

!Cidr [ ipBlock, count, cidrBits ]

CloudFormation Template Editor:

You can use any editor of your choice to write your CloudFormation templates. Since you will deal with lots of Yaml or JSON code, make sure your editor settings are enabled for proper indentation and highlight any indentation issues.

I prefer Visual Studio Code, due to its support for various extensions which provide assistance for writing well-indented code, highlighting any indentation errors, CloudFormation documentation, and linting support. To know more about configuring vscode for CloudFormation, refer Up your AWS CloudFormation game with Visual Studio Code

CloudFormation lint is a very useful feature, which ensures that your template is error-free and highlights any CloudFormation related errors even before you launch your template. It’s done beforehand for you by the cfn-python-lint.

How to launch a CloudFormation template?

Log in to the AWS CloudFormation console and click Create New Stack to launch the stack creation wizard. Select your template and follow the wizard.

You can also use AWS CLI to create your stack as shown below.

aws cloudformation create-stack --stack-name test-stack 
--template-url https://s3.amazonaws.com/mybucket/testvpc.yml
--parameters ParameterKey=KeyName,ParameterValue=YOUR_KEY_NAME, ...

A complete example:

So far, this article covered the key concepts of the AWS CloudFormation. Now, its time to showcase a working example where we will be using these concepts to create a CloudFormation template that deploys a VPC and public and private subnets. This template will auto calculate CIDR ranges for the required subnets using !Cidr intrinsic function and only expects a VPC CIDR range for this calculation.

Network Automation Template

Summary

I hope this post was easy to follow and helped you to better understand the application of AWS CloudFormation for automating your AWS infrastructure.

Good Luck.

— — — — — —

Follow us on Twitter 🐦 and Facebook 👥 and join our Facebook Group 💬.

To join our community Slack 🗣️ and read our weekly Faun topics 🗞️, click here⬇