Your Web News in One Place

Help Webnuz

Referal links:

Sign up for GreenGeeks web hosting
June 28, 2021 03:43 pm GMT

Provisioning an RDS Database with CloudFormation

In part 1, we automated the provisioning of your Amazon EC2 instance using AWS CloudFormation. When you built the EC2 instance manually in the past, you were seeing inconsistencies between environments, had to manually test your infrastructure setup, manually deploy your team's web app, which happens all to infrequently. All of this has been error prone and time consuming. Bugs introduced into production code are more difficult and expensive to fix and your customers were ultimately the ones who suffered.

In addition to the EC2 instance, you also need a Postgresql database. In the past, it was running on the same instance as your webapp and sometimes web app traffic impacted the database and vice versa. In this post, part 2, you'll add an Amazon RDS Postgresql database to the CloudFormation template you built in part 1 so that both the EC2 instance and the database can be provisioned together as a set of resources. And since RDS is a managed service, you wont have to do operating system patches or database patches its all managed for you. And eventually, you want to introduce a continuous integration/continuous deployment (CI/CD) pipeline to automate the build, test, and deploy phases of your release process. Both part 1 and part 2 set you up to do that.

As a reminder, here's what we covered and where we're going:

  • automate the provisioning of your Amazon EC2 instance using AWS CloudFormation (part 1),
  • add an Amazon RDS Postgresql database to your stack with CloudFormation (this post, part 2), and
  • create an AWS CodePipeline with CloudFormation (part 3).

Prerequisites

To work through the examples in this post, youll need:

  • an AWS account (you can create your account here if you dont already have one),
  • the AWS CLI installed (you can find instructions for installing the AWS CLI here), and
  • a key-pair to use for SSH (you can create a key-pair following these instructions).

Unfamiliar with CloudFormation or feeling a little rusty? Check out part 1 or my Intro to CloudFormation post before getting started.

Updating the CloudFormation Template

First we'll add the RDS database and then we'll add a security group to allow inbound traffic on port 5432. At the end of this post, youll delete the stack youve created and any snapshots so that you dont incur any charges and then you can (quickly) customize and recreate the stack in the future.

Just want the code? Grab it here.

Lets get started!

1. Add RDS Postgresql Database

First, we'll add an RDS database resource with the type AWS::RDS::DBInstance to the CloudFormation template. We set the Engine to the database engine we want to use, in this case postgres.

# 05_rds.yamlAWSTemplateFormatVersion: 2010-09-09Description: Part 2 - Add a database with CloudFormationParameters:  AvailabilityZone:    Type: AWS::EC2::AvailabilityZone::Name  EnvironmentType:    Description: 'Specify the Environment type of the stack.'    Type: String    Default: dev    AllowedValues:      - dev      - test      - prod  AmiID:    Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>    Description: 'The ID of the AMI.'    Default: /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2  KeyPairName:    Type: String    Description: The name of an existing Amazon EC2 key pair in this region to use to SSH into the Amazon EC2 instances.  DBInstanceIdentifier:    Type: String    Default: 'webapp-db'  DBUsername:    NoEcho: 'true'    Description: Username for Postgresql database access    Type: String    MinLength: '1'    MaxLength: '16'    AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'    ConstraintDescription: Must begin with a letter and contain only alphanumeric characters.    Default: 'postgres'  DBPassword:    NoEcho: 'true'    Description: Password Postgresql database access    Type: String    MinLength: '8'    MaxLength: '41'    AllowedPattern: '[a-zA-Z0-9]*'    ConstraintDescription: Must contain only alphanumeric characters.Mappings:  EnvironmentToInstanceType:    dev:      InstanceType: t2.nano    test:      InstanceType: t2.micro    prod:      InstanceType: t2.smallResources:  WebAppInstance:    Type: AWS::EC2::Instance    Properties:      AvailabilityZone: !Ref AvailabilityZone      ImageId: !Ref AmiID      InstanceType: !FindInMap [EnvironmentToInstanceType, !Ref EnvironmentType, InstanceType]      KeyName: !Ref KeyPairName      SecurityGroupIds:        - !Ref WebAppSecurityGroup  WebAppSecurityGroup:    Type: AWS::EC2::SecurityGroup    Properties:      GroupName: !Join [ '-', [ webapp-security-group, !Ref EnvironmentType ] ]      GroupDescription: 'Allow HTTP/HTTPS and SSH inbound and outbound traffic'      SecurityGroupIngress:        - IpProtocol: tcp          FromPort: 80          ToPort: 80          CidrIp: 0.0.0.0/0        - IpProtocol: tcp          FromPort: 443          ToPort: 443          CidrIp: 0.0.0.0/0        - IpProtocol: tcp          FromPort: 22          ToPort: 22          CidrIp: 0.0.0.0/0  WebAppEIP:    Type: AWS::EC2::EIP    Properties:      Domain: vpc      InstanceId: !Ref WebAppInstance      Tags:        - Key: Name          Value: !Join [ '-', [ webapp-eip, !Ref EnvironmentType ] ]  WebAppDatabase:    Type: AWS::RDS::DBInstance    Properties:      DBInstanceIdentifier: !Ref DBInstanceIdentifier      AllocatedStorage: '5'      DBInstanceClass: db.t2.small      Engine: postgres      MasterUsername: !Ref DBUsername      MasterUserPassword: !Ref DBPassword      Tags:        - Key: Name          Value: !Join [ '-', [ webapp-rds, !Ref EnvironmentType ] ]    DeletionPolicy: Snapshot    UpdateReplacePolicy: SnapshotOutputs:  WebsiteURL:    Value: !Sub http://${WebAppEIP}    Description: WebApp URL  WebServerPublicDNS:    Description: 'Public DNS of EC2 instance'    Value: !GetAtt WebAppInstance.PublicDnsName  WebAppDatabaseEndpoint:    Description: 'Connection endpoint for the database'    Value: !GetAtt WebAppDatabase.Endpoint.Address

In the template above, we've also added new parameters to customize the name of the database, the username, and password, as well as two new outputs to make our work easier: WebServerPublicDNS and WebAppDatabasePublicDNS. We'll use both of these outputs in step 2.

If you're creating your stack from scratch, you can run the create-stack command:

$ aws cloudformation create-stack --stack-name rds-example --template-body file://05_rds.yaml \--parameters ParameterKey=AvailabilityZone,ParameterValue=us-east-1a \ParameterKey=EnvironmentType,ParameterValue=dev \ParameterKey=KeyPairName,ParameterValue=jenna \ParameterKey=DBPassword,ParameterValue=Abcd1234

Or, if you're updating the stack you created in part 1, you can use the update-stack command instead.

Notice that we're only specifying the new DBPassword parameter and relying on the default values for DBInstanceIdentifier and DBUsername that are specified in the template. If you'd like to customize these values, you can add them to the command in the same ParameterKey,ParameterValue format shown above.

DeletionPolicy and UpdateReplacePolicy

You may have also noticed the DeletionPolicy and UpdateReplacePolicy properties set to Snapshot. If and when you ever need to re-provision the database, you'll want to make sure you don't lose your precious data. Some template changes will require the resource to be recreated (as opposed to updated). When this happens, you'll want to be in control of how that happens and what happens to your data. These two properties give you that control. In the case of Snapshot, CloudFormation will create a snapshot of the database when the stack is updated or deleted. You can read more about update and delete behaviors of stack resources here.

After creating or updating the stack, you'll now have your EC2 instance (and supporting resources from part 1) and an RDS database. Right now, no one can access that database instance from the outside world, so next we'll enable inbound traffic to the Postgresql port.

2. Enable Inbound Traffic on Port 5432

To allow inbound traffic on port 5432 so that our EC2 instance can talk to the RDS database, we'll add a security group with type AWS::EC2::SecurityGroup.

# 06_rds.yamlAWSTemplateFormatVersion: 2010-09-09Description: Part 2 - Add a database with CloudFormationParameters:  AvailabilityZone:    Type: AWS::EC2::AvailabilityZone::Name  EnvironmentType:    Description: 'Specify the Environment type of the stack.'    Type: String    Default: dev    AllowedValues:      - dev      - test      - prod  AmiID:    Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>    Description: 'The ID of the AMI.'    Default: /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2  KeyPairName:    Type: String    Description: The name of an existing Amazon EC2 key pair in this region to use to SSH into the Amazon EC2 instances.  DBInstanceIdentifier:    Type: String    Default: 'webapp-db'  DBUsername:    NoEcho: 'true'    Description: Username for Postgresql database access    Type: String    MinLength: '1'    MaxLength: '16'    AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'    ConstraintDescription: Must begin with a letter and contain only alphanumeric characters.    Default: 'postgres'  DBPassword:    NoEcho: 'true'    Description: Password Postgresql database access    Type: String    MinLength: '8'    MaxLength: '41'    AllowedPattern: '[a-zA-Z0-9]*'    ConstraintDescription: Must contain only alphanumeric characters.Mappings:  EnvironmentToInstanceType:    dev:      InstanceType: t2.nano    test:      InstanceType: t2.micro    prod:      InstanceType: t2.smallResources:  WebAppInstance:    Type: AWS::EC2::Instance    Properties:      AvailabilityZone: !Ref AvailabilityZone      ImageId: !Ref AmiID      InstanceType: !FindInMap [EnvironmentToInstanceType, !Ref EnvironmentType, InstanceType]      KeyName: !Ref KeyPairName      SecurityGroupIds:        - !Ref WebAppSecurityGroup  WebAppSecurityGroup:    Type: AWS::EC2::SecurityGroup    Properties:      GroupName: !Join [ '-', [ webapp-security-group, !Ref EnvironmentType ] ]      GroupDescription: 'Allow HTTP/HTTPS and SSH inbound and outbound traffic'      SecurityGroupIngress:        - IpProtocol: tcp          FromPort: 80          ToPort: 80          CidrIp: 0.0.0.0/0        - IpProtocol: tcp          FromPort: 443          ToPort: 443          CidrIp: 0.0.0.0/0        - IpProtocol: tcp          FromPort: 22          ToPort: 22          CidrIp: 0.0.0.0/0  WebAppEIP:    Type: AWS::EC2::EIP    Properties:      Domain: vpc      InstanceId: !Ref WebAppInstance      Tags:        - Key: Name          Value: !Join [ '-', [ webapp-eip, !Ref EnvironmentType ] ]  WebAppDatabase:    Type: AWS::RDS::DBInstance    Properties:      DBInstanceIdentifier: !Ref DBInstanceIdentifier      VPCSecurityGroups:      - !GetAtt DBEC2SecurityGroup.GroupId      AllocatedStorage: '5'      DBInstanceClass: db.t2.small      Engine: postgres      MasterUsername: !Ref DBUsername      MasterUserPassword: !Ref DBPassword      Tags:        - Key: Name          Value: !Join [ '-', [ webapp-rds, !Ref EnvironmentType ] ]    DeletionPolicy: Snapshot    UpdateReplacePolicy: Snapshot  DBEC2SecurityGroup:    Type: AWS::EC2::SecurityGroup    Properties:      GroupName: !Join [ '-', [ webapp-db-security-group, !Ref EnvironmentType ] ]      GroupDescription: Allow postgres inbound traffic      SecurityGroupIngress:      - IpProtocol: tcp        FromPort: 5432        ToPort: 5432        SourceSecurityGroupName:          Ref: WebAppSecurityGroup      Tags:        - Key: Name          Value: !Join [ '-', [ webapp-db-security-group, !Ref EnvironmentType ] ]Outputs:  WebsiteURL:    Value: !Sub http://${WebAppEIP}    Description: WebApp URL  WebServerPublicDNS:    Description: 'Public DNS of EC2 instance'    Value: !GetAtt WebAppInstance.PublicDnsName  WebAppDatabaseEndpoint:    Description: 'Connection endpoint for the database'    Value: !GetAtt WebAppDatabase.Endpoint.Address

In the template above, we've added the database security group and added it to the VPCSecurityGroups property. We've also set SourceSecurityGroupName to the WebAppSecurityGroup we created earlier, to restrict the inbound traffic to that coming from the WebAppSecurityGroup, or our EC2 instance.

To update your stack, you'll use the update-stack command with the same parameters and values as before:

$ aws cloudformation update-stack --stack-name rds-example --template-body file://06_rds.yaml \--parameters ParameterKey=AvailabilityZone,ParameterValue=us-east-1a \ParameterKey=EnvironmentType,ParameterValue=dev \ParameterKey=KeyPairName,ParameterValue=jenna \ParameterKey=DBPassword,ParameterValue=Abcd1234

Now, you should have your EC2 instance (and supporting resources from part 1), an RDS database, and another security group exposing the database port. But how do we know the EC2 instance can talk to the database?

Testing the Security Group

To test your security group to make sure your EC2 instance can talk to the RDS database you provisioned in the last step, you can SSH into the instance and use the psql client to connect to the database.

If you don't remember how to SSH into the instance, you can read more about that in part 1 or grab the WebServerPublicDNS from the Outputs. Using the command below, replace the WebServerPublicDNS and YOUR_KEY_PAIR_NAME parts and SSH into the intstance:

$ ssh -i "YOUR_KEY_PAIR_NAME.pem" ec2-user@WebServerPublicDNS

Once you're in the EC2 instance, you'll need to enable the postgresql library from the Amazon Linux Extras repository and then install it. We could do this all in our CloudFormation template, but this is only for demonstration purposes, so we'll do it manually this one time. You can run these commands to do that:

sudo amazon-linux-extras list # Find the postgresql library and version to enable in the next commandsudo amazon-linux-extras enable postgresql13 -ysudo yum clean metadatasudo yum install postgresql

Now that psql is available, you can test the connection to the database like this, replacing WebAppDatabaseEndpoint with the corresponding value in the Outputs:

$ psql -h WebAppDatabaseEndpoint -U postgres -d postgresPassword for user postgres:psql (13.2, server 12.5)SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)Type "help" for help.postgres=>

Congrats! You've successfully allowed your EC2 instance to talk to your RDS database.

If the connection isn't successful, check the CloudFormation console to make sure the RDS database and security group resources were created successfully. Additionally, you can check to make sure the database has the security group attached and the inbound rule opens up port 5432 (the default postgres port).

Wrapping Up

Delete Your Stack & Snapshots

Dont forget to delete your stack so you dont accrue charges. You can do that with the delete-stack command:

$ aws cloudformation delete-stack --stack-name rds-example

If you left the DeletionPolicy and UpdateReplacePolicy properties set to snapshot and you no longer need those snapshots, then you can also delete those snapshots using the AWS Console so you don't accrue charges for those either.

Navigate to the RDS Management Console. From there, go to the Snapshots menu option. Select the snapshots created from your stack (hint: they will have a snapshot name that starts with your stack name) and select Delete snapshot from the Actions menu.

RDS Management Console showing selected snapshots to be deleted

What You Learned

In this post, we updated the CloudFormation template from part 1 to provision an RDS database and enabled inbound traffic for the database. Now that both your EC2 instance and RDS database (and supporting resources) are all managed with code, you can setup and teardown the stack of resources together. This sets you up for part 3, where we'll create an AWS CodePipeline with CloudFormation (part 3) so we can build, test, and deploy our web app to the through each environment to production.

You can grab the final CloudFormation template we created here.

Like what you read? Follow me here on Dev.to or on Twitter to stay updated!


Original Link: https://dev.to/aws/provisioning-an-rds-database-with-cloudformation-part-2-i6n

Share this article:    Share on Facebook
View Full Article

Dev To

An online community for sharing and discovering great ideas, having debates, and making friends

More About this Source Visit Dev To