Introduction

Amazon S3 is a simple storage service that allows you to store, retrieve any amount of data ensuring scalability, reliability, easy access and inexpensive in nature. It stores data in buckets with different security configurations as per your need. It is designed to provide 99.999999999% durability and 99.99% availability of objects over a given year.

Aws S3 NotificationConfiguration

S3 bucket provides a notification feature which lets you receive notification when certain events occur. Some of the common events supported by buckets are objectCreated, objectDeleted, objectRestore. For more events, you can check out the link here. Likewise, it provides three different ways to notify the events. We will discuss about the type Aws Lamba in this post.

• AWS Lambda
• Simple Notification service (SNS)
• Simple Queue service (SQS)

Customer purchase notification system

To get the practical concept of how S3 notification works, we will build a simple customer purchase notification system. It will comprise of apigateway, Lambda functions, S3 bucket notification and email notification backed by Aws SES. The design of the system is shown in the diagram below. Each resource is briefly explained. Later we will build these resources with a complete cloudformation template. The complete code base is available in the Github link here

API gateway

This is a simple Amazon apigateway to register a new purchase transaction . It consists of a single POST request carrying purchase information in the request body. The request will be handled by the backend lambda. For simplicity, we will not add any authentication in the api. The purchase information will consist of following things. A sample JSON request body is given below.

Note: Email is a required field. It will be used as the destination email address.

Purchase Input Lambda

It will take in the purchase information and is responsible to store the data into the S3 bucket as shown in the diagram. The file created in the bucket will be in json format.

Purchase Processing Lambda

Among the three ways of notification as mentioned earlier, we will use the AWS Lambda where every time a file is created in the S3, the processing lambda will be triggered. It will parse the purchase information and send the email to the desired recipient. The sample email is given below.

System Implementation

The system will be completely based on the AWS technologies. Cloudformation template is used to provision the resources. To know more about cloudformation, visit the official link here. Now, lets dig down into the details.

We will focus mostly on the NotificationConfiguration part of the template which defines the events we want to handle and the notification option we want to choose.

When the cloudformation template is run, the above two resources will do following things :-

• Create a bucket item-store.
• Configures Aws Lamba notification for the event S3:ObjectCreated on that bucket. It means, whenever an object is created in item-store , it will call PurchaseDataProcess lambda. All these settings has to be under the NotificationConfiguration section.
• BucketPermission resource sets the permission on PurchaseDataProcess which allows item-store bucket to invoke the function.

There are few Aws cloudformation intrinsic functions like Sub, Ref, GetAtt. If you dont have idea of these then view the link here.

There are other sections in the cloudformation template like creating API, creating Policy documents and roles, Lambda functions. These are our secondary topics so we wont go in detail on that. The codebase of the Lambda functions is available in the Github link.

System requirements to run

To run this infrastructure in AWS, you need to have following things setup.

• AWS account and the IAM user setup. Link here
• Install and configure awscli. Link here
• Create an aws profile to run awscli commands. Link here. You will use this profile to run the infrastructure.
• SES email verification. Source and destination email has to be verified before sending the email. Link here

How to deploy the infrastructure

I have created a bash script to package and deploy the cloudformation template. You may need to change few configuration parameters before you can deploy it to the AWS. Set the parameters as per your need. Set the profile name you created in the above steps.
This script will package the source code, save the package in the separate bucket s3-demo-lab (you have to create it separately) and finally deploy it.

Run ./deploy.sh

How to call the POST http request

To call the post request, you can use the call.sh script. It uses the curl request to call the deployed Api. Before calling it, put all your purchase details in the event.json file. The file contains the post request JSON data. A successful response will return message Customer purchase recorded.

Conclusion

I hope with this, you get the knowledge of S3 bucket notification feature, its types and detail steps to configure LambdaConfiguration. Please go through the help links if you are confused in any step. Please leave the feedback and suggestions.