An Interest In:
Web News this Week
- April 3, 2024
- April 2, 2024
- April 1, 2024
- March 31, 2024
- March 30, 2024
- March 29, 2024
- March 28, 2024
Some of Our Sources
- Slashdot
- Pearsonified
- The Logo Smith
- Spoon Graphics
- Smashing Magazine
- Naldz Graphics
- 24 Ways
- Stylized Web
- Wal You
- Dev To
Help Webnuz
Referal links:
Golang Tutorial: Upload Bulk Files to aws S3 Using Golang
AWS S3 or Amazon S3 is a storage system from Amazon to store and retrieve files from anywhere on the web. It is a highly scalable, reliable, fast, inexpensive data storage system from Amazon. It provides easy to use developer kit to store and retrieve files.
In this tutorial you will learn how to upload bulk files to S3 bucket aws golang. We will use the Go AWS SDK to upload files to AWS S3.
The AWS SDK for Go requires Go 1.15 or later versions. You can view your current version of Go by running the following command.
go versionFor information about installing or upgrading your version of Go, see https://golang.org/doc/install
Before you can use the AWS SDK for Go V2, you must have an Amazon account. See How do I create and activate a new AWS account? for details
Download the AWS Golang SDK for using in your codebase using below command
go get github.com/aws/aws-sdk-go/awsWe will cover this tutorial step by step to upload bulk files to AWS S3 Bucket server using Golang with example code.
Load require packages
We will create main.go file and import necessary packages we need to upload the files into AWS S3 Bucket.
package mainimport ( "bytes" "fmt" "log" "net/http" "os" "github.com/aws/aws-sdk-go/aws" "github.com/aws/aws-sdk-go/aws/credentials" "github.com/aws/aws-sdk-go/aws/session" "github.com/aws/aws-sdk-go/service/s3")Custom struct for receiver
We will be using a Session struct so we can use same it as receiver, and we will pass this receiver to our main upload function,
Via using a custom struct were eliminating the duplication from our code. Well be using only single session for our all bulk files to upload into s3 bucket aws golang.
type Session struct { S3Session *session.Session}Set up main configuration and AWS session
We will setup configuration using AWS S3 REGION and SECRET ID and SECRET KEY and create single AWS session to upload multiple files to AWS S3. Then we will call method upload() and pass AWS session instance and file details to upload file to AWS S3 server.
func main() { paths := []string{"", ""} credential := credentials.NewStaticCredentials( os.Getenv("SECRET_ID"), os.Getenv("SECRET_KEY"), "", ) awsConfig := aws.Config{ Region: aws.String(os.Getenv("REGION")), Credentials: credential, } s, err := session.NewSession(&awsConfig) if err != nil { log.Println("failed to create S3 session:", err.Error()) return } se := Session{s} err = se.upload(paths) if err != nil { log.Println(err.Error()) return }}Create Upload Method to Upload Bulk Files to AWS S3
We will create method upload() to upload files to AWS S3 server. We will open the file one by one and store into buffer and then put the file to AWS S3 using PutObject() method from S3. We will also specify options in the PutObjectInput when uploading the file. We will also enable AES256 encryption on files using ServerSideEncryption options of s3.PutObjectInput struct.
func (s Session) upload(paths []string) error { for _, path := range paths { upFile, err := os.Open(path) if err != nil { log.Printf("failed %s, error: %v", path, err.Error()) continue } defer upFile.Close() upFileInfo, err := upFile.Stat() if err != nil { log.Printf("failed to get stat %s, error: %v", path, err.Error()) continue } var fileSize int64 = upFileInfo.Size() fileBuffer := make([]byte, fileSize) upFile.Read(fileBuffer) // uploading _, err = s3.New(s.S3Session).PutObject(&s3.PutObjectInput{ Bucket: aws.String(os.Getenv("BUCKET_NAME")), Key: aws.String(path), ACL: aws.String("public-read"), // could be private if you want it to be access by only authorized users Body: bytes.NewReader(fileBuffer), ContentLength: aws.Int64(int64(fileSize)), ContentType: aws.String(http.DetectContentType(fileBuffer)), ContentDisposition: aws.String("attachment"), ServerSideEncryption: aws.String("AES256"), StorageClass: aws.String("INTELLIGENT_TIERING"), }) if err != nil { log.Printf("failed to upload %s, error: %v", path, err.Error()) continue } url := "https://%s.s3-%s.amazonaws.com/%s" url = fmt.Sprintf(url, os.Getenv("BUCKET_NAME"), os.Getenv("REGION"), path) fmt.Printf("Uploaded File Url %s
", url) } return nil}you can checkout more article like this on my personal blog
Original Link: https://dev.to/mavensingh/golang-tutorial-upload-bulk-files-to-aws-s3-using-golang-6kg
Dev To
An online community for sharing and discovering great ideas, having debates, and making friendsMore About this Source Visit Dev To