An Interest In:
Web News this Week
- April 2, 2024
- April 1, 2024
- March 31, 2024
- March 30, 2024
- March 29, 2024
- March 28, 2024
- March 27, 2024
June 20, 2022 11:59 am GMT
Original Link: https://dev.to/gondar00/how-to-bulk-upload-files-to-s3-using-node-script-c37
How to bulk upload files to S3 using node script
In this article let's see a small nodejs script which can be run to upload files to amazon S3 bucket.
Prerequisite
I am going to assume you understand what amazon s3 is using for. If not you can visit this link to learn more.
Writing the upload script
// uploadTos3.jsconst fs = require('fs');const readDir = require('recursive-readdir');const path = require('path');const AWS = require('aws-sdk');const mime = require('mime-types');AWS.config.update({ region: 'ap-south-1', accessKeyId: process.env.AWS_ACCESS_KEY_ID, secretAccessKey: process.env.SECRET_ACCESS_KEY, maxRetries: 3});const directoryPath = path.resolve(__dirname, './images');const getDirectoryFilesRecursive = (dir, ignores = []) => { return new Promise((resolve, reject) => { readDir(dir, ignores, (err, files) => (err ? reject(err) : resolve(files))); });};const generateFileKey = fileName => { const S3objectPath = fileName.split('/images/')[1]; return `website-assets/${S3objectPath}`;};const s3 = new AWS.S3();const uploadToS3 = async () => { try { const fileArray = await getDirectoryFilesRecursive(directoryPath, [ 'BUILD_ID' ]); fileArray.map(file => { // Configuring parameters for S3 Object const S3params = { Bucket: 'public-read-bucket', Body: fs.createReadStream(file), Key: generateFileKey(file), ACL: 'public-read', ContentType: mime.lookup(file), ContentEncoding: 'utf-8', CacheControl: 'immutable,max-age=31536000,public' }; s3.upload(S3params, function(err, data) { if (err) { // Set the exit code while letting // the process exit gracefully. console.error(err); process.exitCode = 1; } else { console.log(`Assets uploaded to S3: `, data); } }); }); } catch (error) { console.error(error); }};uploadToS3();
Note: Im uploading all the assets to website-assets folder. The cloudfront distribution will have that as a base path.
Run the command node uploadToS3
to upload all the files to S3 bucket.
Original Link: https://dev.to/gondar00/how-to-bulk-upload-files-to-s3-using-node-script-c37
Share this article:
Tweet
View Full Article
Dev To
An online community for sharing and discovering great ideas, having debates, and making friendsMore About this Source Visit Dev To