{"id":1513,"date":"2022-11-15T12:07:11","date_gmt":"2022-11-15T12:07:11","guid":{"rendered":"https:\/\/www.workfall.com\/learning\/blog\/?p=1513"},"modified":"2025-09-24T10:23:31","modified_gmt":"2025-09-24T10:23:31","slug":"how-to-upload-large-files-1gb-and-beyond-to-aws-s3-using-nestjs-backend-and-reactjs-frontend","status":"publish","type":"post","link":"https:\/\/learning.workfall.com\/learning\/blog\/how-to-upload-large-files-1gb-and-beyond-to-aws-s3-using-nestjs-backend-and-reactjs-frontend\/","title":{"rendered":"How to upload large files (1GB and beyond) to AWS S3 using NestJS (backend) and ReactJS (frontend)?"},"content":{"rendered":"Reading Time: <\/span> 5<\/span> minutes<\/span><\/span>\n
\"How<\/figure>\n\n\n\n

When dealing with file uploads, you must be aware that files are uploaded in buffers stored in memory and if the file is larger than the allocated memory in your VM, it may run out of memory and the application might crash. For example, if your allocated memory is 2GB, if you try to upload a file 2GB in size, the memory will max out.<\/p>\n\n\n\n

This calls for a change in how we carry out file uploads. As a result, multipart uploads have come to the rescue. We shall look at how we can write code on both the server and the client to accept multipart file uploads. On the client side, we shall use React and NestJS on the server side. We shall also use an S3 bucket for storing our files.<\/p>\n\n\n\n

NestJS is the most rapidly growing NodeJS framework in TypeScript and ReactJS. As it helps to break down the most complex of structures and work with individual components. AWS S3<\/a> is used for scalable object storage in the cloud and provides data availability, security, and performance that are unmatched in the market. These objects are often stored in binary format. We shall use the aws-sdk<\/code> npm package to interact with Amazon S3 in this blog.<\/p>\n\n\n\n

In this blog, we will demonstrate how to<\/p>\n\n\n\n

  1. configure AWS S3 buckets<\/li>
  2. write server-side applications using NestJS<\/li>
  3. write client-side applications using ReactJS<\/li><\/ol>\n\n\n\n

    Hands-on<\/h2>\n\n\n\n
    \"How<\/figure>\n\n\n\n

    In this hands-on, we will be implementing the following:<\/p>\n\n\n\n

    1. Creating a multipart upload<\/strong> – This makes a call to AWS S3 to create a unique uploadId which will be used to trace the parts that are being uploaded back to the original file for consolidation purposes.
      <\/li>
    2. Uploading parts<\/strong> – This is the main part where the file chunks are uploaded and part of the metadata in this operation is the uploaded.
      <\/li>
    3. Multipart upload completion <\/strong>– This is where the file is consolidated back into one complete object in S3 using the various parts that were uploaded.<\/li><\/ol>\n\n\n\n

      Let\u2019s configure the AWS S3 bucket: <\/strong><\/p>\n\n\n\n

      We shall begin by creating an S3 bucket as shown in the following image. You can type in the search bar in the top navigation; \u201cS3\u201d.<\/p>\n\n\n\n

      Give your bucket a name, select a region, and use all the other default values\/configurations.<\/p>\n\n\n\n

      \"upload<\/figure>\n\n\n\n
      \"\"\/<\/figure>\n\n\n\n
      \"upload<\/figure>\n\n\n\n

      Next, we need to Edit the CORS configuration for our S3 bucket as shown below.<\/p>\n\n\n\n

      \"upload<\/figure>\n\n\n\n

      We now need to create a new User Group and Assign them a policy that grants access to S3. Click on Create Group.<\/p>\n\n\n\n

      \"\"\/<\/figure>\n\n\n\n

      Select The checked permission policy as shown below.<\/p>\n\n\n\n

      \"upload<\/figure>\n\n\n\n

      We can create a new user and add them to the User Group with Full S3 access. Click on Add users and give your user a name.<\/p>\n\n\n\n

      \"upload<\/figure>\n\n\n\n
      \"upload<\/figure>\n\n\n\n

      You can skip the tags and go straight to click on Create user.<\/p>\n\n\n\n

      \"\"\/<\/figure>\n\n\n\n

      Server Side Application(NestJS)<\/strong><\/p>\n\n\n\n

      Code repository – https:\/\/github.com\/workfall\/multipart-upload-backend<\/a><\/p>\n\n\n\n

      Back on the server-side code (NestJS), we will use microservice architecture.<\/p>\n\n\n\n

      maint.ts<\/code><\/p>\n\n\n\n

      \"upload<\/figure>\n\n\n\n

      shared-service.service.ts <\/code><\/p>\n\n\n\n

      This is the part that aws-sdk<\/code> comes into play.<\/p>\n\n\n\n

      \"\"\/<\/figure>\n\n\n\n
      \"\"\/<\/figure>\n\n\n\n

      shared-service.controller.ts <\/code><\/p>\n\n\n\n

      \"upload<\/figure>\n\n\n\n

      Client Side Application (React)<\/strong><\/p>\n\n\n\n

      Code repository – https:\/\/github.com\/workfall\/multipart-upload-frontend<\/a><\/p>\n\n\n\n

      On the client side, we shall consume the endpoints we created and also most importantly determine the number of chunks a given file will have. Which will also be used to get pre-signed URLs for each part being uploaded.<\/p>\n\n\n\n

      After all the chunks have been uploaded we have to send a request to S3 to complete the upload of the file and put together the chunks to form one file. If not, S3 will assume that the upload is incomplete and you won\u2019t see your file in the AWS console.<\/p>\n\n\n\n

      FileUploadComponent.tsx <\/code><\/p>\n\n\n\n

      \"upload<\/figure>\n\n\n\n

      Uploader.tsx<\/code><\/p>\n\n\n\n

      This file has all the logic to deal with uploading the file. It gets the chunks of the file, gets a pre-signed URL for each part, and gives it a part number. The parts are sent as chunks using the pre-signed URL.<\/p>\n\n\n\n

      \"\"\/<\/figure>\n\n\n\n
      \"upload<\/figure>\n\n\n\n
      \"\"\/<\/figure>\n\n\n\n
      \"upload<\/figure>\n\n\n\n
      \"\"\/<\/figure>\n\n\n\n
      \"\"\/<\/figure>\n\n\n\n
      \"upload<\/figure>\n\n\n\n

      App.tsx<\/code><\/p>\n\n\n\n

      \"\"\/<\/figure>\n\n\n\n
      \"\"\/<\/figure>\n\n\n\n

      Conclusion<\/h2>\n\n\n\n

      In this blog, we looked at how to perform multi-part uploads to Amazon S3 using ReactJS frontend and NestJS backend. The multi-part upload method with pre-signed URLs is ideal for very large files and it is also secure because the pre-signed URL is usually short-lived. Summary:<\/p>\n\n\n\n