{"id":876,"date":"2022-02-22T11:41:50","date_gmt":"2022-02-22T11:41:50","guid":{"rendered":"https:\/\/www.workfall.com\/learning\/blog\/?p=876"},"modified":"2025-08-19T12:01:49","modified_gmt":"2025-08-19T12:01:49","slug":"replicate-existing-s3-objects-using-s3-batch-replication","status":"publish","type":"post","link":"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/","title":{"rendered":"How to easily replicate existing S3 objects using S3 batch replication?"},"content":{"rendered":"<span class=\"rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Reading Time: <\/span> <span class=\"rt-time\">10<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span>\n<p><img src=\"https:\/\/lh6.googleusercontent.com\/0Wmb54tBuyG2UPLp-1Y_wnczNbBlroPpMozEJYLmf8DtiPTa95xCiBbvQ1QSIpE36M3-Kn43IBfUAmYP6eZfgVPBfXHFste7ffFXAC2lCg-HfqLxciChL_3jUGclRA\" style=\"width: 1600px;\"><\/p>\n\n\n\n<p class=\"has-text-align-justify\">For successful corporate operations, data access is essential. However, data is frequently blocked or corrupted due to device problems, cyberattacks, and natural disasters. Businesses and IT specialists are forced to work hours to recreate and recover data that has been destroyed. Data replication is the best strategy to protect data and assure reliable data access. Having a replica can also help enterprises with a large number of locations access data more quickly.<\/p>\n\n\n\n<p class=\"has-text-align-justify\">Recently, AWS announced a new feature for S3 Batch Replication which comes in handy in this situation. It makes replicating existing data from a source bucket to one or more destinations simple. You can use this feature to replicate an unlimited number of objects in a single job.<\/p>\n\n\n\n<p class=\"has-text-align-justify\">In this blog, we will explore <a href=\"https:\/\/docs.aws.amazon.com\/AmazonS3\/latest\/userguide\/s3-batch-replication-batch.html\" target=\"_blank\" rel=\"noreferrer noopener\">S3 Batch Replication<\/a>, its use cases, when to use it, how to use S3 Batch Replication and its pricing model. We will also demonstrate step-by-step instructions on how to replicate existing objects using S3 Batch Replication.<\/p>\n\n\n\n<p>In this blog, we will cover:<\/p>\n\n\n\n<ul><li>What is S3 Batch Replication?<\/li><li>Cross Region Replication VS Same Region Replication<\/li><li>How does it work?<\/li><li>Use Cases of S3 Batch Replication<\/li><li>Why Use S3 Batch Replication?<\/li><li>When to use S3 Batch Replication?<\/li><li>Pricing of S3 Batch Replication<\/li><li>Hands-on<\/li><li>Conclusion<\/li><\/ul>\n\n\n\n<h2><strong>What is S3 Batch Replication?<\/strong><\/h2>\n\n\n\n<p class=\"has-text-align-justify\">Amazon Simple Storage Service (S3) Replication is an elastic, fully managed, inexpensive technology that replicates objects between buckets.&nbsp; S3 Replication provides the highest flexibility and functionality in cloud storage, enabling you to address your data sovereignty and other business requirements.<\/p>\n\n\n\n<p class=\"has-text-align-justify\">You can use Amazon S3 Replication to automatically replicate S3 objects across different AWS Regions using S3 Cross-Region Replication (CRR) or between buckets in the same AWS Region using S3 Same-Region Replication (SRR).<\/p>\n\n\n\n<p>It allows you to replicate data to multiple destination buckets in the same AWS Region or in other AWS Regions.&nbsp;<\/p>\n\n\n\n<p class=\"has-text-align-justify\">The metrics and notifications provided by S3 Replication allow you to keep a close watch on the replication process.&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td><strong>Then<\/strong><\/td><td><strong>Now<\/strong><\/td><\/tr><tr><td>S3 batch replication replicates newly created objects to 2 or more AWS buckets within the same AWS account and different AWS accounts.<\/td><td>S3 batch replication now replicates existing objects to 2 or more buckets within the same AWS account and different AWS accounts.<\/td><\/tr><tr><td>Backfill data to newly formed buckets with newly uploaded objects.<\/td><td>Backfill data to newly formed buckets with existing objects.<\/td><\/tr><tr><td>Cannot retry objects that failed to copy to the destination.<\/td><td>Can retry objects that failed to copy to the destination.<\/td><\/tr><tr><td>Objects were replicated with the same account between different regions.<\/td><td>Now, Objects can be replicated cross-accounts as well.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h2><strong>Cross Region Replication(CRR) VS Same Region Replication(SRR)<\/strong><\/h2>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh6.googleusercontent.com\/rzx58UYeMR5o-B4i9zw8jZklqEEO6fPuaxd4rkdgSoKtwwrbR29GzIL5pCgHjMlfkxakpkc07Y14IFAsvlqADDzZPcBuhXJeXZS8_3tbCfjSDoV0DGsnYfk4Qq9M7w\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-justify\"><strong>Cross-Region Replication<\/strong> and <strong>Same Region Replication<\/strong>, both allow you to replicate data at a bucket level, a shared prefix level, or an object level using S3 object tags. Moreover, both these features allow you to alter account ownership for the object replicas to safeguard data from any accidental deletion. However, there are a few key fundamental differences to make note of.<\/p>\n\n\n\n<h2><strong>How does it work?<\/strong><\/h2>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/xbeB7bgv3mhN5fPVk1naLodrjW1HzbXg_-Rejt35MSnn_dgTUVfNHuXTMTq571-t0JSKBImPQ-w7WTVYG0i6EFpttvh-F662LF7ELGGWBnQGwL3si_h1klkr7KYThA\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<h2><strong>Use cases of S3 Batch Replication<\/strong><\/h2>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/NFX_ZfYGKT43l0890mof0tH0cXeSYz66LOxD-qOeoNSU1EC89YIeB_tJyvuATkoHYUP5tC89ryTOjeWyu_Z6KLdzaV5vIT3KwhZuEw_QCJUHDG_Oy2I5AH4raDZdUQ\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-justify\">Complemented by CRR and SRR, S3 Batch Replication can handle any size of data and provides a fully managed solution for data sovereignty and compliance, disaster recovery, and performance improvement. You can begin with S3 Batch Replication with only a few clicks in the S3 console or a single API request.<\/p>\n\n\n\n<ul><li><strong>Backfill newly created buckets<\/strong> \u2014If a new multi-region storage initiative demands, you can use Batch Replication to set up new buckets and backfill them with existing objects from another bucket.<\/li><li><strong>Retry Application<\/strong> &#8211; You can use Batch Replication to retry replication for a variety of reasons, failure to replicate objects, initially, when objects replicated in one destination need to be replicated elsewhere, or when replicating replica objects from another source<\/li><li><strong>Migration <\/strong>&#8211; Batch replication can help you with the migration of existing objects across buckets, irrespective of the Region or the account they belong to, without affecting metadata and version ID.<\/li><\/ul>\n\n\n\n<h2><strong>Why use S3 Batch Replication?<\/strong><\/h2>\n\n\n\n<ul><li>For disaster recovery purposes, customers may choose to duplicate their data to a new AWS Region. To do so, they&#8217;ll need to migrate existing data into the new destination bucket.<\/li><li>Data replication for globally expanding companies. Consider the case of a UK-based animation business that has recently opened a new studio in Singapore. To reduce latency for their employees, they will be required to duplicate all internal files and in-progress media files to the APAC (Singapore) Region.<\/li><li>Customers experiencing mergers and acquisitions are obligated to hand over ownership of present data from one AWS account to another.<\/li><li>Customers end up implementing sophisticated methods to replicate existing objects between buckets. Furthermore, duplicating objects between buckets do not maintain object metadata like version ID and object creation time.<\/li><\/ul>\n\n\n\n<p class=\"has-text-align-justify\">S3 Batch Replication proposes a new potential through S3 Batch Operations that eliminates the need for customers to brainstorm solutions for replicating existing objects between buckets. It offers an easy way to copy present objects from a source bucket to multiple destinations. This capability empowers you to replicate any number of objects with a single job.<\/p>\n\n\n\n<h2><strong>When to use S3 Batch Replication?<\/strong><\/h2>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/yJF4b9QO45kFzjsTYb6SCoiAg3DOjm5nwCw5ZVHd3E0jG9VfikvyaY6FqjdZuLPRmuNvuHQOYsFbvN3OTWnNjyZ20Yx6WbZ1Io-mKn2yfXGnOu2D-aMNQTpO8hvXrg\" alt=\"When to use S3 Batch Replication?\"\/><\/figure>\n\n\n\n<ul><li><strong>Replicate existing objects: <\/strong>use S3 Batch Replication to replicate objects that were included in the bucket before the replication rules were set.<\/li><li><strong>Replicate objects that previously failed to replicate: <\/strong>retry replicating objects with the S3 Replication rules that could not be replicated before, owing to factors like insufficient permissions.<\/li><li><strong>Replicate objects that have already been replicated to a different destination: <\/strong>you may need to keep numerous copies of your data in different AWS accounts or Regions. S3 Batch Replication can replicate previously replicated objects to new destinations.<\/li><li><strong>Replicate replicas of objects generated by a replication rule: <\/strong>S3 Replication produces replicas of objects in destination buckets. Object replicas cannot be replicated via live replication. S3 Batch Replication alone can replicate these replica objects.<\/li><\/ul>\n\n\n\n<h2><strong>Pricing of S3 Batch Replication<\/strong><\/h2>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh5.googleusercontent.com\/4i1Bwjbhccft9lZmE1nGBjKr2KsFcJiVfEuyn0kCw_Hw-ob0AkzFtjfg3jD-YNjqBcguNO6FvCmVsnypvSk0YinTDG0hsf2v3ACzE0-n0NncBBI21zNJwulfOThFCw\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<ul><li><strong>S3 Data transfer cost:<\/strong> pricing includes cross-region data transfer OUT from S3 within AWS.<\/li><li><strong>S3 Storage cost:<\/strong> pricing includes storage of objects in the destination S3 bucket.<\/li><li><strong>Batch Operations job cost: <\/strong>pricing per batch operation job in a particular region is $0.25<\/li><li><strong>Batch Operations manifest cost:<\/strong> pricing is $0.015 per 1 million objects in the source bucket.<\/li><li><strong>Batch Operations object cost:<\/strong> pricing for 1 million objects processed is $1.00.<\/li><\/ul>\n\n\n\n<h2><strong>Hands On<\/strong><\/h2>\n\n\n\n<p class=\"has-text-align-justify\">Let\u2019s do S3 batch replication. We will be using S3 and IAM services in this demonstration. First, we have to create a source bucket. You can give it a unique name as we have given \u201c<strong>workfallbucket<\/strong>\u201d. Upload some objects into the bucket as shown below.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/lwS7QcJ9f-_KS0ra0HCxKrsjW1eAo1IHNONMIpLr9CHuGsGvNDMftG8rAI_N275ON4-iSyNH_1xYWoGNPyASpNm0k8159-fc5QWybWUBx7Vt1vaKedU-0oQsG0LWE9z8ND5Os5Mq\" alt=\"\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-justify\">Click on the bucket that you just created and navigate to the <strong>management <\/strong>tab. For creating S3 batch replication, we first have to set up the replication rule in the source bucket. For creating the replication rule, remember to enable the versioning on both the source and destination buckets.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/tc_NySYOXmmU44Amj7aaJi6r7DGmsUsbKSiAtd-iYXDJESbK9o4K1uL3-qUybIy1zTVMOhS00NiMDRki7lqn4pklCyufloO3BTqpZAiJBrPi-9s_MtpmyC-474Qx2-Cm89HTKLy8\" alt=\"\"\/><\/figure>\n\n\n\n<p>As you will scroll down a little, you will see Replication rules there. Click on create replication rule.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/hNUtxHf-S3O2UbqBcTumB9Hcc4Qu01iOB5-L5cTju47lFTbdfP3962sZxhyeE_WLCmd5sPIaIioXV_hcLPffvkYb8D0AEtM8CONpSPzzaZRH_FfUKr2DiJP-36a7SCoE6EWA3P3g\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<p>You will be redirected to the new window to create the replication rule.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/InLqXzzta-EFHTgAAHMvKTXvgUecdHIyHKf-GsPyvBoK1VCr8rQIsA6n2TNZBKX0begEYbxa9JOgY0nS9pOsJaBa9JW1m5W2rXxKgbq6qf7Z2FtjDxNBBDEYq4pv-Rk57yTa7sDI\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<p>Provide a unique name to the replication rule. Keep the status for the rule as enabled.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh6.googleusercontent.com\/ge6aX0eRWu447mt9b3gqiuj0FTreERIgx9gXd-Dsnu4iAxetScxJYQOO_6QSzba0W5hhAHkqaLWEhyVGARd7CyWvq-VA3B0LBUDgwP6PcE7k0MBVSi7ufia_t92xQ7PSSlsjAnsw\" alt=\"\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-justify\">For choosing the rule scope, select Apply to all objects in the bucket. If you choose \u201c<strong>Limit the scope of this rule<\/strong>\u201d, you have to provide the prefix to filter the objects in the bucket.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh5.googleusercontent.com\/1bQePE5xxMji-PKtyzS7jaxh8SJaiVz-qit_WF6uPbKYkXGAlalfKgixpUC_BUCGE-YmTb4dBy3jTi_H59PsDAyDm9PB1J9g3wemxwTPTGjmrzZVvDwNydzeVVD9ypX2jjY6m6YR\" alt=\"\"\/><\/figure>\n\n\n\n<p>For Destination, create another bucket with a unique name as we have given\u201d<strong>destinationbucketworkfall<\/strong>\u201d.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh5.googleusercontent.com\/Lq1OkMXNGdiLAYkobu01HxBDwWFan6TMh4cX1cBxMhRJG7f5DwAtFnN0TXtDpDtqkS_AKKqJRsw6Wd2jtWqDXMhOvs8IvV1kEpYt5At5-_CFodFrgogmKLG2S9TJ5l5HEiIkapUB\" alt=\"\"\/><\/figure>\n\n\n\n<p>Select the bucket name from the dropdown as shown below.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh6.googleusercontent.com\/kIIHIXgnUhjrCFoy2nRyVvlagwRTSbpWCrv3GNU6EqjBPthowBP-Yuh_CmvcuLIWUsBERjV9p3gMqVFkiignpor-yB63cKRvCmofRyyvBsb88cf30ag6XGhdmxIhiu3iUi9mbRxH\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<p>Select the destination bucket.<\/p>\n\n\n\n<p>Note: Create the source and destination buckets in different regions.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/FVJ3A5osg4XGgFtGrTDcoqdyd6Gz3vkZ18_kzBvhkPFnQheV_131MzVEsWcWkdP3WFy0E1KQOpUjiOiHGKXdmzjg2j7ZyMfAhZqjjqr7eZU_cajF62Nnziqc4fn99GIDBry5uIM1\" alt=\"\"\/><\/figure>\n\n\n\n<p>Create the IAM role for S3 replication \u201c<strong>S3_Replication_Role_for_Workfallbucket<\/strong>\u201d.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/rtjY-fp47VbhBAARkg05xiK4StBOHbxyZ7dfxdbinf4GBIrQEYWxGtaXig5HuCvYba-5ErmvIcwzs6sMYaPUPWxrVeq2l6wMoM0-QutjW0ihMMHWosMy0OIdZMP0WJEoyWh0KTI0\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<p>Go to IAM. Create a policy with the below configuration. Name it as <strong>S3_Replication_Policy<\/strong>.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/Yz7QiHY6H7dYzCY_kufaU2pkMNwHmgXBLgiT1CMhMrv9xve8o7jT-f2k9MsLKXnijMxiaaGmKBz4MR2-2_JXeaxF4paPapqYKH98olr3bBrmTe_UnLpRh8fcIPKas3Q73JCUpV88\" alt=\"\"\/><\/figure>\n\n\n\n<p>Click on create role. Select S3 from services.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh5.googleusercontent.com\/0Q8pJsYdmMRNgrNtu8dl2jTP5SdspUXKXHbAnQL-ZauZ1z_fhFGCknogk9MO0ihPizzzloQQgKFuAbdaAmOj6l1IagTQgwn6kh8PmKei-7UP5DqAPeRrGk4jMghQsgSO7gCyt7uC\" alt=\"\"\/><\/figure>\n\n\n\n<p>Now, select the policy you just created in the <strong>Add permissions<\/strong> page.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/TEH9iFjuHDOLNvthweFVDp2p39IyP_vG4FF2BOAMHV_bJ9v7iSIOIxfOg2L5qpVjlktrv39ON_-AsRP1emtIBE1sZoJ0b7ytOLLu1b6mrkpilebCbTKmi4lF6q24uEmTZ46GbAOU\" alt=\"\"\/><\/figure>\n\n\n\n<p>Select <strong>S3_Replication_Policy<\/strong> as shown below.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh5.googleusercontent.com\/5WSIto0n41mKOZLhYSJbmRdH3lInqLKHVZUX51__842SssqwZsv9dUKMGfPkMnd91z0XUDSl5zPeJ__AZgpE3Y8ln6_81rq9S2Z3ONWLCFAzaZ_yMhvl_Xba3pALHPhfYfEMDUr6\" alt=\"\"\/><\/figure>\n\n\n\n<p>Here you can review your role and click on create.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/bBSMWWeBs2JGolHAecGRq5nu3_QHq-xjtlB7TwWlfZE3g1JkZzhhuP-AjyNjb1VSkDe5rzOjB8xDLDzuWLovxoN-qTOgEhtyl6ZurO0NJdz5vPciq2xThYjZPCwjV5X0KeG9skYU\" alt=\"\"\/><\/figure>\n\n\n\n<p>Click on create role. The role is created successfully.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/MHs3UwrqpV6JP6vW72Df5wshYUYoK5bIwdtoCMfiNmEa4RWow8GA3Mw0GGLazTc980Bt4Z-_d-qWzPDkJSJbC5qEoeff1I6ZbCyjR7f0JbuZyWsAOo914_FE2LU3yT7IYdNcM2r3\" alt=\"\"\/><\/figure>\n\n\n\n<p>Choose the role from the dropdown as shown below.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/3sN9PQ3xyMr1KD2_zw55o-IZm_gI4QpR3i3f3XYZLRwKkAx2QDPlWZW23yy6E2wZ6T2x6D4DQOZ6CU17i86Hzr5WGJEKKrOTceIb_jT7ctivXPs7MpeHR71I5zj0HUUnuprE3u16\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<p>For this implementation, there is no need for Encryption. You can leave it unchecked.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh6.googleusercontent.com\/q2KkJYCnHij4t34B6cvkxMqniIXkAoP5zImb-rx-vKkNflUGLCmq3IDj4MZJB0GMkAF9UbBywnf7_X8IMOpdVhetzWcFRmH97pXmb-iWzxOMcvsR29Wh78kLFRUV6XTbIRi7SKt8\" alt=\"\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-justify\">For the Destination storage class, you can leave it blank for this implementation. Using the Destination storage class, you can change the storage class for replicated objects.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/2ByJuu1FcbD7Z266SNF5uNOwKBTUoy0vQXV1FKcKyIrihFX9mfTMdyjJTGNDi_Doc8MIeK2AKwcHyh5iYNrqloBchJK_UVBzaYjw1Cs33xgg_dIwjnHC432Eu8RaBkKKRDCovhEm\" alt=\"\"\/><\/figure>\n\n\n\n<p>Leave this unattended..<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/qZn8n77fAVgLg5r-XDjeBRiUxxl0-A8_8rrbNDMb6Hf6loCIxznjh7FLHSsRj6uMBBl_ydLdt-wUnIr56_1jVYIMy-pwVlaHIcMLcVjsJTzP-5tw_tsZ1YxglHMvOWgi_H7M-pFh\" alt=\"\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-justify\">Click on save. We will receive a pop-up with the message \u201c<strong>Replicate existing objects<\/strong> \u201c. You can replicate the existing objects of the source bucket using this.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh6.googleusercontent.com\/TS1llJXJ5FzlOHhpvsMFWN_pkWf0_YrV9sHXdQL4akM8eqroKMwvK-voifQHmh4OCtAMNyEdGWS3PpTmp9tRjfgiDzr6bpfW9ol8IGFi1v2AuI-fnLLJ3IOFuHGnvFGhmBOgsO3z\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<p>Click on \u201c<strong>Yes, replicate existing objects<\/strong>\u201d and click on Submit.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/tOdItspJWjpiY-ikWVerjd76ARfokJYXDtzk-kRwCNxY-LHPYTTkqJShO64U-qYofmeuNvHRu5XsCpSkZak7ge5VfWBZiwGoHQ19t4-_QHKQw9pCSH-vwGf05NJND1p6uSLhjzsc\" alt=\"\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-justify\">As you will click on submit. You will be redirected to the Batch Operations job page. Now we will create the Batch operations job.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/b6-jRecFjyBq17PpUG8Fz8V3ghD2mZpCnXxkWWE4FO8MLItMz4FVYq-uVfHtqpaCGvd8sLgom8Yb5hgPzU5PuPkEB-_r6e_3zpkMMbRz3ecg-ThxWB_aBEeev0TOnhB-bM8gEEY4\" alt=\"\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-justify\">Select \u201c<strong>Wait to run the job when it\u2019s ready<\/strong>\u201d. Using this you can manually run the job and if you want to review the job details or manifest file, you can do that before running the job.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh6.googleusercontent.com\/TuDvmFPmmIsUJuMAeZQviSlxM1RbM3EjRpjteizdF7vJsA0s_yiIKtFFeY3CHDbLkXQia786qPXFQBjJ7jVKKzRHzvlPoIjUwYGwPKxrHel470TUIfFRkVegZuGuiWBO1oAC9vC9\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<p>Select \u201c<strong>Save Batch Operations manifest<\/strong>\u201d. Provide the destination where you want your manifest to be saved.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/VLpKctw5TmBvFQHQ73CV4AS5Jf2jyHA7hBx5DM_4X1XeigP83xhPQliBiquW799poQk8CT6_xvUX9H8AEqv-LjDpOABbZvK8-D3wr7knOumg-XopQdOlAH5BxWSl56q7FUqbpIBS\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<p>Specify the folder within the bucket where you want your manifest to be saved.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/SgN-JSMj7XWpXx9meOXLtp594Xo2LUnw13P2gJ7a-Q4WS2YaXjD28yHx-1ahhWk5tzOdERQFiWEQD-j9ZC0XO5elnlTlzjhcnNU_UBKtoCC5zA-ct7wXFRELZZaUpETogUCTebj3\" alt=\"\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-justify\">In the completion report section, provide the path where your completion report will be saved. Specify the folder within the bucket for the exact location.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh6.googleusercontent.com\/-jfMu_ulzA_U5iTA9Y8ycHagYGCUhwK1dc0TKk_VxM8jMRvOyr5VBPZSgd7ZWcDYHYwDCDGw-kqjHXG4gJnT8WEEKwrlU9Gs_CuIr5ysrewaz_MglduAdJKW1Z69SQ0Ihlttt9NN\" alt=\"\"\/><\/figure>\n\n\n\n<p>For the S3 batch operations job, you have to create the S3 batch operation role.<\/p>\n\n\n\n<p>The image below shows the creation of the S3 batch operations policy.<\/p>\n\n\n\n<p class=\"has-text-align-justify\">Mention the following permissions in the <strong>S3_BatchOperations_Policy<\/strong>. Provide the source bucket ARN and manifest and completion report bucket ARNs.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/VEmnWWjmjCGax8a-1V-aV-mkVZQ7Otu4HiAVbPQKplF0IopmgW4z3uRoQfQFSO5W3mjb0csLAL90vZkESUaVJOXCZroFLBx3WWnYo1CTg0ERlWTduowTF-N9D-i_tuq-LivMEnyE\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<p>Click on create policy as shown below.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/2NwCfXhVUNZHlisoRLtSP6g0-36kV-x9-n_F3r-qh_rIdCnxQYoDDLg9o44itKohTn_00M2V34j09Ip-ytfO4bRnDNbg1ZxebvId_btQI5YCq4xv5oOLyjW4DCKVZIE4p4MNHJv1\" alt=\"\"\/><\/figure>\n\n\n\n<p>Create a role for S3 batch operation.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/D0Hnp2KAkr2WjqG-egRlqO2A92T8QepNIeo2IcRL2hHyhUSwhuVZ8rzxUzZEgVkEhcziRLhqz493HLY6BulFXIjpyYN1TfOSD-cRPt2YtoK5MmKuq08XSRBBZ_6czd5K72PMu5Pc\" alt=\"\"\/><\/figure>\n\n\n\n<p>Provide the IAM role that you just created during batch job creation and click on Create job.<\/p>\n\n\n\n<p>You can see the job has been created successfully. The status of the batch job is in the <strong>preparing <\/strong>state.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh5.googleusercontent.com\/hWfBXLpIUiBKOoaE98twKqSChOE7c70PcsO89gwQq4cwA0gNDjzCEzmlDcZYKr8Yv2d8-WUqOM59XFytDZGy_OYoEvEvmdMmwIXYNNoCUpz14qROerkKXoDdXGHOdrPHreTATuue\" alt=\"\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-justify\">Now the status has been changed to \u201c<strong>Awaiting your confirmation to run<\/strong>\u201d. Here you can see the total number of objects that would be replicated is 5. There is a unique job id that is associated with every job.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh6.googleusercontent.com\/B0c3wQ_hcCIg6PQmnk0DkHKjSYvAYGLCPPmhxngxhrZiiWu1BI1ifxEK-YREi6387MG1y4KWUi8e-7kJ9F8zTRF_7gF7UOBkg1Gh4om6wdwvp5ffW_YAIbBM1BiSnQ9C0BdizrGf\" alt=\"\"\/><\/figure>\n\n\n\n<p>Select the job that is just created and click on \u201c<strong>Run job<\/strong>\u201d.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/GjhZ_Tucyw35ylQ8xn8c0x5EcZPl4vIB2xq589hrcN8phHYAdWySirHMF1e9M3RK1PQAmGRdjEebR1CccdGmzXcqAy_F2tqJMl7VdaDbHSG4y8h7KCnYxmqkhp9QikKF6NhctO0K\" alt=\"\"\/><\/figure>\n\n\n\n<p>You will be redirected to the page where you can review the job details.&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh5.googleusercontent.com\/R5Y3l4ARQMuCic3mDDOGJexLJQo7jNdsxzCumanrcgUE0psMZfHZxERUQV0gXCOh6Ak7_7DTHzoYqwGFL7IwRpveGwLlbmpmZi21YnSwMeQsjh50nnaPoDFTDFGyffwnUOyLdTVA\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/MvmRkQEho9Qm_2Us2MJQMAvDwmOfhVYhpjZWOlLC9CH0cQXaCQa74Tacfc_tI1faZgn7-etMgs7mdEnESqz-KjOfQmZEdRpln4Tt5T02h342C3mhjr-gO0JHwI9zzqNqOHv5ldj3\" alt=\"\"\/><\/figure>\n\n\n\n<p>After you&#8217;ve gone over everything, go to the bottom of the page and select &#8220;Run job.&#8221;<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh3.googleusercontent.com\/OcZhhcHS5RWJ6L5lBYWmiejZCM7k1cN8SXpBJ-TKsDmhKP9NZpgp2HvRy3bOD4SmHeyQ3erTOa8zl0wKqLM3NJ9QvQfTatNHzbOzNp2Qevw5AMh7aeIDP9CVivzSQuVciSZccSE8\" alt=\"\"\/><\/figure>\n\n\n\n<p>Now you can see the status of your job as <strong>Ready<\/strong>.&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/wuHtTdHvar_Ltm6rJ5AjeVXxLc8DLOmV-EEndoISOceVPp4eynimwNQR9chIiAL40b80ImbK7PrFPvf-_FukRQlh-ua-gz0LRZqx_BaAw_HKJwOm4wE1wZna0ut2a9BR3va1WwgV\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<p>Now the job gets completed. All the objects in the source bucket are replicated to the destination bucket.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh4.googleusercontent.com\/zFXAoUauqH8RNPKvpw4iegIB-3ozPN82cl93wVoEojg6eEsQBfZq9IsrDTz85GP1rh_KZUugZ8oZ8UuPSnnzk2AOGZaA-vMni3DiKouOS4B-XmBL8ipIfOWdrKX0s5GkyELERrZi\" alt=\"\"\/><\/figure>\n\n\n\n<p>When you open your destination bucket, you will see the objects as shown below.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/lh5.googleusercontent.com\/M5EQZAfABnUxohmtuXeBqVNMEXUGJFNTCygNkByfl2uyaIgZ0-QsUdpyO59K_xE2CctEDIZchN8TfBI94w-Ps1TsxgCVkSUsaqVAPuFhUJXsMVWJJW4zlr39_NQhNwTFaRLPct9A\" alt=\"How to easily replicate existing S3 objects using S3 batch replication?\"\/><\/figure>\n\n\n\n<p class=\"has-text-align-justify\">By following this step-by-step implementation you\u2019ll be able to perform the S3 Batch Replication successfully. You can try performing the replication for the failed jobs objects and also newly uploaded objects or replicating objects to new buckets cross-account or in the same-account.<\/p>\n\n\n\n<h2><strong>Conclusion<\/strong><\/h2>\n\n\n\n<p class=\"has-text-align-justify\">In this blog, we have explored Cross-Region Replication vs Same Region Replication, S3 Batch Replication, its use cases, when to use it, how to use S3 Batch Replication, and its pricing module. It allows you to replicate data to multiple destination buckets in the same AWS Region or in other AWS Regions. It also backs two-way replication between two or more buckets in the same or different AWS Regions. We also demonstrated step-by-step instructions on how to replicate existing objects using S3 Batch Replication. Stay tuned to keep getting all updates about our upcoming new blogs on AWS and relevant technologies.&nbsp;<\/p>\n\n\n\n<p>Meanwhile \u2026<\/p>\n\n\n\n<p><strong>Keep Exploring -&gt; Keep Learning -&gt; Keep Mastering<\/strong><\/p>\n\n\n\n<p class=\"has-text-align-justify\">This blog is part of our effort towards building a knowledgeable and kick-ass tech community. At <a href=\"https:\/\/www.workfall.com\/\">Workfall<\/a>, we strive to provide the best tech and pay opportunities to AWS-certified talents. If you\u2019re looking to work with global clients, build kick-ass products while making big bucks doing so, give it a shot at<a href=\"https:\/\/www.workfall.com\/partner\/\"> workfall.com\/partner<\/a> today.<\/p>\n","protected":false},"excerpt":{"rendered":"<p><span class=\"rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Reading Time: <\/span> <span class=\"rt-time\">10<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span> For successful corporate operations, data access is essential. However, data is frequently blocked or corrupted due to device problems, cyberattacks, and natural disasters. Businesses and IT specialists are forced to work hours to recreate and recover data that has been destroyed. Data replication is the best strategy to protect data and assure reliable data access. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":877,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"spay_email":""},"categories":[2],"tags":[3,257,141,258,6],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v19.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>How to easily replicate existing S3 objects using S3 batch replication? - The Workfall Blog<\/title>\n<meta name=\"description\" content=\"Replicate existing data from a source bucket to one or more destinations with AWS&#039;s new feature for S3 Batch Replication.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to easily replicate existing S3 objects using S3 batch replication? - The Workfall Blog\" \/>\n<meta property=\"og:description\" content=\"Replicate existing data from a source bucket to one or more destinations with AWS&#039;s new feature for S3 Batch Replication.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/\" \/>\n<meta property=\"og:site_name\" content=\"The Workfall Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/facebook.com\/workfall\" \/>\n<meta property=\"article:published_time\" content=\"2022-02-22T11:41:50+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-08-19T12:01:49+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2022\/02\/Cover-Images_Part2-3.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"627\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@workfall\" \/>\n<meta name=\"twitter:site\" content=\"@workfall\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Workfall\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"18 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/#organization\",\"name\":\"Workfall - Hire #Kickass Coders On Demand\",\"url\":\"https:\/\/learning.workfall.com\/learning\/blog\/\",\"sameAs\":[\"https:\/\/www.instagram.com\/workfall\/\",\"https:\/\/www.linkedin.com\/company\/workfall\/\",\"https:\/\/facebook.com\/workfall\",\"https:\/\/twitter.com\/workfall\"],\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/i1.wp.com\/18.141.20.153\/learning\/blog\/wp-content\/uploads\/2021\/10\/cropped-WF_logo.png?fit=400%2C400\",\"contentUrl\":\"https:\/\/i1.wp.com\/18.141.20.153\/learning\/blog\/wp-content\/uploads\/2021\/10\/cropped-WF_logo.png?fit=400%2C400\",\"width\":400,\"height\":400,\"caption\":\"Workfall - Hire #Kickass Coders On Demand\"},\"image\":{\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/#website\",\"url\":\"https:\/\/learning.workfall.com\/learning\/blog\/\",\"name\":\"The Workfall Blog\",\"description\":\"#Tech #Remote #Jobs\",\"publisher\":{\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/learning.workfall.com\/learning\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#primaryimage\",\"url\":\"https:\/\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2022\/02\/Cover-Images_Part2-3.png\",\"contentUrl\":\"https:\/\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2022\/02\/Cover-Images_Part2-3.png\",\"width\":1200,\"height\":627,\"caption\":\"Amazon S3 Existing Objects Replication - AWS\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#webpage\",\"url\":\"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/\",\"name\":\"How to easily replicate existing S3 objects using S3 batch replication? - The Workfall Blog\",\"isPartOf\":{\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#primaryimage\"},\"datePublished\":\"2022-02-22T11:41:50+00:00\",\"dateModified\":\"2025-08-19T12:01:49+00:00\",\"description\":\"Replicate existing data from a source bucket to one or more destinations with AWS's new feature for S3 Batch Replication.\",\"breadcrumb\":{\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/learning.workfall.com\/learning\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How to easily replicate existing S3 objects using S3 batch replication?\"}]},{\"@type\":\"Article\",\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#webpage\"},\"author\":{\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/#\/schema\/person\/cab8236044692bc5b27606b13167794a\"},\"headline\":\"How to easily replicate existing S3 objects using S3 batch replication?\",\"datePublished\":\"2022-02-22T11:41:50+00:00\",\"dateModified\":\"2025-08-19T12:01:49+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#webpage\"},\"wordCount\":1983,\"publisher\":{\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2022\/02\/Cover-Images_Part2-3.png\",\"keywords\":[\"AWS\",\"datareplication\",\"s3\",\"S3batchreplication\",\"workfall\"],\"articleSection\":[\"AWS Cloud Computing\"],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/#\/schema\/person\/cab8236044692bc5b27606b13167794a\",\"name\":\"Workfall\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/learning.workfall.com\/learning\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2023\/09\/avatar_user_1_1693914404-96x96.png\",\"contentUrl\":\"https:\/\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2023\/09\/avatar_user_1_1693914404-96x96.png\",\"caption\":\"Workfall\"},\"sameAs\":[\"https:\/\/www.workfall.com\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How to easily replicate existing S3 objects using S3 batch replication? - The Workfall Blog","description":"Replicate existing data from a source bucket to one or more destinations with AWS's new feature for S3 Batch Replication.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/","og_locale":"en_US","og_type":"article","og_title":"How to easily replicate existing S3 objects using S3 batch replication? - The Workfall Blog","og_description":"Replicate existing data from a source bucket to one or more destinations with AWS's new feature for S3 Batch Replication.","og_url":"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/","og_site_name":"The Workfall Blog","article_publisher":"https:\/\/facebook.com\/workfall","article_published_time":"2022-02-22T11:41:50+00:00","article_modified_time":"2025-08-19T12:01:49+00:00","og_image":[{"width":1200,"height":627,"url":"https:\/\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2022\/02\/Cover-Images_Part2-3.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_creator":"@workfall","twitter_site":"@workfall","twitter_misc":{"Written by":"Workfall","Est. reading time":"18 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Organization","@id":"https:\/\/learning.workfall.com\/learning\/blog\/#organization","name":"Workfall - Hire #Kickass Coders On Demand","url":"https:\/\/learning.workfall.com\/learning\/blog\/","sameAs":["https:\/\/www.instagram.com\/workfall\/","https:\/\/www.linkedin.com\/company\/workfall\/","https:\/\/facebook.com\/workfall","https:\/\/twitter.com\/workfall"],"logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/learning.workfall.com\/learning\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/i1.wp.com\/18.141.20.153\/learning\/blog\/wp-content\/uploads\/2021\/10\/cropped-WF_logo.png?fit=400%2C400","contentUrl":"https:\/\/i1.wp.com\/18.141.20.153\/learning\/blog\/wp-content\/uploads\/2021\/10\/cropped-WF_logo.png?fit=400%2C400","width":400,"height":400,"caption":"Workfall - Hire #Kickass Coders On Demand"},"image":{"@id":"https:\/\/learning.workfall.com\/learning\/blog\/#\/schema\/logo\/image\/"}},{"@type":"WebSite","@id":"https:\/\/learning.workfall.com\/learning\/blog\/#website","url":"https:\/\/learning.workfall.com\/learning\/blog\/","name":"The Workfall Blog","description":"#Tech #Remote #Jobs","publisher":{"@id":"https:\/\/learning.workfall.com\/learning\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/learning.workfall.com\/learning\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#primaryimage","url":"https:\/\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2022\/02\/Cover-Images_Part2-3.png","contentUrl":"https:\/\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2022\/02\/Cover-Images_Part2-3.png","width":1200,"height":627,"caption":"Amazon S3 Existing Objects Replication - AWS"},{"@type":"WebPage","@id":"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#webpage","url":"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/","name":"How to easily replicate existing S3 objects using S3 batch replication? - The Workfall Blog","isPartOf":{"@id":"https:\/\/learning.workfall.com\/learning\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#primaryimage"},"datePublished":"2022-02-22T11:41:50+00:00","dateModified":"2025-08-19T12:01:49+00:00","description":"Replicate existing data from a source bucket to one or more destinations with AWS's new feature for S3 Batch Replication.","breadcrumb":{"@id":"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/learning.workfall.com\/learning\/blog\/"},{"@type":"ListItem","position":2,"name":"How to easily replicate existing S3 objects using S3 batch replication?"}]},{"@type":"Article","@id":"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#article","isPartOf":{"@id":"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#webpage"},"author":{"@id":"https:\/\/learning.workfall.com\/learning\/blog\/#\/schema\/person\/cab8236044692bc5b27606b13167794a"},"headline":"How to easily replicate existing S3 objects using S3 batch replication?","datePublished":"2022-02-22T11:41:50+00:00","dateModified":"2025-08-19T12:01:49+00:00","mainEntityOfPage":{"@id":"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#webpage"},"wordCount":1983,"publisher":{"@id":"https:\/\/learning.workfall.com\/learning\/blog\/#organization"},"image":{"@id":"https:\/\/learning.workfall.com\/learning\/blog\/replicate-existing-s3-objects-using-s3-batch-replication\/#primaryimage"},"thumbnailUrl":"https:\/\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2022\/02\/Cover-Images_Part2-3.png","keywords":["AWS","datareplication","s3","S3batchreplication","workfall"],"articleSection":["AWS Cloud Computing"],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/learning.workfall.com\/learning\/blog\/#\/schema\/person\/cab8236044692bc5b27606b13167794a","name":"Workfall","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/learning.workfall.com\/learning\/blog\/#\/schema\/person\/image\/","url":"https:\/\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2023\/09\/avatar_user_1_1693914404-96x96.png","contentUrl":"https:\/\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2023\/09\/avatar_user_1_1693914404-96x96.png","caption":"Workfall"},"sameAs":["https:\/\/www.workfall.com"]}]}},"jetpack_featured_media_url":"https:\/\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2022\/02\/Cover-Images_Part2-3.png","jetpack-related-posts":[{"id":453,"url":"https:\/\/learning.workfall.com\/learning\/blog\/create-vaults-in-aws-s3-glacier-to-store-and-archive-data\/","url_meta":{"origin":876,"position":0},"title":"How to create vaults in AWS S3 Glacier to store and archive data?","date":"November 9, 2021","format":false,"excerpt":"We are in the data age, so along with data creation and retrieval, storing and archiving data is equally important. For ages, storing and archiving data is always costly! We generally overpay for data archiving because we are forced to make an expensive upfront payment for archiving solutions. Since we\u2026","rel":"","context":"In &quot;AWS Cloud Computing&quot;","img":{"alt_text":"AWS Glacier","src":"https:\/\/i0.wp.com\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2021\/11\/Glacier.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":614,"url":"https:\/\/learning.workfall.com\/learning\/blog\/how-to-do-database-migration-using-aws-database-migration-service-dms-from-on-premise-ec2-to-rds\/","url_meta":{"origin":876,"position":1},"title":"AWS DMS: Migrating On-Premise\/EC2 Databases to RDS Made Simple","date":"November 11, 2021","format":false,"excerpt":"Do you want to migrate your on-premise databases to AWS with minimal or no downtime? In today\u2019s fast-paced world,\u00a024\/7 application and database availability are the basic criteria to consider. In this case, some of your applications may not be able to afford significant downtime while migrating on-premises databases. In this\u2026","rel":"","context":"In &quot;AWS Cloud Computing&quot;","img":{"alt_text":"Database Migration using AWS Database Migration Service (DMS)","src":"https:\/\/i0.wp.com\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2021\/11\/DMS.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":1498,"url":"https:\/\/learning.workfall.com\/learning\/blog\/how-to-etl-api-data-to-aws-s3-bucket-using-apache-airflow\/","url_meta":{"origin":876,"position":2},"title":"How to ETL API data to AWS S3 Bucket using Apache Airflow?","date":"November 1, 2022","format":false,"excerpt":"2.5 quintillion bytes of data are produced every day with 90% of it generated solely in the last 2 years (Source: Forbes). Data is pulled, cleaned, transfigured & then presented for analytical purposes & put to use in thousands of applications to fulfill consumer needs & more. While generating insights\u2026","rel":"","context":"In &quot;AWS Cloud Computing&quot;","img":{"alt_text":"How to ETL API data to AWS S3 Bucket using Apache Airflow?","src":"https:\/\/i0.wp.com\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2022\/11\/Cover-Images_Part2-2.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":1513,"url":"https:\/\/learning.workfall.com\/learning\/blog\/how-to-upload-large-files-1gb-and-beyond-to-aws-s3-using-nestjs-backend-and-reactjs-frontend\/","url_meta":{"origin":876,"position":3},"title":"How to upload large files (1GB and beyond) to AWS S3 using NestJS (backend) and ReactJS (frontend)?","date":"November 15, 2022","format":false,"excerpt":"When dealing with file uploads, you must be aware that files are uploaded in buffers stored in memory and if the file is larger than the allocated memory in your VM, it may run out of memory and the application might crash. For example, if your allocated memory is 2GB,\u2026","rel":"","context":"In &quot;AWS Cloud Computing&quot;","img":{"alt_text":"How to upload large files (1GB and beyond) to AWS S3 using NestJS (backend) and ReactJS (frontend)?","src":"https:\/\/i1.wp.com\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2022\/11\/Cover-Images_Part2-3.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":419,"url":"https:\/\/learning.workfall.com\/learning\/blog\/aws-s3-storage-lens-analytics-solution-for-organization-wide-visibility\/","url_meta":{"origin":876,"position":4},"title":"AWS S3 Storage Lens \u2014 Analytics Solution For Organization-Wide Visibility","date":"November 3, 2021","format":false,"excerpt":"S3 Storage Lens is a recently launched service of AWS for S3 which provides S3 storage analytics and insights. S3 storage lens is also the first-ever cloud storage analytics solution initially introduced by AWS. The S3 storage lens increases object storage visibility into S3 buckets. When you are storing objects\u2026","rel":"","context":"In &quot;AWS Cloud Computing&quot;","img":{"alt_text":"AWS S3 Storage Lens \u2014 Analytics Solution For Organization-Wide Visibility","src":"https:\/\/i1.wp.com\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2021\/11\/s3lens.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":897,"url":"https:\/\/learning.workfall.com\/learning\/blog\/how-to-fetch-contents-of-json-files-stored-in-amazon-s3-using-express-js-and-aws-sdk\/","url_meta":{"origin":876,"position":5},"title":"How to fetch contents of JSON files stored in Amazon S3 using Express.js and AWS SDK?","date":"April 5, 2022","format":false,"excerpt":"A plethora of banks employ JSON for data transfer between the bank and the client servers. With the help of Amazon S3 data lakes, the banks use the data to train AI\/ML models and perform advanced analytics. JSON is an open standard file format and data interchange format that stores\u2026","rel":"","context":"In &quot;AWS Cloud Computing&quot;","img":{"alt_text":"File Storage in Amazon S3 using Express.js & AWS SDK","src":"https:\/\/i0.wp.com\/learning.workfall.com\/learning\/blog\/wp-content\/uploads\/2022\/04\/Cover-Images_Part2.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"https:\/\/learning.workfall.com\/learning\/blog\/wp-json\/wp\/v2\/posts\/876"}],"collection":[{"href":"https:\/\/learning.workfall.com\/learning\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/learning.workfall.com\/learning\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/learning.workfall.com\/learning\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/learning.workfall.com\/learning\/blog\/wp-json\/wp\/v2\/comments?post=876"}],"version-history":[{"count":5,"href":"https:\/\/learning.workfall.com\/learning\/blog\/wp-json\/wp\/v2\/posts\/876\/revisions"}],"predecessor-version":[{"id":1768,"href":"https:\/\/learning.workfall.com\/learning\/blog\/wp-json\/wp\/v2\/posts\/876\/revisions\/1768"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/learning.workfall.com\/learning\/blog\/wp-json\/wp\/v2\/media\/877"}],"wp:attachment":[{"href":"https:\/\/learning.workfall.com\/learning\/blog\/wp-json\/wp\/v2\/media?parent=876"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/learning.workfall.com\/learning\/blog\/wp-json\/wp\/v2\/categories?post=876"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/learning.workfall.com\/learning\/blog\/wp-json\/wp\/v2\/tags?post=876"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}