Published on 2011-12-20 by John Collins. Please follow me on Twitter for more:
Recently I had to download a large number of files from an Amazon S3 bucket, and found that I could not carry out a bulk download operation via the AWS Management Console web application, which was a little disappointing to say the least.
Luckily, someone has developed a 3rd party application for carrying out operations like this on S3 called Bucket Explorer. I have used this to download over seventeen thousand files from my S3 bucket in a few hours, so I am very happy with it.
One word of warning though: it gobbles up a lot of memory. On my work station with 8GB of RAM, it used up 2.5GB of RAM (in a JVM session) while downloading my files for a few hours, so you probably should be using a powerful machine to run this program.
You can download a 30-day trail of Bucket Explorer from their website here: