Delete files in S3 older than a certain date


#1

In a non-production (predev) environment, due to manually blowing away queue entries, etc, I have a bunch of orphaned files that I need to get rid of. I am not too sure how many there are, it could be many thousands of files. I am hesitant to do S3.listFiles or countFiles with -1 argument in case it kills the server, so I want to run it for 1,000 files at a time (takes about 2 minutes to delete 1,000 files, S3 is just that way). I want to compare the date on the file to a specific point in time and if the file was created prior to that date, nuke it!


#2

credit to Alistair Atkinson for this solution:

var referenceDate = new DateTime(“2018-07-06T00:00:00”)
S3.listFiles(‘s3://c3–origin-predev-input/stream/inbox/CanonicalWellStreamMeasurement/’, 1000).files.forEach(function(f) {
if(f.lastModified.isBefore(referenceDate)) {
f.delete();
}
});