Granting S3File Permissions


#1

I have a file that I’ve exported to S3 using a MetricResultExportJob. I need to apply the AWS ACL bucket-owner-full-control to that file after it’s been exported.

I’m trying to do the following, resulting in the listed error:

var file = S3.openFile({url: "s3://path-to-some-bucket/somefile.parquet”})
file.upsertGrants([{permission: AwsS3GrantPermission.FULL_CONTROL}])

"AwsS3Client exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: 7FE34863231EDD9B; S3 Extended Request ID: vv6/YBjebik9ifx90X9nWAYCtQcIy/jtwRi12lnke9RS7hXO6d1gdpI3GVToSBxP9HQeBkoSza4=)
errorCode="NoSuchKey"; errorType="Client"; requestId="7FE34863231EDD9B"; serviceName="Amazon S3"; httpStatusCode="404""

What do I need to do differently here? My end-goal is to apply the bucket-owner-full-control permission to an uploaded file (ref: https://aws.amazon.com/premiumsupport/knowledge-center/s3-bucket-owner-access/).


#2

this looks like a problem with the file url, are you sure it’s correct? (can you run file.readString())


#3

Just sloppy editing on my part I think, the updated url is a full path to the target file and I can readString() successfully.


#4

is there a way to apply the “AWS ACL bucket-owner-full-control” directly? or do we need the Canonical AWS ID of the bucket owner? just running file.upsertGrants([{permission: AwsS3GrantPermission.FULL_CONTROL}]) does not work.


#5

this error was a bug in v7.8 and fixed in v7.8.1


#6

There is a ticket to address this enhancement (bucket ownership on write). Should be part of 7.8.1.someReleaseLargerThan1000.

We are waiting for this to test this, so we can cut Lamdba out of the loop and reduce the number of vulnerabilities (by writing directly to a customer bucket location in a way they are able to read the file).

I’ll update this thread once we have verified with successful testing and put the minimum build here for reference.


#7

Example to set grants for S3 File in v7.8:

var file = S3File.createFile(“pathToFile”)
var grants = file.grants()
grants.push(AwsS3Grant.make({permission: AwsS3GrantPermission.FULL_CONTROL, canonicalGrantee: AwsS3GrantCanonicalGrantee.make({displayName: “bucket_owner”, id: “canonicalAwsId”})}))
file.upsertGrants(grants)


#8

@jonathonbraun @akatkinson What was the final result here? We are experiencing a similar issue running on 7.8.2.482. We have been tasked with writing to a customer S3. I can create a file, but attempting to call write gets a permission error, and calling file.grants() gives a nosuchkey error. Is there an accepted way to set these permissions? We just need to add --acl bucket-owner-full-control, essentially.


#9

You need to create the file and then upsert the grants like the snippet above


#10

@jgodbey-c3iot This is a function we used to set the FULL_CONTROL permission on a file:

function setGrantsFullControl(url) {
  var canonicalId = TenantConfig.configStr("Exports-S3Bucket-CanonicalID");
  var permissions = [{ permission: AwsS3GrantPermission.FULL_CONTROL, canonicalGrantee: { id: canonicalId } }];
  S3File.make({ url: url }).upsertGrants(permissions);
}

Basically, because we were the creator of this file, it was ok for us to set the grants in this way (potentially overwriting other existing grants). However, to preserve existing grants, you are correct to call file.grants(), and I’ve no idea why you see that error, sorry.