aws_s3_bucket_info needs ListBuckets permission even if a bucket name is given. #2183
Comments
abikouo
added
needs_info
|
That does work yes, thanks! I installed the collection from |
|
I also have a workaround that requires no bucket permissions / boto3 installation... (S3 redirects to the correct region for a bucket even without permissions, It also returns the correct region in a header, we use that here) - name: Check the correct region from the "wrong endpoint" error message
ansible.builtin.uri:
url: "https://s3.dualstack.us-east-1.amazonaws.com/{{ backup_s3_bucket }}"
follow_redirects: none
status_code:
- 301 # Wrong region
- 403 # Correct region
register: region_check_request
when:
- backup_s3_endpoint is not defined
- backup_s3_region is not defined
- backup_s3_bucket is defined
- backup_s3_bucket != ''
- name: Get correct region from header on response
ansible.builtin.set_fact:
backup_s3_region: "{{ region_check_request.x_amz_bucket_region }}"
when:
- backup_s3_endpoint is not defined
- backup_s3_region is not defined
- backup_s3_bucket is defined
- backup_s3_bucket != ''
``` |
In theory it would be possible to implement all of our modules without technically requiring boto3 (they're just calling an HTTP based API after all...). However, doing so would result in a massive amount of duplicate code. While I can see your use case, and I understand where you're coming from, there comes a point of diminishing returns. Similarly, when it comes to working around missing permissions on a bucket, I can see the value in making the list_buckets call optional (technically it's not superfluous, it's returning creation_date too), but hacking around missing |
I agree that that workaround does not make sense in the module, but it might be useful for someone else (and I have somewhere that I can find it was well in the future) (It was mainly my solution before the change to get rid of ListBuckets) (I currently install boto3 / botocore on the managed nodes just for this one step, so I'm thinking of keeping it) Edit: The updated version no longer need xmltodict (It uses a header instead of parsing the response) |
Summary
When using aws_s3_bucket_info with a supplied bucket name to determine the bucket location, an unnecessary ListBuckets call means that more permissions are needed than what is needed to determine a bucket's location.
get_bucket_listalways call list_buckets, even if a bucket name is given, which fails if the IAM role / user does not have ListBuckets permissions (but has sufficient permissions to get the requested info)(Yes, I'm likely on some unsupported versions, but the logic in the code will still have the same results on up to date versions)
Issue Type
Bug Report
Component Name
s3_bucket_info
Ansible Version
Collection Versions
AWS SDK versions
Configuration
(Versions retrieved from inventory host, since boto is needed there. AWS CLI was installed after the run, before the version check (to get the expected results)
OS / Environment
Ubuntu 20.04 on WSL1
Steps to Reproduce
Bucket policy:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "denyInsecureTransport", "Effect": "Deny", "Principal": "*", "Action": "s3:*", "Resource": [ "arn:aws:s3:::removed-backup-us-east-1-59c5daac/*", "arn:aws:s3:::removed-backup-us-east-1-59c5daac" ], "Condition": { "Bool": { "aws:SecureTransport": "false" } } }, { "Sid": "BucketAccess", "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::account-num:role/NodeRole" ] }, "Action": [ "s3:Put*", "s3:List*", "s3:Get*", "s3:Delete*" ], "Resource": [ "arn:aws:s3:::removed-backup-us-east-1-59c5daac/*", "arn:aws:s3:::removed-backup-us-east-1-59c5daac" ] } ] }Expected Results
AWS CLI running with the same credentails (the instance's role) (the region in the name is not always accurate)
The role does not have permission to use ListBuckets:
Actual Results
Code of Conduct
The text was updated successfully, but these errors were encountered: