Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GetObject returns NoSuchKey for objects with // in their path, even if they exist #5342

Closed
1 task
TrueAstralpirate opened this issue Jan 16, 2025 · 3 comments
Closed
1 task
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.

Comments

@TrueAstralpirate
Copy link

Describe the bug

I have a bucket on which s3cmd ls --recursive s3://mybucket/ returns

2025-01-10 13:31            0  s3://mybucket//a.txt
2025-01-10 13:33            0  s3://mybucket//b.txt
2025-01-16 07:49            0  s3://mybucket/abc//c.txt
2025-01-16 07:40            0  s3://mybucket/d.txt

When I try to use

client.GetObjectWithContext(ctx, &s3.GetObjectInput{
	Bucket: aws.String("mybucket"),
	Key:    aws.String("/a.txt"),
})

it returns error NoSuchKey, status code: 404.
Changing the blank name of a directory is not an option, so how do I use GetObject of such objects?
Key: aws.String("a.txt")
or
Key: aws.String("//a.txt")
don't work also.

Same with the b.txt and c.txt.
d.txt works fine, so problem is not with a bucket.

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

GetObject returns object on

client.GetObjectWithContext(ctx, &s3.GetObjectInput{
	Bucket: aws.String("mybucket"),
	Key:    aws.String("/a.txt"),
})

Current Behavior

GetObject returns 404 on

client.GetObjectWithContext(ctx, &s3.GetObjectInput{
	Bucket: aws.String("mybucket"),
	Key:    aws.String("/a.txt"),
})

Reproduction Steps

  1. Create a bucket with blank folders
  2. Try to call GetObject on such objects

Possible Solution

No response

Additional Information/Context

No response

SDK version used

v1.55.5

Environment details (Version of Go (go version)? OS name and version, etc.)

go1.23.1 darwin/arm64

@TrueAstralpirate TrueAstralpirate added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jan 16, 2025
@TrueAstralpirate
Copy link
Author

It seems like SDK trims these slashes, so the key changes from the one in the GetObject arguments unfortunately.

@lucix-aws
Copy link
Contributor

This is unfortunately a known wontfix - people have built up a dependency (often without even realizing) on this behavior over the years, we have to keep it on by default to preserve backwards compatibility.

You can shut off path normalization w/ https://pkg.go.dev/github.com/aws/aws-sdk-go/aws#Config.WithDisableRestProtocolURICleaning.

@lucix-aws lucix-aws closed this as not planned Won't fix, can't repro, duplicate, stale Jan 16, 2025
Copy link

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.
Projects
None yet
Development

No branches or pull requests

2 participants