-
Notifications
You must be signed in to change notification settings - Fork 253
hdi_isfolder shall always be set in lower case on backend #3312
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR fixes a bug where duplicate hdi_isfolder metadata keys were being created on HNS (Hierarchical Namespace Storage) accounts. The issue occurred because Go canonicalizes the key as "Hdi_isfolder" when retrieving properties, but the backend always returns "hdi_isfolder" in lowercase during ListBlob API responses. During account-to-account copy or sync operations, AzCopy would set the camel-cased version, while the backend would also add the lowercase version, resulting in duplicate keys.
Key Changes:
- Normalized
hdi_isfoldermetadata key to always be set in lowercase on the backend - Removed conditional logic that could set either "Hdi_isfolder" or "hdi_isfolder"
- Ensured the camel-cased variant is deleted before setting the lowercase version
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
az storage blob list --account-name vikasfusename --container-name testcntgo1 --auth-mode login --include m --prefix "abcd" |
| } | ||
| } | ||
|
|
||
| // TODO (gapra): figure out better way to deal with hdi_isfolder metadata key capitalization |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should apply this to is_symlink too
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While copying symlinks we are creating the metadata with all lower case explicitly:
func (s *blobSymlinkSender) SendSymlink(linkData string) error {
err := s.getExtraProperties()
if err != nil {
return fmt.Errorf("when getting additional folder properties: %w", err)
}
s.metadataToApply["is_symlink"] = to.Ptr("true")
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In case of blob to blob copy with preserve-symlink flags here is my output:
GET https://vikasfuseblob.blob.core.windows.net/testcntgo1/abcd%2Fa.lnk?...
Accept: application/xml
User-Agent: AzCopy/10.32.0~preview.1 azsdk-go-azblob/v1.6.2 (go1.25.1; linux)
X-Ms-Client-Request-Id: 9602ecc6-11c6-468a-6ec4-373ccafae6e3
x-ms-version: 2025-07-05
--------------------------------------------------------------------------------
RESPONSE Status: 200 OK
...
X-Ms-Lease-Status: unlocked
X-Ms-Meta-Is_symlink: true
...
--------------------------------------------------------------------------------
PUT https://vikasfusename.blob.core.windows.net/testcntgo1/abcd%2Fa.lnk?...
...
x-ms-blob-type: BlockBlob
x-ms-meta-is_symlink: true
x-ms-version: 2025-07-05
...
Description
Customer ICM
Duplicate Hdi_isfolder keys in metadata of directories on HNS account.
Feature / Bug Fix:
When azcopy retreives properties from backend, Hdi_isfolder is camel cased by Golang. Later code (in case of account to account copy or sync) reuses the same and we end up adding "Hdi" in camel case while setting the properties on destination. When backend returns back the ListBlob API responce it add "hdi_isfolder" for all directories (all in lower case). This creates a duplicate key in metadata ("hdi" and "Hdi", one set by azcopy during sync and one added by backend during list on dfs account).
Related Links:
ICM
Type of Change
How Has This Been Tested?
This needs manual validation