-
Notifications
You must be signed in to change notification settings - Fork 253
Add support for AMLFS style posix metadata #3317
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds support for Azure Managed Lustre File System (AMLFS) style POSIX metadata properties, enabling compatibility with AMLFS when preserving POSIX properties during copy operations. The implementation introduces a new --posix-properties-style flag that allows users to specify whether to use 'standard' or 'amlfs' formatting for POSIX metadata attributes.
Key changes:
- Introduces
PosixPropertiesStyleenum type with Standard (default) and AMLFS variants - AMLFS style uses octal formatting for file permissions and a new timestamp format for modification times
- Updates metadata read/write logic to support both standard and AMLFS formatting styles
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| common/fe-ste-models.go | Defines new PosixPropertiesStyle enum with Standard and AMLFS variants, adds AMLFS timestamp format constant |
| common/unixStatAdapter.go | Updates metadata read/write functions to handle AMLFS-style owner/group/permissions/modtime formatting, adds logic to detect and parse octal permissions |
| common/unixStatAdapter_test.go | Adds test coverage for AMLFS-style metadata serialization and deserialization |
| ste/JobPartPlan.go | Increments data schema version to 20, adds PosixPropertiesStyle field to job part plan header |
| ste/mgr-JobPartTransferMgr.go | Threads PosixPropertiesStyle through transfer info structure |
| ste/sender-*.go | Updates all blob sender implementations to pass PosixPropertiesStyle to metadata functions |
| ste/xfer-anyToRemote-fileProperties.go | Code formatting cleanup (brace alignment) |
| common/rpc-models.go | Adds PosixPropertiesStyle field to copy job part order request |
| cmd/copy.go | Adds --posix-properties-style CLI flag with standard/amlfs options |
| cmd/copyEnumeratorInit.go | Propagates posixPropertiesStyle to job part order |
| cmd/copyValidation.go | Updates validation call to include posixPropertiesStyle parameter |
| azcopy/validationUtil.go | Adds validation to ensure --posix-properties-style is used with --preserve-posix-properties |
| azcopy/syncOptions.go | Adds posixPropertiesStyle field and updates validation for sync operations |
| azcopy/syncEnumerator.go | Propagates posixPropertiesStyle to sync job initialization |
| ste/md5Comparer.go | Adds author name to TODO comment (unrelated change) |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| s.accessTime = time.Unix(0, at) | ||
| } | ||
|
|
||
| // Always store ModTime in standard POSIX style |
Copilot
AI
Dec 9, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The comment is misleading. This code reads ModTime from metadata and attempts to parse it in either standard (Unix nanoseconds) or AMLFS (formatted string) style. A better comment would be:
// ModTime can be stored in either standard (nanoseconds) or AMLFS (formatted string) format| // Always store ModTime in standard POSIX style | |
| // ModTime can be stored in either standard (nanoseconds) or AMLFS (formatted string) format |
…rage-azcopy into wendi/amlfs-flag copy refactor.
…ity, and revert unrelated changes
Description
Added a new flag
--posix-properties-stylethat will toggle AMLFS style attributes. Specifically, mode is saved in octal and modTime is saved in a new Time format.Related Links
Type of Change
How Has This Been Tested?
Thank you for your contribution to AzCopy!