Merged StorageMover NFS Phase 1 changes with azcopy main's NFS symlink support changes #3291
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add standard blob symlink tests * panics * catch nil body * Fix issues with missing logs or pprof * One wild goose chase later Don't no-op if the POSIX SIP isn't implemented
* fix version text in current dir * empty dir check * version checker test * moved init out of version check * fix comment --------- Co-authored-by: Gauri Lamunion <[email protected]>
* Add missing --exclude-path flag to azcopy remove command Co-authored-by: gapra-msft <[email protected]> * e2e remove test on files & blob * comment * use Blob loc, rename blobs * add folders to expected list * remove repeat testing --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: gapra-msft <[email protected]>
* Concurrency value bug fix with test case * Add test case * Added TC * Added TC
* clean up stale files, decide when to warn * fix lint * added test * fix lint * address comments * address comments * address comments * rename * more detail in info msg * message * more details to info msgs * string * address comments * small comment * fix lint
* only allow explicit version checking * remove VC from e2e tests * added test for check-version flag * remove file * e2e + unit test * removed unit test, ensure backward compat * line endings diff
* Correct incorrect folder tracker behavior * Invert test for thoroughness * Fix tests
* Implement request priority policy * remove unused * comments/lint
* redact * more tests --------- Co-authored-by: Gauri Lamunion <[email protected]>
* Fix symlink direct targets * lint * Undo accidental test edit * Helper Methods * Error out on lone symlink, expand testing
* Added properly error message for system to system container copy * Added properly error message for system to system container copy * Added properly error message for system to system container copy * Added properly error message for system to system container copy * Update common/util.go Co-authored-by: Gauri Lamunion <[email protected]> * Added E2E test for system container copy scenario * added a check to see if the containername is encoded * Added E2E test * Added E2E test * Added E2E test * Added E2E test * Added E2E test * incorporated review comments * Introduce flags for NFS over REST support (#2968) * Added flags description (#2994) * Added support for copy/sync from Local linux <-> Azure Files NFS and Azure Files NFS <-> Azure Files NFS (#2987) * Fixed golangci-lint error * Add the --nfs flag check for download scenario (#3022) * resolved conflicts * Add NFS tests to E2E framework (#3036) * Default handling for hardlink for Azure Files NFS (#3033) * Add default permissions preservation logic if --preserve-permissions is set to false (#3047) * Default handling for symlink,hardlink and special files for Azure Files NFS - Phase-1 (#3046) * fixed golint error * fixed golint * Update hardlink flag name and check the sudo permission for the process (#3078) * Update ChangeLog for 10.30.0~preview.1 release * Update flag description and error message * Update ChangeLog for 10.30.0~preview.1 release * Update flag description and error message * Update ChangeLog.md Co-authored-by: Vikas Bhansali <[email protected]> * Throw error and fail the job if destination share is not present for SMB and NFS both (S2S Copy) (#3099) * Remove --nfs flag and add NFS combinations in --from-to flag (#3102) * fixed golangci-lint errora * fixed golangci-lint errors * updates help message for NFS support (#3132) * Add support for transfer data Azure Files NFS <-> Azure Files SMB (#3087) * Clean up places where we call location.FileSMB || location.FileNFS (#3212) * Add symlink and special file skipped count to job progress log (#3218) * Symlink support for Local to Azure Files NFS (#3201) * Symlink support for NFS <-> NFS, NFS -> Local and SMB<->NFS (#3238) * Introduce FileSMB for non File s2s or upload/download paths (#3211) * while downloading with --preserve-permissions=false dont set the defaults for uid/gid (#3246) --------- Co-authored-by: wonwuakpa-msft <[email protected]> Co-authored-by: Gauri Lamunion <[email protected]> Co-authored-by: Vikas Bhansali <[email protected]>
* Catch EADDRNOTAVAILABLE * Merge linux and darwin helpers * Header
* remove decoding from special chars * checkpoint * removed flag, default to NO decoding for all rm commands * remove object content validation * fix type for validation * remove dry run check
* Refactor platform retry policy * Apply suggestion from @Copilot Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Gauri Lamunion <[email protected]>
* Add initial index.html for redirecting to GitHub wiki * Remove outdated website * Apply suggestion from @Copilot Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: wonwuakpa-msft <[email protected]> Co-authored-by: Gauri Lamunion <[email protected]> Co-authored-by: Copilot <[email protected]>
…ink support from main
bewurman-msft
requested review from
otendolkar,
samskan and
shnayak-msft
as code owners
…ounts for E2E tests when no account key is provided
…ded new func DefaultSyncOrchestratorTestMode to fix tests that rely on it but were not properly assigned it
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Merged changes from mover/stage2 with changes from main (specifically commit ID d698076) that add support for following symlinks for NFS -> NFS Azure Fileshare migrations.
This PR is being published early so any merge conflicts and any potential conflicting behavior that weren't flagged as merge conflicts (e.g. both branches added to a function in different places) can be reviewed while testing is ongoing.
Type of Change
How Has This Been Tested?
Testing is ongoing
Thank you for your contribution to AzCopy!