[WIP] Copying large files from an Azure virtual machine to a local server is unstable and prone to failure during the process.

[Copilot]

Implementation Plan for Chunk-Level Resume Functionality

This addresses the issue of copying large files (3TB VHD) from Azure VM to local server with unstable network connections. The current resume functionality restarts entire transfers rather than resuming from completed chunks.

Current Status: Analysis Complete ✅

• Analyze current AzCopy architecture and resume functionality
• Understand chunk-based transfer mechanism and job plan structure
• Identify core issue: no chunk-level resume capability
• Design minimal-change solution approach

Implementation Plan:

• Step 1: Add chunk progress tracking to JobPartPlanTransfer structure

  • Add bit field or range list to track completed chunks
  • Ensure backward compatibility with existing job plans

• Step 2: Modify chunk completion logic to persist progress

  • Update chunk completion handlers to mark chunks as done
  • Ensure immediate persistence to job plan file

• Step 3: Update resume logic to skip completed chunks

  • Modify transfer scheduling to check completed chunks
  • Only schedule remaining chunks for transfer

• Step 4: Add large file optimizations

  • Environment variables for large file tuning
  • Improved chunk sizing for large transfers

• Step 5: Testing and validation

  • Create tests for chunk-level resume
  • Verify backward compatibility
  • Test with large file scenarios

Expected Outcome:

• True breakpoint resume: 3TB file that fails at 50% will resume from 50% instead of restarting
• Improved stability for large file transfers over unreliable networks
• Backward compatible with existing functionality

Fixes #2998.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.