-
Notifications
You must be signed in to change notification settings - Fork 249
Description
Summary
Add native support for SharePoint Online Document Libraries as both a source and a destination in AzCopy.
This would enable efficient, direct, cloud-to-cloud data transfers between SharePoint Online and Azure Data Lake Storage Gen2 (ADLS), eliminating the need for local staging or complex workarounds.
Current Limitation
AzCopy v10 currently only supports Azure Storage endpoints (Blob, ADLS Gen2, Files, Tables).
For organizations that need to move data between Microsoft 365 (SharePoint) and Azure (ADLS), the lack of direct integration creates unnecessary friction:
Users must download files locally (via OneDrive sync or manual download) and then upload to ADLS.
Workarounds require Azure Data Factory (ADF) + Microsoft Graph API or Logic Apps, which add cost, latency, and complexity.
There is no efficient, native, cloud-to-cloud migration path for SharePoint ↔ Azure Storage.
Proposed Solution
Extend AzCopy to support SharePoint Online document libraries as both sources and destinations by leveraging Microsoft Graph API and Entra ID (Azure AD) OAuth2 authentication.
Example Scenarios
SharePoint → ADLS (Source)
azcopy copy "https://contoso.sharepoint.com/sites/Finance/Shared%20Documents/Reports/" `
"https://mystorageaccount.dfs.core.windows.net/datalake/raw/finance/reports" `
--recursive
ADLS → SharePoint (Destination)
azcopy copy "https://mystorageaccount.dfs.core.windows.net/datalake/outbound/reports/*" `
"https://contoso.sharepoint.com/sites/Finance/Shared%20Documents/Reports/" `
--recursive
Required Capabilities
- Authenticate with Entra ID / OAuth2 (same as AzCopy already does for ADLS).
- Traverse folders recursively.
- Handle modern SharePoint sites and libraries.
- Preserve metadata (timestamps, filenames).
- Support both bulk one-time migrations and incremental syncs.
Business Justification
Enterprises often store operational, financial, and compliance data in SharePoint but need to land it in ADLS for analytics, machine learning, and archiving.
Today, this requires staging locally or using expensive/complex pipelines (ADF, Logic Apps, third-party tools).
Direct support in AzCopy would:
- Cut migration time from hours/days to minutes.
- Reduce infrastructure and storage costs by eliminating local staging.
- Simplify cloud-to-cloud workflows for hybrid M365 + Azure use cases.
- Provide a consistent, scriptable, DevOps-friendly toolchain.
Pain Points Solved
Inefficiency: Current methods require two hops (SharePoint → Local → ADLS).
Complexity: ADF/Graph API setups require engineering expertise for what should be a simple transfer.
Performance: Local network bandwidth becomes the bottleneck instead of cloud-to-cloud speeds.
Cost: Organizations spend money on third-party migration tools or extra Azure services just to move files.
Benefits
Enable direct, secure, high-performance migrations between SharePoint and ADLS.
Standardize on AzCopy for all enterprise file transfer scenarios (not just Azure Storage).
Improve data agility by reducing barriers between Microsoft 365 collaboration and Azure analytics.
In short: Supporting SharePoint Online document libraries in AzCopy would unlock a highly-requested, business-critical workflow: efficient, scriptable, cloud-to-cloud data movement between Microsoft 365 and Azure.