AzCopy 10.30.1 is still failing with nil pointer dereference #3219
Description
Which version of the AzCopy was used?
10.30.1
I am using from this endpoint: https://github.com/Azure/azure-storage-azcopy/releases/download/v10.30.1/azcopy_linux_amd64_10.30.1.tar.gz
Which platform are you using? (ex: Windows, Mac, Linux)
Linux (ubuntu 22.04)
What command did you run?
azcopy copy "${INPUT_PATH}" /mnt/datadisk/ --recursive
What problem was encountered?
I encountered the same issue addressed by #3186 even after updating to 10.30.1 and confirming the version with azcopy --version.
Here is the error message:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x11606e7]
goroutine 401 [running]:
github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/file.FormatDownloadStreamResponse(_, _)
/home/cloudtest/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/storage/[email protected]/file/responses.go:248 +0x307
github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/file.(*Client).DownloadStream(_, {_, _}, _)
/home/cloudtest/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/storage/[email protected]/file/client.go:558 +0x1ae
github.com/Azure/azure-storage-azcopy/v10/ste.(*blobFSDownloader).GenerateDownloadFunc.func1()
/mnt/vss/_work/1/s/ste/downloader-blobFS.go:87 +0x1fe
github.com/Azure/azure-storage-azcopy/v10/ste.(*blobFSDownloader).GenerateDownloadFunc.createDownloadChunkFunc.createChunkFunc.func2(0x5f5e100?)
/mnt/vss/_work/1/s/ste/sender.go:207 +0x294
github.com/Azure/azure-storage-azcopy/v10/ste.(*jobMgr).chunkProcessor(0xc0002eac08, 0x9e)
/mnt/vss/_work/1/s/ste/mgr-JobMgr.go:1055 +0xab
created by github.com/Azure/azure-storage-azcopy/v10/ste.(*jobMgr).poolSizer in goroutine 105
/mnt/vss/_work/1/s/ste/mgr-JobMgr.go:957 +0x236
How can we reproduce the problem in the simplest way?
Similar to existing issue, transferring many files over a span of time reproduces this at various stages in the process. Sometimes low percent, sometimes high, and sometimes it succeeds.
Have you found a mitigation/solution?
I managed to copy my files after retrying many times but this instability makes my automation pipeline unreliable and thus unusable. Would love guidance on triage.