nf-azure uses hardcoded blob.core.windows.net, cannot work with Azure China (*.chinacloudapi.cn)

[dongspy]

Hi,

I’m trying to run Nextflow with the nf-azure plugin on Azure China (21Vianet), using an Azure Batch account and an Azure Storage account that both live in the China cloud.

When I configure azure.storage.accountName to point to my storage account (for example mblob) and run a simple pipeline with azurebatch executor, I get the following error:

Nov-18 12:09:13.149 [main] ERROR c.a.c.h.netty.NettyAsyncHttpClient - java.net.UnknownHostException: mblob.blob.core.windows.net: Name or service not known

In Azure China, the storage endpoint is:

mblob.blob.core.chinacloudapi.cn

There is no mblob.blob.core.windows.net DNS entry in this environment, so the hostname lookup fails.

From my investigation:

• azure.batch.endpoint can be configured and works fine when I set it to
  https://<batch-account>.<region>.batch.chinacloudapi.cn

• However, for storage, nf-azure appears to assume the global Azure endpoint *.blob.core.windows.net and there is currently:

  • no azure.storage.endpoint / endpointSuffix / environment setting in nextflow.config
  • no documented way to override the storage endpoint or to tell nf-azure that it should use the Azure China cloud (*.chinacloudapi.cn)

As a result, the plugin always tries to talk to:

https://<account>.blob.core.windows.net

which does not exist in Azure China and leads to UnknownHostException.

---

Expected behavior

When running Nextflow with nf-azure against Azure China:

• nf-azure should be able to use the correct storage endpoint, e.g.
  https://<account>.blob.core.chinacloudapi.cn,
  and not hardcode blob.core.windows.net.

• Ideally there would be some configuration such as:

  • azure.storage.endpoint or
  • azure.storage.endpointSuffix or
  • azure.environment = 'AzureChinaCloud'

that allows users to switch the storage endpoint to the proper cloud (China, Gov, etc.).

[bentsherman]

Hi @dongspy , making the storage endpoint configurable seems reasonable. However I won't be able to test any fix since I'm not in China. Are you willing to test with a local build if I make a PR?