Description
As mentioned in #448, we at Loculus (https://github.com/loculus-project/loculus/actions) have consistently been getting errors using datasets since about 2024-02-04 - but only on the Hetzner servers we run our previews on. Note: We can't reproduce locally. Our servers make a fair number of requests, so it's possible we've been rate limited or blocked - however without getting a proper 429 http status code.
We noticed that from this server, requests to api.ncbi.nlm.nih.gov time out:
curl -v api.ncbi.nlm.nih.gov
* Host api.ncbi.nlm.nih.gov:80 was resolved.
* IPv6: 2607:f220:41e:4290::110
* IPv4: 130.14.29.110
* Trying [2607:f220:41e:4290::110]:80...
* Immediate connect fail for 2607:f220:41e:4290::110: Network is unreachable
* Trying 130.14.29.110:80...
* connect to 130.14.29.110 port 80 from 172.17.0.2 port 35892 failed: Connection timed out
* Failed to connect to api.ncbi.nlm.nih.gov port 80 after 129534 ms: Could not connect to server
* closing connection #0
curl: (28) Failed to connect to api.ncbi.nlm.nih.gov port 80 after 129534 ms: Could not connect to server
I'm just opening this issue for general awareness, and to help other datasets users who might be getting the same error.
I'm aware of the banner on NCBI websites and will email the help desk:
Some NLM-NCBI services and products are experiencing heavy traffic, which may affect performance and availability. We apologize for the inconvenience and appreciate your patience. For assistance, please contact our Help Desk at [email protected].
For debugging purposes, our requests should be coming from 65.108.0.35.