"Failed to call external services." error

[ThibauldMichel]

I am running the following command on a linux OS (Ubuntu 24.04.1 LTS) in order to download files from the SRA repository with ncbi-entrez-direct (19.2.20230331+dfsg-3ubuntu0.24.04.2).

project='PRJNA1005976[All Fields] AND rbcL[All Fields]'

output="/usr/projet_directory"

mkdir "$output"
cd "$output"
esearch -db sra -query "$project"  | efetch -format runinfo > runinfo.csv
cat runinfo.csv | tail -n +2 | cut -d "," -f 1 > SRR.numbers
cat SRR.numbers | parallel fastq-dump --split-files --origfmt --gzip {}

The output show either a successful download, or a "Failed to call external services." message.
It seems it is stochastic: the files successfully downloaded change each time the script is run.

Furthermore, the script can fail as well if run alone (without the use of the SRR.numbers list and of the parallel command).
I have tried to run the script with a delay, thinking the NCBI website could have a time restriction between two calls. It seems it is not the case as the following command is not working.

cat SRR.numbers | parallel --delay 3 "fastq-dump --split-files --origfmt --gzip {}"

Would have experienced with random issue before?

[klymenko]

Run prefetch for each run before running fastq-dump.

[ThibauldMichel]

Thank you very much for the quick fix!
I have tried it previously, but the code might has been incorrect as it did not work.
Here is how it worked for me:

project='PRJNA1005976[All Fields] AND rbcL[All Fields]'

output="/usr/projet_directory"

mkdir "$output"
cd "$output"
esearch -db sra -query "$project"  | efetch -format runinfo > runinfo.csv
cat runinfo.csv | tail -n +2 | cut -d "," -f 1 > SRR.numbers
cat SRR.numbers | parallel "prefetch {} && fastq-dump --split-files --origfmt --gzip {}"

[klymenko]

We cannot help you with use of parallel.