Skip to content

Semi-regular ConcurrentModificationException: null error #6589

@jfy133

Description

@jfy133

Bug report

We are semi-regularly getting the following ConcurrentModificationException error on nf-core/createtaxdb. See the error and stack trace below.

It happens particularly regularly on GitHub actions, but also sometimes when running locally - particularly when using -resume after an interruption occurs.

The exact process where the error occurs does vary being mostly FIND_CONCATENATE_DNA, sometimes FIND_CONCATENATE_AA (this is the same module imported twice), and very occasionally MALT_BUILD.

Expected behavior and actual behavior

The pipeline runs through without crashing.

Steps to reproduce the problem

Unfortunately this appears to be too complicated to make a minimal reprex, I've tried playing around with various area of the code to see if anything changes but it doesn't appear so.

The fastest way to (maybe) reproduce is:

git clone [email protected]:nf-core/createtaxdb.git && mkdir -p createtaxdb/testing && cd createtaxdb/testing
nextflow run ../main.nf -profile test,docker --outdir ./results --dbname foo

You can try just running a few times to see if it comes up.

But I've found when running locally it's faster to encounter the error by forceably killing the run after a few processes have run, but FIND_CONCATENATE_DNA has not yet started, via ctrl + c.
Then run again but with -resume

nextflow run ../main.nf -profile test,docker --outdir ./results --dbname foo -resume

Program output

Nov-20 13:45:34.898 [Actor Thread 27] ERROR nextflow.processor.TaskProcessor - Error executing process > 'NFCORE_CREATETAXDB:CREATETAXDB:PREPROCESSING:FIND_CONCATENATE_DNA (1)'

Caused by:
  java.util.ConcurrentModificationException


java.util.ConcurrentModificationException: null
        at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1013)
        at java.base/java.util.ArrayList$Itr.next(ArrayList.java:967)
        at nextflow.processor.TaskInputResolver.normalizeInputToFiles(TaskInputResolver.groovy:110)
        at nextflow.processor.TaskInputResolver.resolve(TaskInputResolver.groovy:69)
        at org.codehaus.groovy.vmplugin.v8.IndyInterface.selectMethod(IndyInterface.java:355)
        at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
        at nextflow.processor.TaskProcessor.resolveTaskInputsV1(TaskProcessor.groovy:1912)
        at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
        at nextflow.processor.TaskProcessor.resolveTaskInputs(TaskProcessor.groovy:1851)
        at org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
        at nextflow.processor.TaskProcessor.invokeTask(TaskProcessor.groovy:670)
        at nextflow.processor.InvokeTaskAdapter.call(InvokeTaskAdapter.groovy:52)
        at groovyx.gpars.dataflow.operator.DataflowOperatorActor.startTask(DataflowOperatorActor.java:120)
        at groovyx.gpars.dataflow.operator.ForkingDataflowOperatorActor.access$001(ForkingDataflowOperatorActor.java:35)
        at groovyx.gpars.dataflow.operator.ForkingDataflowOperatorActor$1.run(ForkingDataflowOperatorActor.java:58)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
Nov-20 13:45:34.906 [Actor Thread 27] DEBUG nextflow.Session - Session aborted -- Cause: java.util.ConcurrentModificationException

Environment

  • Nextflow version: 25.10.0 build 10289
  • Java version: openjdk version "17.0.13-internal"
  • Operating system: Ubuntu 24.04.3 LTS
  • Bash version: GNU bash, version 5.2.21(1)-release (x86_64-pc-linux-gnu)

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions