[Question] Add children to Flow at runtime

[gillesgw]

Hello,

I’m trying to solve the following problem:

I receive JSON messages from a Redis stream. The first message corresponds to an initialization job that I must run first. Then, there are a certain number of messages that trigger content generation jobs in the database. These jobs can be executed as they arrive from the stream (it doesn’t matter if some finish before others).

Finally, the last message triggers an “end” job, which must only be executed after all the generation jobs are finished. This “end” job must therefore wait for all the generation jobs created beforehand.

I’m struggling to make this work while using as many of BullMQ’s native features as possible. FlowProducer seems to require knowing all the children in advance… I also tried using the { parent: { id, queue } } option but without success.

What would be the best solution to solve this problem?

Thanks