[bug] CI parallism needlessly running duplicate task

[dokterbob]

Describe the bug

Using parallelism in CI, I noticed that some jobs were needlessly duplicated twice.

Specific example: synergyai-nl/svelte-langgraph#49 (comment)

In this case, the job which takes the longest (frontend:install-playwright) is triggered by frontend:test-e2e, which (together with some other jobs) ends up being run twice, adding about 2m to overall build time.

Expected behavior

Jobs at the end of the graph (e.g. which are not depended on) do not run within the same job.

Or even better; the dependencies of jobs which are not dependent on are distributed based on their dependencies. E.g. frontend jobs would execute in one and backend jobs in another job.

Screenshots

Environment

GHA ubuntu-latest.

Additional context

Relevant tasks are in:

• moon.yml
• moon.yml

[milesj]

I wouldn't say this is a bug right now, just a missing feature. The distribution simply chunks all the targets into buckets, without taking any dependencies into account. It's very rudimentary.

[dokterbob]

I wouldn't say this is a bug right now, just a missing feature. The distribution simply chunks all the targets into buckets, without taking any dependencies into account. It's very rudimentary.

The bug is that there's overlap in the buckets. The dependency suggestion is more a nice to have. :)

[milesj]

There's overlap because of the dependencies.

If A and B both depend on C, and bucket 1 contains A/C, and bucket 2 contains B, then bucket 2 will also inherit C because it must run the dependency.

[dokterbob]

There's overlap because of the dependencies.

If A and B both depend on C, and bucket 1 contains A/C, and bucket 2 contains B, then bucket 2 will also inherit C because it must run the dependency.

That would be the most plausible explanation. But that's not what's happening.

Nothing depends on frontend:e2e, yet it gets executed twice. Depending frontend:install-playwright with it, loosing any advantage of parallelism in the process.

[milesj]

Paste the --log trace output.

[rhuanbarreto]

I have the same case of having to run the playwright container. The solution is to use the root project for this task and make all the project tasks to depend on the root task. They you don't duplicate as the reference is only one.