How to Speed up CI Tests

[elizabethmcnish]

Hi there, :)

We use Pants Build in our monorepo, and now that our test suite has grown to a considerable size (100s), we are finding that the tests are taking up to 30 mins to complete in CI.

In CI, we use Github coupled with AWS EC2 runners to complete the tasks.

Having already implemented the Pants recommended optimisations to ensure tests are running efficiently in parallel, we now want to split the tests up into groups, and run each "group" of tests on a separate runner.

Of course, we can do this using additional args like --tag and --filter-address-regex, but are there any easier ways to do this without having to split the one GH workflow into separate GH workflows?

[sureshjoshi]

There are a handful of approaches, each of which applies case by case to your situation.

I think you're looking for sharding... I've never needed to try this, but you might also be able to use it alongside other target selection (filters, tags, etc).
https://www.pantsbuild.org/stable/reference/goals/test#shard
https://www.pantsbuild.org/blog/2022/09/01/introducing-pants-2-13#easy-parallel-execution-in-ci-with-test-sharding

Example from Pants CI:

pants/.github/workflows/test.yaml

Line 1033 in d250c80

./pants test --shard=0/10 ::

There may also be a world where you could use xdist for this: https://www.pantsbuild.org/stable/reference/targets/python_test#xdist_concurrency

Other suggestions for faster CI are the standard ones mostly... Cache more stuff, do less work, etc... Nothing revolutionary on those points.

You could also try approach 1 from this page, where you only run over transitively changed files. Your mileage may vary: https://www.pantsbuild.org/stable/docs/using-pants/using-pants-in-ci#approach-1-only-run-over-changed-files