Flaky unit tests

[MikeMcC399]

Current behavior

When unit tests are run in CircleCI they are sometimes failing

(When run locally, running unit tests modifies the contents of the committed file data/database.json.)

Desired behavior

Unit tests should reliably pass when running in CircleCI.

Test code to reproduce

yarn test:unit:ci

Debug Logs

Failing test https://app.circleci.com/pipelines/github/cypress-io/cypress-realworld-app/6408/workflows/b31dcc2f-3bd8-4ad9-9f5b-5d8924473524/jobs/40315 (Run Unit Tests)

yarn run v1.22.22
$ vitest --run

 RUN  v3.2.4 /root/project

 ✓ src/__tests__/bankaccounts.test.ts (4 tests) 61ms
 ✓ src/__tests__/users.test.ts (5 tests) 34ms
 ✓ src/__tests__/notifications.test.ts (7 tests) 462ms
 ✓ src/__tests__/likes.test.ts (2 tests) 117ms
 ↓ src/__tests__/generateSeedData.test.ts (8 tests | 8 skipped)
 ✓ src/utils/__tests__/transactionUtils.test.ts (6 tests) 28ms
 ✓ src/__tests__/contacts.test.ts (5 tests) 27ms
 ✓ src/__tests__/transactions.test.ts (15 tests | 2 skipped) 325ms

⎯⎯⎯⎯⎯⎯ Failed Suites 1 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  src/__tests__/comments.test.ts [ src/__tests__/comments.test.ts ]
SyntaxError: Malformed JSON in file: /root/project/data/database.json
Unterminated string in JSON at position 270336 (line 8005 column 20)
 ❯ FileSync.read node_modules/lowdb/adapters/FileSync.js:37:30
     35|           var data = readFile(this.source, 'utf-8').trim();
     36|           // Handle blank file
     37|           return data ? this.deserialize(data) : this.defaultValue;
       |                              ^
     38|         } catch (e) {
     39|           if (e instanceof SyntaxError) {
 ❯ LodashWrapper.db.read node_modules/lowdb/lib/main.js:32:21
 ❯ module.exports node_modules/lowdb/lib/main.js:51:13
 ❯ backend/database.ts:92:12
 ❯ src/__tests__/comments.test.ts:2:1

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯


 Test Files  1 failed | 7 passed | 1 skipped (9)
      Tests  42 passed | 10 skipped (52)
   Start at  14:14:17
   Duration  6.87s (transform 4.90s, setup 12.16s, collect 27.27s, tests 1.05s, environment 8.15s, prepare 2.32s)

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Exited with code exit status 1

Other

Successful run from https://app.circleci.com/pipelines/github/cypress-io/cypress-realworld-app/6409/workflows/bac3bf07-f698-4a3f-a731-d1dc4c42673f/jobs/40322 where tests were run in a different (random?) order:

yarn run v1.22.22
$ vitest --run

 RUN  v3.2.4 /root/project

 ✓ src/__tests__/bankaccounts.test.ts (4 tests) 37ms
 ↓ src/__tests__/generateSeedData.test.ts (8 tests | 8 skipped)
 ✓ src/__tests__/users.test.ts (5 tests) 41ms
 ✓ src/utils/__tests__/transactionUtils.test.ts (6 tests) 11ms
 ✓ src/__tests__/likes.test.ts (2 tests) 126ms
 ✓ src/__tests__/comments.test.ts (2 tests) 65ms
 ✓ src/__tests__/contacts.test.ts (5 tests) 25ms
 ✓ src/__tests__/notifications.test.ts (7 tests) 444ms
 ✓ src/__tests__/transactions.test.ts (15 tests | 2 skipped) 261ms

 Test Files  8 passed | 1 skipped (9)
      Tests  44 passed | 10 skipped (54)
   Start at  14:38:26
   Duration  6.32s (transform 4.23s, setup 10.36s, collect 27.35s, tests 1.01s, environment 7.44s, prepare 1.91s)

Done in 7.52s.

[MikeMcC399]

It appears that the test files are not isolated from each other when vitest is running them in parallel.

[MikeMcC399]

As a workaround, it may be necessary to run the unit tests with vitest --no-file-parallelism