Switching to Yarn

Due to the fact that the datagateway repository is a monorepo structure, there have been problems with managing dependencies. There are essentially two strategies one can use with a monorepo: either each package has its own version of its libraries in its own node_modules, or the root package "hoists" some or all of the libraries into the top level node_modules. Hoisting is good as it reduces the amount of packages installed and means that packages that are imported by another (chiefly datagateway-common) share the same libraries as the one that is importing it.

However, hoisting is not natively supported by npm. Lerna attempts to solve this by modifying package.json files during bootstrapping, aka putting the dependencies as dependencies of the root package and installing and then removing the dependencies after install. This mangling however can ruin lockfiles and in general hoisted bootstrap doesn't play well with lockfiles, which can cause packages updating during development to cause build issues.

One solution to this is to just hoist using lerna and npm anyway, and deal with any problems that arise in the future. This is needed due to Material-UI needing to be hoisted and this doesn't work with selective hoisting (the current used strategy).

Another solution is to instead switch package managers and use yarn instead of npm. Yarn is an alternative to npm that is developed by Facebook. The motivation for switching is that yarn has a feature called "workspaces" which is effectively built in support for monorepo structures and hoisting with dependable lockfiles.

Yarn pros:

• Natively supports workspaces, nicely solving our problem
• yarn.lock file applies to whole workspace, which means better lockfiles for our monorepo structure
• Might be faster than regular npm + lerna bootstrap

Yarn cons:

• NodeJS automatically comes with npm, whereas yarn needs to be installed seperately
• Could potentially confuse developers who are used to using npm.

yarn vs npm command differences:

also see Migrating from npm

npm	yarn
npm install	yarn or yarn install
npm install [package]	yarn add [package]
npm install [package] --save-dev	yarn add [package] --dev
npm uninstall [package]	yarn remove [package]

All of the other npm commands (e.g. npm start, npm test, npm run x) can instead be run like yarn start, yarn test, yarn x (note how we don't need to run part, although yarn run x would still work)