windows compatibility: monorepo does not symlink #1487

bholloway · 2019-08-13T00:49:41Z

From @1337cookie https://github.com/keystonejs/keystone-5/pull/1463#issuecomment-519957463

@MadeByMike and I saw the same issue. It is not seen when using admin privillages but that is definitely not advised and not a solution.

C:\Users\user\Documents\web\keystone-5>bolt
⚡️   bolt v0.23.6 (node v10.16.1)
�   [1/4] Validating project...
�   [2/4] Installing project dependencies...
$ C:\Users\user\AppData\Local\Yarn\Data\global\node_modules\bolt\node_modules\.bin\yarn config get user-agent yarn/1.17.3 npm/? node/v10.16.1 win32 x64
yarn install v1.17.3
[1/4] Resolving packages...
[2/4] Fetching packages...
info [email protected]: The platform "win32" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
info [email protected]: The platform "win32" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning "apollo-server-express > apollo-server-core > [email protected]" has incorrect peer dependency "[email protected] - 14.1.x".
warning "graphql-playground-react > [email protected]" has incorrect peer dependency "react@>=15.5 <16".
warning "graphql-playground-react > [email protected]" has incorrect peer dependency "react-dom@>=15.5 <16".
[4/4] Building fresh packages...
$ yarn build:build-field-types && preconstruct dev && node ./packages/build-field-types/bin.js dev  && yarn run --silent contributing-guide
yarn run v1.17.3
$ rimraf packages/build-field-types/lib && babel packages/build-field-types/src --out-dir packages/build-field-types/dist --ignore '**/__tests__/**/*','**/__mocks__/**/*'
Successfully compiled 42 files with Babel.
Done in 4.41s.
� info project is valid!
� error { [Error: EPERM: operation not permitted, symlink 'C:\Users\user\Documents\web\keystone-5\packages\arch\packages\alert\src\index.js' -> 'C:\Users\user\Documents\web\keystone-5\packages\arch\packages\alert\dist\alert.cjs.js.flow']
� error   errno: -4048,
� error   code: 'EPERM',
� error   syscall: 'symlink',
� error   path:
� error    'C:\\Users\\user\\Documents\\web\\keystone-5\\packages\\arch\\packages\\alert\\src\\index.js',
� error   dest:
� error    'C:\\Users\\user\\Documents\\web\\keystone-5\\packages\\arch\\packages\\alert\\dist\\alert.cjs.js.flow' }
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
error Error: spawn C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/bolt/node_modules/.bin/yarn ENOENT
error     at notFoundError (C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/cross-spawn/lib/enoent.js:11:11)
error     at verifyENOENT (C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/cross-spawn/lib/enoent.js:46:16)
error     at ChildProcess.cp.emit (C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/cross-spawn/lib/enoent.js:33:19)

The text was updated successfully, but these errors were encountered:

bholloway · 2019-08-13T00:50:25Z

I will install windows VM and try to investigate this.

1337cookie · 2019-08-13T08:45:26Z

Here's the output from my machine with a fresh clone of the repo. It seems you're right about the privileges but I then encounter a different error.
Windows 10 version 1809.

Running with non-elevated privileges: click to show

C:\Users\user\Documents\test\keystone-5>bolt
⚡️   bolt v0.23.6 (node v10.16.1)
🔎   [1/4] Validating project...
📦   [2/4] Installing project dependencies...
$ C:\Users\user\AppData\Local\Yarn\Data\global\node_modules\bolt\node_modules\.bin\yarn config get user-agent yarn/1.17.3 npm/? node/v10.16.1 win32 x64
yarn install v1.17.3
[1/4] Resolving packages...
[2/4] Fetching packages...
info [email protected]: The platform "win32" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
info [email protected]: The platform "win32" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning "apollo-server-express > apollo-server-core > [email protected]" has incorrect peer dependency "[email protected] - 14.1.x".
warning "graphql-playground-react > [email protected]" has incorrect peer dependency "react@>=15.5 <16".
warning "graphql-playground-react > [email protected]" has incorrect peer dependency "react-dom@>=15.5 <16".
[4/4] Building fresh packages...
$ yarn build:build-field-types && preconstruct dev && node ./packages/build-field-types/bin.js dev  && yarn run --silent contributing-guide
yarn run v1.17.3
$ rimraf packages/build-field-types/lib && babel packages/build-field-types/src --out-dir packages/build-field-types/dist --ignore '**/__tests__/**/*','**/__mocks__/**/*'
Successfully compiled 42 files with Babel.
Done in 4.16s.
🎁 info project is valid!
🎁 error { [Error: EPERM: operation not permitted, symlink 'C:\Users\user\Documents\test\keystone-5\packages\arch\packages\alert\src\index.js' -> 'C:\Users\user\Documents\test\keystone-5\packages\arch\packages\alert\dist\alert.cjs.js.flow']
🎁 error   errno: -4048,
🎁 error   code: 'EPERM',
🎁 error   syscall: 'symlink',
🎁 error   path:
🎁 error    'C:\\Users\\user\\Documents\\test\\keystone-5\\packages\\arch\\packages\\alert\\src\\index.js',
🎁 error   dest:
🎁 error    'C:\\Users\\user\\Documents\\test\\keystone-5\\packages\\arch\\packages\\alert\\dist\\alert.cjs.js.flow' }
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
error Error: spawn C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/bolt/node_modules/.bin/yarn ENOENT
error     at notFoundError (C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/cross-spawn/lib/enoent.js:11:11)
error     at verifyENOENT (C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/cross-spawn/lib/enoent.js:46:16)
error     at ChildProcess.cp.emit (C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/cross-spawn/lib/enoent.js:33:19)

Running with elevated privileges (Administrator): click to show

C:\Users\user\Documents\test\keystone-5>bolt
⚡️   bolt v0.23.6 (node v10.16.1)
🔎   [1/4] Validating project...
📦   [2/4] Installing project dependencies...
$ C:\Users\user\AppData\Local\Yarn\Data\global\node_modules\bolt\node_modules\.bin\yarn config get user-agent yarn/1.17.3 npm/? node/v10.16.1 win32 x64
yarn install v1.17.3
[1/4] Resolving packages...
success Already up-to-date.
$ yarn build:build-field-types && preconstruct dev && node ./packages/build-field-types/bin.js dev  && yarn run --silent contributing-guide
yarn run v1.17.3
$ rimraf packages/build-field-types/lib && babel packages/build-field-types/src --out-dir packages/build-field-types/dist --ignore '**/__tests__/**/*','**/__mocks__/**/*'
Successfully compiled 42 files with Babel.
Done in 1.36s.
🎁 info project is valid!
🎁 success created links!
🎁 info project is valid!
🎁 success created links!
📝 Contributing to KeystoneJS?
🔗 Read the full Contributing Guide (https://github.com/keystonejs/keystone-5/blob/master/CONTRIBUTING.md)
Done in 4.52s.
🔗   [3/4] Linking workspace dependencies...
error Error: Can't read shim path from 'C:/Users/user/Documents/test/keystone-5/packages/keystone/bin/cli.js', it doesn't appear to be a cmd-shim
error     at cb (C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/bolt/dist/modern/utils/fs.js:154:75)
error     at AnyPromise (C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/typeable-promisify/index.js:13:3)
error     at new Promise (<anonymous>)
error     at promisify (C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/typeable-promisify/index.js:12:40)
error     at readCmdShim (C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/bolt/dist/modern/utils/fs.js:154:42)
error     at cmdShim (C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/bolt/dist/modern/utils/fs.js:96:88)
error     at createWindowsSymlink (C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/bolt/dist/modern/utils/fs.js:122:18)
error     at Object.symlink (C:/Users/user/AppData/Local/Yarn/Data/global/node_modules/bolt/dist/modern/utils/fs.js:134:18)

bholloway · 2019-08-13T11:38:36Z

When installing Git in windows.

I'll check the box and see how it goes.

bholloway · 2019-08-13T13:45:48Z

Stuck in hell. Had to install python, windows build tools. Build tools install failed. 😒

gautamsi · 2019-08-13T13:53:00Z

I do want this to work on Windows. for now I used WSL (Windows Subsystem for Linux) for Windows 10 so that I can get through any issue.

1337cookie · 2019-08-13T17:45:47Z

Seems like a windows/symlink limitation put in place as some sort of security measure.
The short answer is Microsoft would like you to enable "developer mode" in windows if you want to create a symlink from a non-elevated command shell.

How to enable developer mode CLICK HERE TO SHOW

Open settings (Win + I) and click Update and Security.

Click "For Developers" on the left menu.

Select the "Developer mode" radio box and wait for it to install.

I personally didn't require a restart or to sign out/sign in.

More related stuff @ ember.js docs including how to use "Local Security Policy" to change this setting, https://ember-cli.com/user-guide/#enabling-symlinks :
To create symlinks the account running Ember CLI must have the SeCreateSymbolicLinkPrivilege. Users in the Administrators group have this permission already. However, if UAC (User Access Control) is enabled, users in the Administrators group must run their shell using Run As Administrator because UAC strips away certain permissions from the Administrators +group, including SeCreateSymbolicLinkPrivilege.

Windows developer blog info https://blogs.windows.com/windowsdeveloper/2016/12/02/symlinks-windows-10/:
For users who have Developer Mode enabled, the mklink command will now successfully create a symlink if the user is not running as an administrator.

Mystery solved?

1337cookie · 2019-08-13T21:06:08Z

Stuck in hell. Had to install python, windows build tools. Build tools install failed. 😒

npm install -g --production [email protected]

bholloway · 2019-08-14T11:38:44Z

Restarted installation with a fresh VM snapshot.

Enable windows Settings > For developers > developer mode
Install git with symlink option
Install nvm-windows and nvm install latest and nvm use ... that version
[admin] npm install -g --production [email protected]
[admin] npm install -g node-gyp (per this issue with node-gyp)
clone the repo
bolt install

Now that I go through the motions I realise that I have used "developer mode" previously.
I'm not sure the git symlink option is helpful, possibly a detail to determine later.

I'm encountering can't read shim path too now.

bholloway · 2019-08-14T11:40:58Z

Possibly boltpkg/bolt#207

1337cookie · 2019-08-15T11:52:05Z

Possibly boltpkg/bolt#207

Yeah I also came across that one.

So I built the bolt package including the changes in this PR @ boltpkg/bolt#215 I had to install and use the WSL with ubuntu, node and yarn, to build it. (Thanks for reminding me the WSL exists @gautamsi )

Using "yarn link" on the updated bolt directory in windows and "yarn link bolt" in the keystone directory, running "bolt", "bolt dev" and "bolt demo meetup dev" seems to work as expected, with the todo and meetup demo working properly in the browser.

Click here to see output from "bolt", "bolt dev" and "bolt demo meetup dev"

C:\Users\usernameremoved\Documents\web\keystone-5>bolt
⚡️   bolt v0.23.6 (node v10.16.1)
�   [1/4] Validating project...
�   [2/4] Installing project dependencies...
$ C:\Users\usernameremoved\Documents\web\bolt\node_modules\.bin\yarn config get user-agent yarn/1.12.3 npm/? node/v10.16.1 win32 x64
yarn install v1.12.3
[1/4] Resolving packages...
success Already up-to-date.
$ yarn build:build-field-types && preconstruct dev && node ./packages/build-field-types/bin.js dev  && yarn run --silent contributing-guide
yarn run v1.12.3
$ rimraf packages/build-field-types/lib && babel packages/build-field-types/src --out-dir packages/build-field-types/dist --ignore '**/__tests__/**/*','**/__mocks__/**/*'
Successfully compiled 42 files with Babel.
Done in 1.42s.
� info project is valid!
� success created links!
� info project is valid!
� success created links!
� Contributing to KeystoneJS?
� Read the full Contributing Guide (https://github.com/keystonejs/keystone-5/blob/master/CONTRIBUTING.md)
Done in 4.53s.
�   [3/4] Linking workspace dependencies...
�   [4/4] Linking workspace binaries...
�  success Installed and linked workspaces.
�   Done in 24.51s.

C:\Users\usernameremoved\Documents\web\keystone-5>bolt dev
⚡️   bolt v0.23.6 (node v10.16.1)
$ yarn demo todo dev
$ yarn --cwd ./demo-projects todo dev
$ yarn --cwd todo dev
$ cross-env NODE_ENV=development DISABLE_LOGGING=true keystone dev
i Command: keystone dev
√ Validated project entry file ./index.js
√ Keystone server listening on port 3000
√ Initialised Keystone instance
√ Connected to database
√ Keystone instance is ready �
� Keystone Admin UI:   http://localhost:3000/admin
� GraphQL Playground:  http://localhost:3000/admin/graphiql
� GraphQL API:         http://localhost:3000/admin/api

C:\Users\usernameremoved\Documents\web\keystone-5>bolt demo meetup dev
⚡️   bolt v0.23.6 (node v10.16.1)
$ yarn --cwd ./demo-projects meetup dev
$ yarn --cwd meetup dev
$ cross-env NODE_ENV=development DISABLE_LOGGING=true keystone dev
i Command: keystone dev
√ Validated project entry file ./index.js
√ Keystone server listening on port 3000
√ Initialised Keystone instance
- Connecting to database[ ready ] compiled successfully
\ Connecting to database[ wait ]  compiling ...
- Connecting to database[ ready ] compiled successfully
√ Connected to database
√ Keystone instance is ready �
� Keystone Admin UI:   http://localhost:3000/admin
� GraphQL Playground:  http://localhost:3000/admin/graphiql
� GraphQL API:         http://localhost:3000/admin/api
[ event ] build page: /
[ wait ]  compiling ...
[ ready ] compiled successfully

Running "bolt demo blog dev" however shows a new error which might not be related.

Click here to see "bolt demo blog dev" output

⚡️   bolt v0.23.6 (node v10.16.1)
$ yarn --cwd ./demo-projects blog dev
$ yarn --cwd blog dev
$ cross-env NODE_ENV=development DISABLE_LOGGING=true node -r dotenv-safe/config `yarn bin`/keystone dev
internal/modules/cjs/loader.js:638
    throw err;
    ^

Error: Cannot find module 'C:\Users\usernameremoved\Documents\web\keystone-5\demo-projects\blog\`yarn'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
    at Function.Module._load (internal/modules/cjs/loader.js:562:25)
    at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
    at startup (internal/bootstrap/node.js:283:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
error Error: spawn ../bolt/node_modules/.bin/yarn ENOENT
error     at notFoundError (C:/Users/usernameremoved/Documents/web/bolt/node_modules/cross-spawn/lib/enoent.js:11:11)
error     at verifyENOENT (C:/Users/usernameremoved/Documents/web/bolt/node_modules/cross-spawn/lib/enoent.js:46:16)
error     at ChildProcess.cp.emit (C:/Users/usernameremoved/Documents/web/bolt/node_modules/cross-spawn/lib/enoent.js:33:19)

Regardless I think the symlink issue can be solved by documenting the "developer mode" option in the docs for the demo projects and pushing for boltpkg/bolt#215 to be merged/reassessed.

bholloway · 2019-08-15T12:10:06Z

That branch from the bolt PR did work for me in windows.

I spoke to one of the bolt maintainers in person today - They will need me to show why it works and determine whether it is the best change.

So I’ll working on that now.

MadeByMike · 2019-08-20T06:11:43Z

Seems like this could be "fixed" by documenting the issue?

bholloway · 2019-08-20T06:42:12Z

Yes @MadeByMike docs for the "Developer Mode".

We also need this bolt PR to merge boltpkg/bolt#241. So a PR to keystone to update the dependency.

1337cookie · 2019-09-22T07:44:31Z

PR 241 has been merged in bolt 0.24.1 (funny how the PR # and version match 🤔simulation theory confirmed?)

gautamsi · 2019-10-11T23:46:38Z

after #1727 I see this error

🎁 error @arch-ui/controls the dist directory isn't included in the published files for this package, please add it to the files field in the package's package.json
error Command failed with exit code 1.                                                                                  info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

gautamsi · 2019-10-12T00:19:50Z

PR #1727 deprecated use of bolt and now insist on yarn, I have latest yarn installed (v1.19.1).

Vultraz · 2019-11-01T09:16:46Z

@gautamsi getting that error too. No idea how to fix :( The specific package it complains about does seem to vary, though.

EDIT: the error seems to be coming from preconstruct dev. Even running in an elevated shell or with Developer Mode on doesn't help.

Vultraz · 2019-11-01T13:46:13Z

Ok, after hours and hours of looking I've tracked down the dist directory isn't included in the published files problem! It's a bug in preconstruct Opened a PR.

Pinging @mitchellhamilton

MadeByMike added the core label Aug 20, 2019

This was referenced Nov 1, 2019

Mention symlink caveats when installing on Windows #1889

Merged

Update preconstruct to 0.3.2 #1892

Merged

timleslie closed this as completed in #1892 Nov 3, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

windows compatibility: monorepo does not symlink #1487

windows compatibility: monorepo does not symlink #1487

bholloway commented Aug 13, 2019

bholloway commented Aug 13, 2019

1337cookie commented Aug 13, 2019

bholloway commented Aug 13, 2019

bholloway commented Aug 13, 2019

gautamsi commented Aug 13, 2019

1337cookie commented Aug 13, 2019 •

edited

Loading

1337cookie commented Aug 13, 2019

bholloway commented Aug 14, 2019

bholloway commented Aug 14, 2019

1337cookie commented Aug 15, 2019 •

edited

Loading

bholloway commented Aug 15, 2019

MadeByMike commented Aug 20, 2019

bholloway commented Aug 20, 2019

1337cookie commented Sep 22, 2019 •

edited

Loading

gautamsi commented Oct 11, 2019

gautamsi commented Oct 12, 2019

Vultraz commented Nov 1, 2019 •

edited

Loading

Vultraz commented Nov 1, 2019 •

edited

Loading

windows compatibility: monorepo does not symlink #1487

windows compatibility: monorepo does not symlink #1487

Comments

bholloway commented Aug 13, 2019

bholloway commented Aug 13, 2019

1337cookie commented Aug 13, 2019

bholloway commented Aug 13, 2019

bholloway commented Aug 13, 2019

gautamsi commented Aug 13, 2019

1337cookie commented Aug 13, 2019 • edited Loading

1337cookie commented Aug 13, 2019

bholloway commented Aug 14, 2019

bholloway commented Aug 14, 2019

1337cookie commented Aug 15, 2019 • edited Loading

bholloway commented Aug 15, 2019

MadeByMike commented Aug 20, 2019

bholloway commented Aug 20, 2019

1337cookie commented Sep 22, 2019 • edited Loading

gautamsi commented Oct 11, 2019

gautamsi commented Oct 12, 2019

Vultraz commented Nov 1, 2019 • edited Loading

Vultraz commented Nov 1, 2019 • edited Loading

1337cookie commented Aug 13, 2019 •

edited

Loading

1337cookie commented Aug 15, 2019 •

edited

Loading

1337cookie commented Sep 22, 2019 •

edited

Loading

Vultraz commented Nov 1, 2019 •

edited

Loading

Vultraz commented Nov 1, 2019 •

edited

Loading