node-capnp not building under vagrant-spk meteor stack

[gischer]

For a variety of reasons, I'm working on switching to using vagrant-spk instead of meteor-spk However, when I get to the step in build.sh that says

node install --production

it get this error (and a lot of other stuff, which I have omitted because it seems not to be relevant):

> capnp@0.5.0 install /home/vagrant/bundle/programs/server/npm/node_modules/capnp
> node ./build.js

`linux-x64-v8-7.8` exists; testing
Problem with the binary; manual build incoming
make: Entering directory '/home/vagrant/bundle/programs/server/npm/node_modules/capnp/build'
 CXX(target) Release/obj.target/capnp/src/node-capnp/capnp.o
 ../src/node-capnp/capnp.cc:31:10: fatal error: capnp/dynamic.h: No such file or directory 

[ocdtrekkie]

As @zenhack noted, this could be vagrant-spk-related, though I found the fact that it appears to error out building node-capnp interesting. However, I also wanted to check... were you using current vagrant-spk master? My proposed changes in https://github.com/sandstorm-io/vagrant-spk/pull/290/files use a later version of meteor-spk, which includes a commit fixing locating node-capnp files... so might very well be the fix.

If you weren't using this, perhaps make the change from the linked PR to your setup.sh, then vagrant-spk vm destroy and vagrant-spk vm up to recreate the VM using the newer setup.

[gischer]

I get the same error with both master and update-meteor-spk branches.

[kentonv]

This seems like a vagrant-spk issue, not a node-capnp issue. node-capnp is trying to build itself but libcapnp isn't installed on the system.

[ocdtrekkie]

This seems like a vagrant-spk issue, not a node-capnp issue. node-capnp is trying to build itself but libcapnp isn't installed on the system.

Shouldn't libcapnp already be installed on a system with Sandstorm running on it?

[kentonv]

Shouldn't libcapnp already be installed on a system with Sandstorm running on it?

No... Sandstorm does not install any libraries onto the host system. Plus, it links capnp statically.

[zenhack]

@ocdtrekkie, the capnp schema for both capnproto core and sandstorm are included in <sandstorm bundle directory>/opt/sandstorm/usr/include, but the capnproto library and C++ headers are not.

We should probably adjust both the meteor & node stacks to install capnp. Annoyingly, the version in the debian repositories is too out of date to support the -> stream syntax, so we'll need to get a more recent version from somewhere else.

[gischer]

For what it's worth, when I tried to run this using meteor-spk, I got a different error that seems related. When I ran

meteor-spk dev

I got an error in the sandstorm server console saying it couldn't find 'libkj.so' or something like that. If this seems valuable, I'll go try and recreate for the exact error.

[zenhack]

libkj is part of capnproto, so yeah that's the same issue. I don't think there's much point in going to the trouble of reproducing that as I think we already know what the problem is: you need to install capnproto inside the vagrant box (and the vagrant-spk meteor stack should do this itself). You'll want at least version 0.8 for compatibility with the `-> stream` syntax that I mentioned earlier.

…

-Ian Quoting gischer (2021-04-09 12:53:01)

For what it's worth, when I tried to run this using meteor-spk, I got a different error that seems related. When I ran meteor-spk dev I got an error in the sandstorm server console saying it couldn't find 'libkj.so' or something like that. If this seems valuable, I'll go try and recreate for the exact error. -- You are receiving this because you were mentioned. Reply to this email directly, [1]view it on GitHub, or [2]unsubscribe. Verweise 1. #69 (comment) 2. https://github.com/notifications/unsubscribe-auth/AAGXYPWXFWHFB3UBJJXUCJDTH4WG3ANCNFSM42PSHYHQ

[gischer]

I decided to build capnproto by hand inside my vagrant box. This appeared to be successful, in that I have binaries installed in the box.

However, I'm now getting the thing where meteor/node can't find libkj. This appears to be something subtle, since there is a /usr/lib/libkj.0.5.3.so (the exact name the error complained about not finding) in the box. My best (albeit wild-assed) guess is that this has to do with meteor bundling somehow. Somebody there is setting, or not setting a search path incorrectly, maybe? But I know next to nothing about meteor builds.

For completeness, here are the long forms:

Directory ls:

$ vagrant-spk vm ssh
Calling 'vagrant' 'ssh' in /home/gischer/build/mansia-market-manager/.sandstorm
Linux localhost 4.9.0-4-amd64 #1 SMP Debian 4.9.65-3 (2017-12-03) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Fri May  7 16:38:11 2021 from 10.0.2.2
vagrant@localhost:~$ ls /usr/lib/libkj*
/usr/lib/libkj-0.5.3.so  /usr/lib/libkj-async-0.5.3.so  /usr/lib/libkj-async.so
/usr/lib/libkj.a         /usr/lib/libkj-async.a         /usr/lib/libkj.so

And the error:

** Starting Meteor...
/programs/server/node_modules/fibers/future.js:280
                        throw(ex);
                        ^

Error: libkj-0.5.3.so: cannot open shared object file: No such file or directory
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1208:18)
    at Module.load (internal/modules/cjs/loader.js:1002:32)
    at Module.Mp.load (/programs/server/runtime.js:15:31)
    at Function.Module._load (internal/modules/cjs/loader.js:901:14)
    at Module.require (internal/modules/cjs/loader.js:1044:19)
    at require (internal/modules/cjs/helpers.js:77:18)
    at Object.<anonymous> (/programs/server/npm/node_modules/capnp/src/node-capnp/capnp.js:47:15)
    at Module._compile (internal/modules/cjs/loader.js:1158:30)
    at Module.Mp._compile (/programs/server/runtime.js:50:23)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1178:10)
    at Module.load (internal/modules/cjs/loader.js:1002:32)
    at Module.Mp.load (/programs/server/runtime.js:15:31)
    at Function.Module._load (internal/modules/cjs/loader.js:901:14)
    at Module.require (internal/modules/cjs/loader.js:1044:19)
    at require (internal/modules/cjs/helpers.js:77:18)
    at npmRequire (/programs/server/npm-require.js:133:12)
** HTTP-BRIDGE: App server exited with status code: 1

[zenhack]

Quoting gischer (2021-05-07 13:00:03)

I decided to build capnproto by hand inside my vagrant box. This appeared to be successful, in that I have binaries installed in the box. However, I'm now getting the thing where meteor/node can't find libkj. This appears to be something subtle, since there is a /usr/lib/libkj.0.5.3.so

What version of capnproto did you build? 0.5.3 is *very* old, and you'll need at least 0.8 for it to parse the current sandstorm schema correctly. Based on the error, I imagine either: - You built 0.5.3, in which case I don't know why it's not finding it off-hand, but you should probably update anyway, or more likely, - This file is part of a distro package, rather than the version you built, and so is a red herring -- but it would be good to pin down where the version you built ended up.

[gischer]

I had built from 0.8.0. But the build was messy. I have nuked the box and made a clean one with 0.8.0 (from the tarball). This has resulted in a /usr/local/lib/libkj-0.8.0.so This is installed in /usr/local/lib, whereas the old version was in /usr/lib (I think it was from a package install, as I was trying to figure out how this all fit together). So I think some search path grabbed that and linked against it, even though the newer binaries were present on the system.

I guess I was expecting an overwrite from make install.

Anyway, this new, cleaner build has made progress: It has replaced the name of the missing library in the above message with "libkj-0.8.0.so" The meteor/node runtime now can't find this one, even though it lives in /usr/local/lib

[gischer]

I have learned something about this issue. It appears that, in production mode, Meteor puts itself in a chroot jail inside its bundle. So, it seems that while the capnp libs are installed in the root system of the Vagrant instance, they aren't also installed in the bundle, and thus can't be found. And the node-capnp install does not install these itself, which is what Meteor seems to expect.

The build.sh script could be modified to handle this, though other solutions might be more desirable.