Releasing v6.0.0 #529

aminya · 2022-11-17T02:58:04Z

I'd like to explain the project's status and track the remaining issues for v6.0.0.

The latest beta builds without any issues, and prebuilt binaries for many platforms are provided.

The normal unit tests pass locally (pnpm run test.skip_gc_tests)

Some of the GC tests for some protocols fail (pnpm run test). However, I don't think these GC tests are crucial for functionality since the tests try to rely on GC's unpredictable behaviour.

If there is an issue that actually blocks us from releasing v6.0.0 please leave a comment.

The text was updated successfully, but these errors were encountered:

ni-mkrieg · 2022-11-18T20:40:37Z

thanks, yes would be great if you could prevent the whole rebuild, it is not running for a good while and then I get this error on x64 Mac.

ld: library not found for -lsodium
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Release/zeromq.node] Error 1
rm 7722eaf02fcc073df28ffd1fa831e71544edfd42.intermediate
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/markus.krieg/.nvm/versions/node/v16.13.0/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (node:events:390:28)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
gyp ERR! System Darwin 20.6.0
gyp ERR! command "/Users/markus.krieg/.nvm/versions/node/v16.13.0/bin/node" "/Users/markus.krieg/.nvm/versions/node/v16.13.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/markus.krieg/git/native-access/node_modules/zeromq

aminya · 2022-11-18T20:43:29Z

This issue can be fixed by installing sodium

brew install libsodium gnutls

Could you install zeromq with npm i --verbose. node-gyp-build doesn't give the reason why the prebuild is not picked up.
prebuild/node-gyp-build#54

aminya · 2022-11-18T21:19:35Z

I am working on the node-gyp-build's preinstall logic:
prebuild/node-gyp-build#55

Hopefully, that is merged soon, and you will not need to do a rebuild!

ni-mkrieg · 2022-11-18T21:34:18Z

That would be very cool, even after installing sodium on M1 it is currently still rebuilding indeed. So sodium installed prevented rebuild on x64 but not arm64. Actually I am not 100% sure the brew install for libsodium worked on arm, still getting this line:

ld: library not found for -lsodium

The yarn install --verbose did not contain much useful information either.

verbose 53.652263084 Error: /Users/markus/git/native-access/node_modules/zeromq: Command failed.
Exit code: 1
Command: (shx test -f ./script/build.js || run-s build.js) && node-gyp-build
Arguments: 
Directory: /Users/markus/git/native-access/node_modules/zeromq
Output:
dyld[91022]: missing symbol called

Downloading libzmq source from https://github.com/zeromq/libzmq/releases/download/v4.3.4/zeromq-4.3.4.tar.gz
Building libzmq Release

ni-mkrieg · 2022-11-18T22:15:50Z

Okay so an x64 this is the reason why it rebuilds, I believe because it cannot find libsodium that is needed by your precompiled binary? That would also explain why it does not recompile after I install libsodium via homebrew. Is there a way for you to compile that lib into your prebuilt binary and not dynamically link to it?

Or could you replace libsodium all together with something that is already preinstalled during CI? Does not make sense for you either to spend all this time brew installing it during every CI build...

verbose 108.080591296 Error: /Users/markus/git/native-access/node_modules/zeromq: Command failed.
Exit code: 1
Command: (shx test -f ./script/build.js || run-s build.js) && node-gyp-build
Arguments: 
Directory: /Users/markus/git/native-access/node_modules/zeromq
Output:
node:internal/modules/cjs/loader:1183
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: dlopen(/Users/markus/git/native-access/node_modules/zeromq/prebuilds/darwin-x64/node.napi.glibc.node, 1): Library not loaded: /usr/local/opt/libsodium/lib/libsodium.23.dylib
  Referenced from: /Users/markus/git/native-access/node_modules/zeromq/prebuilds/darwin-x64/node.napi.glibc.node
  Reason: image not found
    at Object.Module._extensions..node (node:internal/modules/cjs/loader:1183:18)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at load (/Users/markus/git/native-access/node_modules/node-gyp-build/index.js:22:10)
    at Object.<anonymous> (/Users/markus/git/native-access/node_modules/node-gyp-build/build-test.js:19:19)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Module.load (node:internal/modules/cjs/loader:981:32) {
  code: 'ERR_DLOPEN_FAILED'
}

aminya · 2022-11-21T21:55:40Z

libsodium is preinstalled on the GitHub runners, and it is very small. I am not sure where you are running this. GitHub doesn't offer an Arm runner. Please contact your CI engineer.

ni-mkrieg · 2022-11-21T23:01:21Z

all good if we do not have to rebuild, was more worried about your own CI.yml file having to do this for each build:

- name: Install Mac-OS Dependencies
        if: contains(matrix.os, 'macos')
        run: |
          brew install libsodium gnutls

were you able to avoid the rebuild on Mac x64 and arm64 - ideally without us having to install libsodium? it worked fine without libsodium installed in beta.6 for Mac x64...

aminya · 2022-11-21T23:09:50Z

That snippet is just to declare the dependencies to be more accurate. In practice it does nothing because it is already installed:
https://github.com/zeromq/zeromq.js/actions/runs/3483046346/jobs/5826083947#step:7:11

You will need the libsodium dynamic library in either case during runtime. Without that library, the connections will not be secure.

Regarding the rebuild, I am still handling the external node-gyp-build issues. Please stay tuned.

bfan1256 · 2022-11-22T03:42:28Z

I have done brew install and ran a verbose and can see that libsodium is found But it just happens at the final linking step that it just can't find -lsodium

I'm on M1 Mac ARM (osx64)

ld: library not found for -lsodium
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Release/zeromq.node] Error 1
rm 7722eaf02fcc073df28ffd1fa831e71544edfd42.intermediate
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (node:events:390:28)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
gyp ERR! System Darwin 21.6.0
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/weigfan/code/shade-cinema/shade-plugin/node_modules/zeromq

td-krzysiek · 2022-11-22T11:43:35Z

Hello,

Exactly the same issue here on M1, this is after brew reinstall libsodium gnutls. Full log:

2022-11-22T11_35_47_985Z-debug-0.log

DonJayamanne · 2022-11-25T10:50:27Z

Hi, Thank you for looking into releasing a new version.
I'm one of the engineers working on the Jupyter extension for VS Code
This package is used to communicate with Jupyter kernels (python kernels) over zmq

We found that the package fails in some cases which you may be aware of
You have requested this new version to be tested here microsoft/vscode-jupyter#10386 (comment)

I'll test this and get back to you with a confirmation.
Again thanks for your contribution to this project

aminya · 2023-01-05T20:22:39Z

Reported the issue upstream.
zeromq/libzmq#4484

My temporary solution would be to include the dynamic libraries in the package by copying them beside the .node files in the prebuilds directory. Contributions for doing this are welcome!

smitalm · 2023-01-24T14:37:05Z

@aminya is this still blocking 6.0.0 release? Linked upstream issue was closed zeromq/libzmq#4484

aminya · 2023-01-25T08:46:06Z

Notified the core members to see what we can do about the upstream issue.

denyeo · 2023-06-12T10:31:38Z

I've not been able to get the prebuilt binaries on Node 18.16 on Ubuntu 20.04 Focal on WSL 1.2.5.0 (x64), kernel 5.15.90.1. I also tried the node:18.15.0-slim Docker image, but it doesn't get the binaries either.

It looks like all the latest node images are built on Debian 11, which only has glibc 2.31, while zeromq.js 6.0.0 requires glibc 2.33 to build (based on the error messages I get). I believe glibc shouldn't be upgraded, since it's a system package. I'd also prefer using prebuilt binaries instead of building.

Is there a publicly available, recent node image that works for prebuilt binaries? Alpine ones? Otherwise, which Debian/Alpine releases have the glibc/libc6/stdlibc++ versions needed to use the binaries? Just Ubuntu 22.04? @aminya Would be grateful for any information you could provide!

edit: Seems like node:18-alpine does get the binary. Too bad about Debian

Bartel-C8 · 2023-06-12T13:59:44Z

The glibc dependency on linux(/debian) is awful.

I use zeromq.js in a production environment where I don't have any influence on the host OS. Which means I already experienced users which could not open the application because they had an older glibc.
I now build on a Debian 9 ancient docker image to maximise glibc compatibility, which uses v2.24.

It would be awesome if the prebuilt did the same thing, so maximum compatibility is guaranteed by default. Now indeed it's being build by Ubuntu 22, which uses glibc 2.33.
(Which means e.g. that Ubuntu 18 and Ubuntu 20 are not supported, unless built yourself on an older machine).
Would this be possible in the current CI setup @aminya ?

aminya · 2023-06-12T17:35:45Z

Just to clarify, this issue is not unique to zeromq.js, and it only happens for Linux where the dependencies are shared libraries:
nodejs/node-gyp#2297 (comment)

What we can do is to compile on multiple Ubuntu versions so that it covers all the Glibcxx versions. Sometimes binaries compiled with older Glibcxx work on the newer versions.

Bartel-C8 · 2023-06-13T08:19:19Z

@aminya : Won't the binary always work when compiled with an older glibc version? As newer glibc versions are always backwards compatible?

So best would be to compile (just one?) prebuilt with the (reasonably) oldest glibc version? That way, like in my case, an end-provider should not care about it's build system and glibc version, as automatically the prebuilt with the old glibc compatibility is linked?

Maybe also good to mention in the readme docs that this could be an issue.

aminya · 2023-06-13T17:52:07Z

The issue is not GlibC. I think you have mistaken GLibCXX for GLIBC here.

Bartel-C8 · 2023-06-13T18:01:20Z

Could be, not that familiar with the difference between them, but the error I got was:

node:electron/js2c/asar_bundle:2 Uncaught Error: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /tmp/.mount-cAKD9Q/resources/app.asar.unpacked/node_modules/zeromq/build/Release/zeromq.node)

So it seems to be GLIBC, no?
When I fixed that one, indeed, the next dependency error was GLIBCXX

Uncaught Error: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /tmp/.mount-13pmTw/resources/app.asar.unpacked/node_modules/zeromq/build/Release/zeromq.node)

I see that you have changed the CI to use Ubuntu 20.04, which is a good step in the right direction.

But in the problematic use-case I experienced the system was using these libs:

file /lib/x86_64-linux-gnu/libc.so.6
/lib/x86_64-linux-gnu/libc.so.6: symbolic link to libc-2.27.so

file /usr/lib/x86_64-linux-gnu/libstdc++.so.6
/usr/lib/x86_64-linux-gnu/libstdc++.so.6: symbolic link to libstdc++.so.6.0.25

Ubuntu 18.04.6 LTS e.g. still has glibc 2.27 (https://repology.org/project/glibc/versions). And it just recently dropped LTS state (31 May 2023) , so not unimaginable that people are still using it...

aminya · 2023-06-13T18:10:01Z

Released v6.0.0-beta.17. Please update your versions, and try again.
https://github.com/zeromq/zeromq.js/releases/tag/v6.0.0-beta.17

denyeo · 2023-06-14T03:56:47Z

Thank you @aminya for the changed prebuild, and @Bartel-C8 for the detailed feedback!

aminya · 2023-06-25T04:22:54Z

I issued a statement regarding the status of zeromq.js here:

#576

aminya · 2024-06-17T19:23:51Z

Status update

Zeromq is in a much better status as of the recent release. We have prebuilt binaries for many platforms. Most of the tests are passing. The remaining failing tests are related to Proxies and some GC tests, which I don't consider a blocker. But I appreciate contributions. Given that curve security is optional upstream, we don't have issues regarding Libsodium for now. So we can probably go for the v6 release soon.

aminya · 2024-08-13T04:45:02Z

v6.0.0 was released! Here we go:
https://github.com/zeromq/zeromq.js/releases/tag/v6.0.0

aminya added bug question 6.x awaiting reply and removed bug labels Nov 17, 2022

aminya pinned this issue Nov 17, 2022

aminya mentioned this issue Nov 17, 2022

[6.x] Troubleshooting memory leaks #466

Closed

This comment was marked as resolved.

Sign in to view

aminya mentioned this issue Nov 18, 2022

fix: compile the build typescript files for production #531

Merged

This comment was marked as resolved.

Sign in to view

This comment was marked as outdated.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as outdated.

Sign in to view

td-krzysiek mentioned this issue Nov 24, 2022

fix missing sodium link_directories zeromq/libzmq#4466

Merged

This comment was marked as resolved.

Sign in to view

aminya closed this as completed in 99500f2 Nov 25, 2022

softwarenerd mentioned this issue Feb 18, 2023

zeromq hack posit-dev/positron#189

Closed

This comment was marked as off-topic.

Sign in to view

aminya mentioned this issue Jun 13, 2023

ci: build on Ubuntu 20.04 to support older GlibCXX #571

Merged

aminya mentioned this issue Jun 15, 2023

build: use official Findlibsodium + add option to download and build libsodium zeromq/libzmq#4562

Closed

coolaj86 mentioned this issue Jun 25, 2023

Just release v6.18.0 already #575

Closed

anggoran mentioned this issue Aug 10, 2023

Deno panicked trying to run ZeroMQ from NPM package denoland/deno#18345

Open

douglas-linder-hireup mentioned this issue Sep 14, 2023

Having issues installing on a Macbook Air M1 yunabe/tslab#62

Open

This was referenced Jun 18, 2024

test: use WeakRef instead of weak-napi for GC tests + chore: add debugging configuration for VsCode #631

Merged

feat: support build options via npmrc #633

Merged

fix: reduce production build dependencies #635

Merged

This was referenced Jul 15, 2024

Can not get it run on windows jehna/humanify#10

Open

Dynamic require of "assert" is not supported Error when importing Wakaru Unpacker pionxzh/wakaru#134

Open

Cannot install zeromq 5 on Node 19 #551

Closed

aminya closed this as completed in c3d4666 Aug 13, 2024

aminya unpinned this issue Aug 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Releasing v6.0.0 #529

Releasing v6.0.0 #529

aminya commented Nov 17, 2022 •

edited

Loading

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as outdated.

ni-mkrieg commented Nov 18, 2022

This comment was marked as outdated.

aminya commented Nov 18, 2022 •

edited

Loading

This comment was marked as outdated.

aminya commented Nov 18, 2022

ni-mkrieg commented Nov 18, 2022

ni-mkrieg commented Nov 18, 2022 •

edited

Loading

aminya commented Nov 21, 2022 •

edited

Loading

ni-mkrieg commented Nov 21, 2022

aminya commented Nov 21, 2022

bfan1256 commented Nov 22, 2022 •

edited

Loading

td-krzysiek commented Nov 22, 2022 •

edited

Loading

This comment was marked as resolved.

This comment was marked as outdated.

This comment was marked as resolved.

DonJayamanne commented Nov 25, 2022

aminya commented Jan 5, 2023

smitalm commented Jan 24, 2023

aminya commented Jan 25, 2023

This comment was marked as off-topic.

denyeo commented Jun 12, 2023 •

edited

Loading

Bartel-C8 commented Jun 12, 2023

aminya commented Jun 12, 2023 •

edited

Loading

Bartel-C8 commented Jun 13, 2023

aminya commented Jun 13, 2023

Bartel-C8 commented Jun 13, 2023 •

edited

Loading

aminya commented Jun 13, 2023 •

edited

Loading

denyeo commented Jun 14, 2023

aminya commented Jun 25, 2023 •

edited

Loading

aminya commented Jun 17, 2024 •

edited

Loading

aminya commented Aug 13, 2024

Releasing v6.0.0 #529

Releasing v6.0.0 #529

Comments

aminya commented Nov 17, 2022 • edited Loading

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as outdated.

ni-mkrieg commented Nov 18, 2022

This comment was marked as outdated.

aminya commented Nov 18, 2022 • edited Loading

This comment was marked as outdated.

aminya commented Nov 18, 2022

ni-mkrieg commented Nov 18, 2022

ni-mkrieg commented Nov 18, 2022 • edited Loading

aminya commented Nov 21, 2022 • edited Loading

ni-mkrieg commented Nov 21, 2022

aminya commented Nov 21, 2022

bfan1256 commented Nov 22, 2022 • edited Loading

td-krzysiek commented Nov 22, 2022 • edited Loading

This comment was marked as resolved.

This comment was marked as outdated.

This comment was marked as resolved.

DonJayamanne commented Nov 25, 2022

aminya commented Jan 5, 2023

smitalm commented Jan 24, 2023

aminya commented Jan 25, 2023

This comment was marked as off-topic.

denyeo commented Jun 12, 2023 • edited Loading

Bartel-C8 commented Jun 12, 2023

aminya commented Jun 12, 2023 • edited Loading

Bartel-C8 commented Jun 13, 2023

aminya commented Jun 13, 2023

Bartel-C8 commented Jun 13, 2023 • edited Loading

aminya commented Jun 13, 2023 • edited Loading

denyeo commented Jun 14, 2023

aminya commented Jun 25, 2023 • edited Loading

aminya commented Jun 17, 2024 • edited Loading

Status update

aminya commented Aug 13, 2024

aminya commented Nov 17, 2022 •

edited

Loading

aminya commented Nov 18, 2022 •

edited

Loading

ni-mkrieg commented Nov 18, 2022 •

edited

Loading

aminya commented Nov 21, 2022 •

edited

Loading

bfan1256 commented Nov 22, 2022 •

edited

Loading

td-krzysiek commented Nov 22, 2022 •

edited

Loading

denyeo commented Jun 12, 2023 •

edited

Loading

aminya commented Jun 12, 2023 •

edited

Loading

Bartel-C8 commented Jun 13, 2023 •

edited

Loading

aminya commented Jun 13, 2023 •

edited

Loading

aminya commented Jun 25, 2023 •

edited

Loading

aminya commented Jun 17, 2024 •

edited

Loading