Would it be possible to integrate pnpm with nixpkgs?

[wmertens]

https://github.com/NixOS/nixpkgs is a very extensive packaging system for Linux, macOS (and even Windows at some point but it bitrotted).

It uses a read-only, hashed store for storing packages, and it has full integration for binary downloads when they are available, so you don't need to build yourself.

I'm wondering if it would be possible to make pnpm use the Nix store for packages instead of the local cache?

This would require:

• store built modules by sha256 hash in the nix store
• get modules from the store using that same hash
• asking the nix store for a binary install if it can find one elsewhere
• ideally, also store combined modules in nix store by hash -> instant installs

as I understand it, you can symlink the entire node_modules folder, so a fully combined node_modules could be hashed and stored in the pnpm-lock file, and then other systems would instantly download the entire directory from a shared cache. Nix takes care of hardlinking identical files.

Would this be hard to implement?

So the advantages would be:

• shared cache for all installs on the same system
• ability to use binary caches for installs across systems (including CI)
• read-only protections

An even deeper integration would build the modules using Nix, so the builds are sandboxed and guaranteed to use only the given dependencies. This is harder, because it requires preparing all build inputs before the build, network access is not allowed. But it would make the builds safer, part of the ecosystem, and make it easy to package Node-based packages.

[zkochan]

I don't have enough info about Nix, so I'd have to learn it first. I did try it in the past, but the learning curve is big.

pnpm was based on ied, which was inspired by Nix.

[yangm97]

Maybe this would be enlightening? While tangential it does a good job explaining some internals of the nix store.

[wmertens]

So there's the derivations which are written in the Nix language, and then you call nix-build on them which generates a .drv file describing the inputs and output hash (hash of the input paths), this is then used to build the output path.

So, one approach would be to make a derivation for every package that you need, check if the result already exists in the store, and if not, first build the package normally and then use that build as an input for the derivation.

Once you have the Nix store path, it's just another cache, just in another location.

So I think the missing parts are translating the package to a derivation with inputs, and then using the nix store as cache.

[VanCoding]

@wmertens I also thought of building a nix-native version of PNPM that uses the Nix-Store to store packages. The problem: Hard-linking between derivations is not possible, and PNPM makes heavy use of this.

Without hardlinking support, the installation process would take much longer I guess and it would take much more disk space (at least until the nix-deamon comes along and cleans it up).

So I guess it wouldn't be a very good idea to use the nix store for this at the moment.

edit
There's a way that could work without hardlinks! The PnP API could be used to work around this. Using this, packages can just be stored in the nix store without any links to their dependencies. To wire them up, a .pnp.cjs can be created in the derivation that consumes all the packages. Before wiring them up, all the dependencies are completely independent and don't know anything about each other.

[FliegendeWurst]

I recently updated pnpm2nix to support pnpm 10: https://github.com/FliegendeWurst/pnpm2nix-nzbr

At the moment it allows caching dependency downloads (tarballs), and runs pnpm install in every build. (pnpm2nix also had functionality for caching node_modules previously, but that didn't work when using pnpm 10 with workspaces and wasted a lot of disk space.)