Temp Fix for Slow pnpm install/update/add/remove

[YFengFly]

After using pnpm for a while, it's become unbearably slow.
#7133
In this issue @zkochan mentioned, "pnpm is too slow is too broad","But with this title and description the issue just attracts people with different unrelated issues".After seeing this, I decided to delete all the node_modules directories installed via pnpm, as well as the .pnpm-store, and then ran Get-ChildItem -Recurse | Where-Object {$_.LinkType -eq 'HardLink'} in Windows PowerShell to ensure all pnpm hard links were cleaned up.

I saw a discussion that offered a great method, and I would recommend doing it like this:
https://github.com/orgs/pnpm/discussions/4413

Deleting all node_modules folder and running pnpm store prune

The execution speed improved significantly afterwards. For example, with the store, pnpm install used to take over 30 minutes, but now, without the store, the first pnpm install only takes 6 minutes.
In this case, the slow speed was due to the impact of hard links. After a clean removal, the speed returned to normal.
Hopefully, this method will be helpful.

[rylanharper]

Wow, you just saved me. This simple fix reduced my install time from 10min to less than 1min lol. You are awesome - thank you!

[YFengFly]

I am very glad to help you. (❁´◡`❁)

[zkochan]

This is really unfortunate. If hard links are so inefficient on Windows then should we recommend using copying? Is it getting slow when there are more than a given number of hard links to the same file?

[rylanharper]

Hey @zkochan I'm not OP, but commented above as this helping resolve my pnpm install issues. I'm actually developing on Mac OS!

I actually kept a log of when I ran pnpm store prune:

/Users/rylanharper/Library/pnpm/store/v3
rylanharper ~ pnpm store prune

Removed all cached metadata files
Removed 254137 files
Removed 11618 packages

So it removed A TON of files I no longer used, and reduced my pnpm install times from 10min+ to less than 1min (45sec) for larger projects. I actually wasn't aware that the store has that big of an impact on performance.

[YFengFly]

It appears that different operating systems and file systems may trigger this issue. In my personal case, the problem occurred after having a large number of hard-linked files. Strangely, on another computer with an SSD, this problem has not surfaced yet.

[zkochan]

On macOS we use reflinks by default. I am also on mac and haven't noticed any issues.

Another issue could be that verifying the content of the store is slow but in that case we print an info message:

pnpm/pkg-manager/core/src/install/index.ts

Lines 301 to 304 in c7eefdd

	if (global['verifiedFileIntegrity'] > 1000) {
	// @ts-expect-error
	globalInfo(`The integrity of ${global['verifiedFileIntegrity']} files was checked. This might have caused installation to take longer.`)
	}

[zkochan]

If you could check whether the performance issue is resolved by using package-import-method=copy or package-import-method=clone, it would provide additional information.

[rylanharper]

Not sure why I didn't see this notice message though, when I had over 200k files that were removed when pruning the store. That may seem like an extreme amount, but I update my dependencies on a weekly/monthly basis (the same as most devs) 🤔

I guess something fell through the cracks for me. However, I can confirm that pnpm store prune solved my issue with 10+ min install times.

[YFengFly]

I am also somewhat confused by this issue. The current known information is as follows:

1. Both HDD and SSD may slow down over time;
2. Slowing down can occur on Windows, MacOS, and Linux;
3. Some known configurations associated with slowdowns (collected from issue 7133):
   a) Windows + NTFS + HDD
   b) Debian GNU/Linux 12 (bookworm)
   c) MacBookPro M2
   d) Fedora 40 + SSD
4. Ineffective attempts include:
   a) HDD: Defragmentation
   Perhaps using a Large Language Model (LLM) to explore potential solutions could be a viable approach? (Such as GPT, Claude, Gemini...)

[youyoumu]

i noticed pnpm is much slower than npm on NTFS.

i'm on linux. i have this 1TB SSD with NTFS file system.
pnpm install takes few minutes, but npm install just takes few seconds.

rm -rf node_modules created by pnpm also takes few minutes, but node_modules created by npm just takes few seconds

everything is instantaneous though if i'm on EXT4 with pnpm