General slowness and watchman

[cespare]

I've been playing around with jj for side projects but I've decided to try using it in anger by switching to it for my work. I quickly ran into some performance issues and I have some questions.

First, here's the size of the repo I'm working with in a few different dimensions:

▶ du -sh .
3.6G    .
▶ du -sh .git
3.0G    .git
▶ git count-objects -vH
count: 2561
size: 14.00 MiB
in-pack: 1950624
packs: 40
size-pack: 2.86 GiB
prune-packable: 479
garbage: 0
size-garbage: 0 bytes
▶ git ls-files | wc -l
23019
▶ git rev-list --all --count
196463

So I'd consider it medium-sized: much larger than a typical open-source project, but much smaller than the largest git repos out there.

Out of the box, here is how long various operations take in a fresh jj/git clone of this repo. (All timings here are the means reported by hyperfine, and I'm running this on a laptop with an Intel Core Ultra 7 155H.)

op	time
jj status	283 ms
jj status --ignore-working-copy	102 ms
jj log -r 'all()' -n 5	303 ms
jj log -r 'all()' -n 5 --ignore-working-copy	117 ms
git status	48 ms
git log -n 5	2.2 ms

I did some reading and found out about fsmonitor and watchman. I installed Watchman and added fsmonitor.backend = "watchman" and then:

op	time
jj status	168 ms
jj status --ignore-working-copy	100 ms
jj log -r 'all()' -n 5	181 ms
jj log -r 'all()' -n 5 --ignore-working-copy	117 ms

Overall, I expect these kinds of everday vcs interactions to be instant. <50 ms for git feels instant. ~100 ms with --ignore-working-copy or ~170 ms with watchman have a noticeable delay. ~300 ms for jj without watchman or --ignore-working-copy feels like a distinct pause.

So some questions:

• Is this normal/expected? Do other people just live with this? I saw references to crazy multi-second delays in some other threads, so maybe other people have a different level of tolerance for slow tools than I do.
• Are there other things I should be doing to speed things up?
• Is it normal to use --ignore-working-copy a lot for speed? I read the documentation for it in jj help status but I'm not sure I understand all the implications. I can set up some short aliases for things like jj status --ignore-working-copy but I'm not sure if this is a bad idea, or at least not the "jj-ish" way.

Also a separate question about Watchman: is the idea to use watchman as the fsmonitor "backend" indefinitely, or does jj plan to implement its own watcher at some point (a la git's fsmonitor)? I found installing Watchman to be very painful (unlike jj) so the idea of needing to rely on it doesn't give me warm fuzzies.

Finally: is there a mailing list for jj that I somehow missed, or is this the best/only place to have discussions about it?

Thanks for any feedback!

[PhilipMetzger]

• Is this normal/expected? Do other people just live with this? I saw references to crazy multi-second delays in some other threads, so maybe other people have a different level of tolerance for slow tools than I do.

yes, the local working copy which the Git backend uses is at most a local maxima, since it walks the whole repository on every command.

Also a separate question about Watchman: is the idea to use watchman as the fsmonitor "backend" indefinitely, or does jj plan to implement its own watcher at some point (a la git's fsmonitor)? I found installing Watchman to be very painful (unlike jj) so the idea of needing to rely on it doesn't give me warm fuzzies.

There are no definite plans for a watchman replacement but the project at some point would like to have some VFS for itself since it can be specialized for its use-cases.

Finally: is there a mailing list for jj that I somehow missed, or is this the best/only place to have discussions about it?

This and the respective Discord are the best place to discuss jj, there's no mailing list.

[cespare]

I was going to ask where the discord is, but I eventually found it on a random doc page. As far as I can tell, it's the only discord link on the website. Perhaps it should be more prominent?

[PhilipMetzger]

this issue is very related #7101 😄

[steveklabnik]

Yeah we are working on the site/docs a lot recently, and the discord will be linked directly from there in the future for sure.

[yuja]

If the repository has lots of merges, the default jj log would be much slower than git log because

1. jj log has to fetch more revisions to reorder them and render the graph, and
2. jj log computes auto-merged tree to show whether a merge revision is (empty) or not.

(2) can be mitigated by indexing changed paths (see jj debug index-changed-paths --help), but the indexing process will take long.