Fails to open 1.5G Firefox profile data (JSON) on 32GB machine

[lyind]

Summary

The program unfortunately failed to open a 1.5G JSON file on a 32GB machine.

While processing the file, the CPU stayed at 100% until system memory (32GB) filled up and janice got OOM killed at rendering phase (3/3), at about 90% done.

[ErikKalkoken]

On what OS are you running?

One possible solution is to increase your swap memory in case it is not yet configured or too small. I would suggest to try with 2 X your RAM, so 64 GB of swap.

[lyind]

Linux on amd64.

I observed memory use in the terminal and it ate up all the ~20 gigabytes of RAM and then started to push (other processes) into swap, before getting OOM-killed.

Didn't look into the code to actually check what methods and structures janice uses to index the JSON file, but leaving this here to indicate that there is improvement potential in the memory direction.

Unfortunately, I can't share the JSON file in question. One can easily produce one though, using full profiling (all sources enabled) with the Firefox profiler:
https://profiler.firefox.com/

[ErikKalkoken]

Did you try with swap? How much have you configured?

[lyind]

8 GB swap. However, it wasn't really used. The memory use of the program grew gradually (with progress) and appeared to be actual working memory, not theoretical virtual memory required by the program (e.g. from oversized allocations or mmap'ed files).