Performance Gap Between GraalJS and V8 on Octane #879
Description
Hi there,
I've recently come across some older talks about Truffle and GraalVM, and I find the concept fascinating. As part of my exploration, I wanted to see if it's really possible to achieve performant programming language implementations with GraalVM and the Truffle framework and decided to benchmark GraalJS against V8.
However, the performance results I observed seem to diverge significantly from the claims I’ve seen (on par with v8). I followed some of the recommendations discussed in older issues, such as:
What I did:
- Enforced warmup for every benchmark.
- Increased warmup time from 1 second to 60 seconds.
- Increased warmup iterations from 32 to 512.
My Environment:
- V8 version:
~/playground/v8/out/x64.release/d8
> V8 version 13.3.0 (candidate)
- GraalJS version:
~/playground/graaljs-24.1.1-linux-amd64/bin/js --version
> GraalVM JavaScript (Oracle GraalVM JVM 24.1.1)
Benchmark Results:
Below are the results I obtained. As you can see, the performance gap between GraalJS and V8 remains significant across most of the benchmarks. I'm wondering whether the performance differences between V8 and GraalJS have changed recently, or if I might have missed something in my setup.
Disclaimer: I'm not particularly familiar with either JavaScript or Java, but I'm genuinely interested in the performance that Partial Evaluation can achieve with a simple interpreter. So, please forgive me if I made any silly mistakes in the benchmark.
| Benchmark | GraalJS Score | V8 Score | % Difference |
|---|---|---|---|
| Richards | 18891.0 | 19490.0 | -3.1% |
| DeltaBlue | 37182.0 | 60438.0 | -38.5% |
| Crypto | 14864.0 | 25040.0 | -40.6% |
| RayTrace | 84150.0 | 55877.0 | +50.6% |
| EarleyBoyer | 27718.0 | 44941.0 | -38.3% |
| RegExp | 4542.0 | 5596.0 | -18.8% |
| Splay | 15735.0 | 20900.0 | -24.7% |
| SplayLatency | 7104.0 | 65056.0 | -89.1% |
| NavierStokes | 23120.0 | 24647.0 | -6.2% |
| PdfJS | 22671.0 | 42284.0 | -46.4% |
| Mandreel | 17634.0 | 29427.0 | -40.1% |
| MandreelLatency | 39254.0 | 127826.0 | -69.3% |
| Gameboy | 57540.0 | 65042.0 | -11.5% |
| CodeLoad | 6737.0 | 20687.0 | -67.4% |
| Box2D | 61922.0 | 66436.0 | -6.8% |
| zlib | 22520.0 | 62815.0 | -64.1% |
| Typescript | 35717.0 | 95938.0 | -62.8% |
| Octane | 22350.0 | 39145.0 | -42.9% |