Performance vs GDScript

[Denneisk]

It was bound to be asked eventually...

Warning

This is NOT a comprehensive benchmark, just a quick little aside for my curiosity.

Everyone knows Lua has a claim to fame for its speed, especially with LuaJIT. This project doesn't use LuaJIT (yet), but I'm still curious, how does its speed fare compared to GDScript? Well, to test this, I went to see if there were any public benchmarks for GDScript that I could use. I chose csharp-gd-inventory-test because it was the first option available and seemed good enough for a quick test. I avoided going for the official Godot benchmarks because those are very in-depth and I didn't really want to get into those weeds right now.

My goal with this code was to simply translate the GDScript code to Lua nearly verbatim, and then optimize that verbatim copy with Lua idioms and speed tricks. The results were pretty nice. On average, optimized Lua performed around 3x faster than GDScript. In a worst case, unoptimized Lua (a verbatim copy of GDScript) runs at least 6x worse here. Another interesting thing is that Lua by itself has some really good cold loop performance, beating C# in these tests on single runs. Whether or not that's coincidence is up to you to decide.

You can see the code I used at this repo. I won't be updating the README to include my results (since I'm lazy), but here's some copy/paste from my own test runs. This was done with an exported project to be as accurate as possible.

This was tested on Godot 4.4.1 on Windows x86_64, with the latest (as of writing) commit of Lua GDExtension 384dc96.

Results

Running Find Space benchmark with 1 iterations:
GDScript time: 0.000034 seconds
C# time: 0.000165 seconds
Lua time: 0.000011 seconds
C# is 0.21 times faster than GDScript
Lua is 3.09 times faster than GDScript
Lua is 15.01 times faster than C#

Running Find Space benchmark with 10 iterations:
GDScript time: 0.000297 seconds
C# time: 0.000015 seconds
Lua time: 0.000072 seconds
C# is 20.48 times faster than GDScript
Lua is 4.13 times faster than GDScript
Lua is 0.20 times faster than C#

Running Find Space benchmark with 100 iterations:
GDScript time: 0.002977 seconds
C# time: 0.000147 seconds
Lua time: 0.000699 seconds
C# is 20.31 times faster than GDScript
Lua is 4.26 times faster than GDScript
Lua is 0.21 times faster than C#

Running Sort Inventory Benchmark:

Running Sort Inventory benchmark with 1 iterations:
GDScript time: 0.000058 seconds
C# time: 0.000024 seconds
Lua time: 0.000020 seconds
C# is 2.46 times faster than GDScript
Lua is 2.90 times faster than GDScript
Lua is 1.18 times faster than C#

Running Sort Inventory benchmark with 10 iterations:
GDScript time: 0.000505 seconds
C# time: 0.000063 seconds
Lua time: 0.000148 seconds
C# is 7.97 times faster than GDScript
Lua is 3.41 times faster than GDScript
Lua is 0.43 times faster than C#

Running Sort Inventory benchmark with 100 iterations:
GDScript time: 0.004462 seconds
C# time: 0.000578 seconds
Lua time: 0.001390 seconds
C# is 7.72 times faster than GDScript
Lua is 3.21 times faster than GDScript
Lua is 0.42 times faster than C#

I'd love to see what LuaJIT could do here.

[gilzoide]

Hey @Denneisk, nice initiative, thanks for bringing this up!

Beware that this optimization you made of using a script local inventory is quite specific to this use case, in most real-world scenarios you would be using an instance property and self.inventory. The "all functions are local" optimization is doable, although not so ergonomic when you have to pass self everywhere.

One interesting variant implementation would be one using an Array (or even a PackedInt32Array) instead of a table, although that would probably make things slower due to more interop with C++.

[Denneisk]

Beware that this optimization you made of using a script local inventory is quite specific to this use case, in most real-world scenarios you would be using an instance property and self.inventory. The "all functions are local" optimization is doable, although not so ergonomic when you have to pass self everywhere.

Yeah, I used a lot of tricks in the Lua code that probably wouldn't scale well, but personally part of why I really like having Lua in Godot is because I'm able to do hacks like that which let me eek out a tiny bit more performance. I should have probably cooked up a better middle-ground case because I admittedly was a little too zealous with micro-optimizations, but after seeing how poorly my first pass of translating the GDScript code ran, I was worried Lua would need all the help it could get.

One interesting variant implementation would be one using an Array (or even a PackedInt32Array) instead of a table, although that would probably make things slower due to more interop with C++.

I thought about that after I finished writing the tests. It's probably worth exploring, but I wanted to stick as discretely to Lua vs GDScript as I could.

[gilzoide]

Yeah, I used a lot of tricks in the Lua code that probably wouldn't scale well, but personally part of why I really like having Lua in Godot is because I'm able to do hacks like that which let me eek out a tiny bit more performance.

Fair enough =D

My point is more about claiming that "On average, optimized Lua performed around 3x faster than GDScript" is fine for your specific use case, but it would be nice to come with a disclaimer that you used non-trivial "speed tricks" that may not be feasible in most use cases. But that's about it, being able to do such tricks is certainly awesome!

---

By the way, I know of an optimization we can already make in the library that will make the general case faster (your verbatim translation, for instance). Right now, for simplicity in implementation, self is a Variant referencing the owner Object. All LuaScriptInstances store their data in a LuaTable, so we can actually pass this table as self instead and attach a metatable to it for accessing owner's properties/methods as fallback. This would make accessing local properties faster because it would not trigger metamethods. I think accessing methods would also be a little bit faster, even if we don't store them directly in this table, but we'd need some measuring on this, I'm not 100% sure 🤔

[gilzoide]

Hey @Denneisk, LuaJIT support was added in the new 0.5.0 release, so you can try it out if you want! There are 2 different zips in the new release, one using Lua 5.4 and the other one using LuaJIT (with Lua 5.2 compatibility enabled). You'll likely not be able to use both in the same project, since they register the same classes and stuff, but you could add a branch in your benchmark repository to test LuaJIT and see if the timing changes. Replacing the addons/lua-gdextension with the one packed with LuaJIT should be enough to change implementations.