Serialization duration varies greatly between obj and stl

[ZephyrRaine]

Hello OpenJSCAD team! First of all thank you for your hard work on this library, this is by far the most promising we encountered for boolean operations on the web.

As we tried implementing different file formats to our tool, we noticed the format has a strong impact on processing times.

    const blob = solidsAsBlob([...finalMeshes], {
      format: outputFormat,
    })
    return blob

When outputFormat is stlb it's way faster than obj, but I can't find the reason for this. Passing triangulate: false (as our mesh should already be triangulated) to the options doesn't seem to change anything either.

Do you have any idea on how we can speed up that process, or what area of obj-serializer could need a PR for this?

Thank you in advance!

[hrgdavor]

my guess is stlb exporter is more optimised. Some formats are easier to make writing more performant than others. Also different people might have been implementing them.

[z3dev]

@ZephyrRaine Welcome!

We don't have performance tests for the import/export functions. It might be interesting but each format has specific features for specific reasons.

It would be great to see some comparison of the performance. What formats have you tried? Can you share some statistics?

[z3dev]

Do you have any idea on how we can speed up that process, or what area of obj-serializer could need a PR for this?

There's only one source file for OBJ serializer; index.js

You are welcome to try some optimizations, but OBJ files are basically text so optimizations may be difficult.

[hrgdavor]

When coding a 3mf serializer I used array insted of += and gained significant performance.

• placing all strings into array bodyArr.push('\n') instead of body += '\n'
• at the end simple bodyArr.join('') produces same string but much faster
• it avoids constant expanding of string (making an ever larger string by creating a new string that has old body and new appended string).
• JS engine is smart enough to calculate new buffer needed to fit all parts when joining with empty string as delimiter (or any delimiter)

[ZephyrRaine]

It would be great to see some comparison of the performance. What formats have you tried? Can you share some statistics?

I'll try to benchmark all formats quickly on my current model. Unfortunately I can't share it but the benchmark will at least give a relative idea!

When coding a 3mf serializer I used array insted of += and gained significant performance.
placing all strings into array bodyArr.push('\n') instead of body += '\n' ...
et the end simple bodyArr.join('') produces same string but much faster as it avoids constant expanding of string (making an ever larger string by creating a new string that has old body and new appended string).

I'll try that on a fork and mesure the improvment! If it works well I'll make a PR, thank you very much!

[ZephyrRaine]

@z3dev here you go! this is quite a big model with successive boolean operations so it's not quite lightweight, but there's definitely something going on with obj.

Export Format Benchmark

amf: 5949.00ms
stl: 4063.00ms
stla: 3854.00ms
stlb: 3662.00ms
dxf: 3722.00ms
obj: 15399.00ms
x3d: 4174.00ms
3mf: 727.00ms
json: 110.00ms
js: 0.00ms
jscad: 0.00ms

I tried your method @hrgdavor, but unfortunately I didn't see any major improvement.
Turns out the culprit was .indexOf(), which apparently has a complexity of O(n). I just replaced it with a Map which has a complexity of O(1), and here are the results :

Export Format Benchmark

amf: 6707.00ms
stl: 4357.00ms
stla: 4475.00ms
stlb: 4090.00ms
dxf: 4217.00ms
obj: 4534.00ms
x3d: 4680.00ms
3mf: 780.00ms
json: 116.00ms
js: 0.00ms
jscad: 0.00ms

I diffed two .obj outputs and got exact matches, so I assume it's working. All tests passing and I created #1408. Thank you for your help, and again, thank you very much for the awesome work on this library. 🙏

[hrgdavor]

did oyu try to combine this with array join instead of body += ... maybe it can give some more perormance ?

[ZephyrRaine]

@hrgdavor when I did that it actually added a bit more rather than reduce the time (negligible, around 1000ms)... I was quite surprised because your suggestion seemed very reasonable to me?! I just assumed there were some underlying js optimization I did not know about and moved on to removing indexOf haha.

[hrgdavor]

sure thing. Optimization must be proven, it sounding good is not enough, that is why I asked if you tried. it may or may not help :). Thanks for doing this optimisation.

[z3dev]

@ZephyrRaine

By the way, V2 changes are released periodically so these changes will probably be released in January. Is that ok?

[ZephyrRaine]

Thank you @z3dev, we'll maintain a fork in the meantime! 🙏