Speed up export, skip xml

[cryham]

Well since it already came up on forum, I think this should be an issue.
https://forums.ogre3d.org/viewtopic.php?p=555617#p555617
Exporting isn't fast.
It's okay for single mesh (not too big) and exporting once a while. Anything more and it's a notable delay.
Best would be already wrting from python to .mesh binary not using xml.

[sercero]

That would be nice, but there should be also an option to export the xml to inspect the resulting mesh and check for errors.

Do you know how to export from phyton to .mesh directly?

[cryham]

Right.
I don't know much about Blender's Python, but isn't it just like regular Python writing to binary file? E.g.
https://diveintopython.org/learn/file-handling/binary-files
So the rest I think would be using OgreNext code as reference for 2.1 format of .mesh file.

[paroj]

So the rest I think would be using OgreNext code as reference for 2.1 format of .mesh file.

that would be a bad idea. OgreNext can load v1 meshes, but Ogre cannot load v2 meshes, so you loose compatibility.

Also, instead of re-implementing the binary format in python, you can just use the C++ implementation through the bindings:
https://ogrecave.github.io/ogre/api/14/working-with-numpy.html

[sercero]

Hey guys, I did some tests to probe the waters before diving in too deep into exporting without going through the XML exporting process.

• SimpleSaxWriter: This is the class used by blender2ogre to write the XML
• MockSaxWriter: A class I created identical to SimpleSaxWriter, but all the methods are a no-op to see how much is the writing of the XML actually slowing us down. (spoilers: its not)

TEST

Exporting a Blenders base Sphere mesh with level 1 SubDiv: 11904 exported vertices

• blender2ogre within Blender SimpleSaxWriter: 6.6 seconds (default blender2ogre, lowest time)
• blender2ogre within Blender MockSaxWriter: 6.4 seconds (~3% speedup)
• blender2ogre from cli SimpleSaxWriter: 1.24 seconds (~5x speedup)
• blender2ogre from cli MockSaxWriter: 0.73 seconds (~8x speedup)

After profiling with cProfile it became clear that writing to the console was a mayor point of contention.
So, I tried the test again removing the progress indicators with the following results:

Progress indicators commented out:

• sys.stdout.write() in mesh.py (line 276)
• bpy.context.window_manager.progress_update() in mesh.py (line 279)

Exporting a Blenders base Sphere mesh with level 1 SubDiv: 11904 exported vertices

• blender2ogre within Blender SimpleSaxWriter (no progress indicator): 0.61 seconds (~10x speedup)
• blender2ogre within Blender MockSaxWriter (no progress indicator): 0.27 seconds (~24x speedup)
• blender2ogre from cli SimpleSaxWriter (no progress indicator): 1.17 seconds (~5.5x speedup)
• blender2ogre from cli MockSaxWriter (no progress indicator): 0.62 seconds (~5.5x speedup)

CONCLUSIONS:

• If you want to avoid touching the code of blender2ogre, the best option is to use the command line to export the meshes.
• For best results you should comment the progress indicators in mesh.py. There are other indicators in skeleton.py, but I think the armature is not usually the worst offender.
• It does not seem to make sense to try an replace the XML exporter just for the sake of speed.

[sercero]

As a result of the profiling, I have already found something to optimize: #199

[paroj]

nice findings! this shows that all optimization should start with profiling.

For me bpy.context.window_manager.progress_update is the slow thing and sys.stdout.write() is largely irrelevant.
The issue here is that someone thought that reporting progress with sub-decimal precision is a good idea, so we update progress for each triangle!
If we switch to updating when progress moved by 1% performance is good: #200

As for the profiling: while introducing MockSaxWriter is a good start, it does not give a fair picture of the expected speedup as one of the major bottlenecks is constructing this string dict here:

blender2ogre/io_ogre/ogre/mesh.py

Lines 323 to 327 in b0c9578

	doc.leaf_tag('position', {
	'x' : '%6f' % x,
	'y' : '%6f' % y,
	'z' : '%6f' % z
	})

where both the dict and the string parts are slow to do per vertex. Without XML, we could keep everything as binary numpy buffers.
However, as we already are sub-second with the progress issue gone, we need to be sure that this is worth the hassle..

[sercero]

@paroj regarding switching to using OGRE directly to create the VertexBuffer, I think that there is some problem when trying to serialize the mesh to disk:

Ogre.MeshSerializer.exportMesh(mesh, "test.mesh", Ogre.Serializer.ENDIAN_NATIVE)
TypeError: descriptor 'exportMesh' for 'Ogre.Ogre.MeshSerializer' objects doesn't apply to a 'Ogre.Ogre.MeshPtr' object

It seems that the function wants a *Ogre::Mesh, but I only have a Ogre::MeshPtr (a sharedptr) and the method getPointer() is deprecated (and missing) and the method get() (from sharedptr) is missing also.

[paroj]

Ogre.MeshSerializer().exportMesh(..

[sercero]

Ogre.MeshSerializer().exportMesh(..

?

[paroj]

note the extra parentheses to create a MeshSerializer instance instead of assuming a static method

[paroj]

apparently this kenshi exporter already implemented skipping XML conversion:
https://github.com/Lucius64/kenshi_io_blender/tree/main/3.6/Kenshi_io_tools/kenshi_blender_tool