Mesh Weight Using Wrong Joint Id

[AMZN-Gene]

Apple's toy drummer asset does not load the correct mesh joint weights

Repro

1. Download Apple's toy drummer
   toy_drummer_idle.zip
2. Open it in Blender and select the drum. Note the mesh name: polySurface223
   
3. Notice "drum_joint" joint controls the drum (polySurface223)
   
4. Load toy_drummer_idle.usdz using TinyUSDZ using the provided code which,
   a. Finds and prints the joint id for the joint named "drum_joint": 8
   b. Finds and prints the "polySurface223" mesh skinned mesh weights

#include "tinyusdz.hh"
#include "str-util.hh"
#include "io-util.hh"
#include "tydra/scene-access.hh"
#include "tydra/render-data.hh"


int main(int argc, char** argv)
{
    const std::string filePath = "C:/Users/genewalt/Downloads/toy_drummer_idle.usdz";
    tinyusdz::USDLoadOptions options;
    tinyusdz::Stage stage;

    tinyusdz::tydra::RenderScene render_scene;
    tinyusdz::tydra::RenderSceneConverter converter;
    tinyusdz::tydra::RenderSceneConverterEnv env(stage);
    std::string usd_basedir = tinyusdz::io::GetBaseDir(filePath);
    env.set_search_paths({ usd_basedir });
    tinyusdz::USDZAsset usdz_asset;
    std::string warn, err;
    LoadUSDZFromFile(filePath, &stage, &warn, &err, options);
    tinyusdz::ReadUSDZAssetInfoFromFile(filePath, &usdz_asset, &warn, &err);

    tinyusdz::AssetResolutionResolver arr;
    tinyusdz::SetupUSDZAssetResolution(arr, &usdz_asset);
    converter.ConvertToRenderScene(env, &render_scene);


    // Iterate all the joints in the hierarchy to find the drummer bone
    int drumJointId = -1;
    std::vector<const tinyusdz::tydra::SkelNode*> skeletonNodes;
    skeletonNodes.push_back(&render_scene.skeletons[0].root_node);
    for (int i = 0; i < skeletonNodes.size(); ++i) {
        const tinyusdz::tydra::SkelNode* skeletonNode = skeletonNodes[i];
        if (skeletonNode->joint_name == "root/drum_joint") {
            printf("Drum joint name: %s, id %i\n", skeletonNode->joint_name.c_str(), skeletonNode->joint_id);
            drumJointId = skeletonNode->joint_id;
            break;
        }

        for (const auto& child : skeletonNodes[i]->children) {
            skeletonNodes.push_back(&child);
        }
    }

    for (const auto& mesh : render_scene.meshes) { 
        if (mesh.prim_name == "polySurface223") {
            const size_t indexCount = mesh.joint_and_weights.jointIndices.size();
            for (size_t i = 0; i < indexCount; ++i) {
                if (mesh.joint_and_weights.jointWeights[i] == 1.0f) {
                    if (drumJointId != mesh.joint_and_weights.jointIndices[i]) {
                        printf("Drum mesh is using the wrong joint id: %i\n", mesh.joint_and_weights.jointIndices[i]);
                    }
                    else {
                        printf("Drum mesh is using the correct joint id: %i\n", mesh.joint_and_weights.jointIndices[i]);
                    }
                }
            }

        }
    }

    return 0;
}

Expected Results
PolySurface223 mesh should be affected by drum_joint index 8

Drum mesh is using the wrong joint id: 3
Drum mesh is using the correct joint id: 3

Actual Results
PolySurface223 mesh is affected by the wrong joint index: 3

Drum joint name: root/drum_joint, id 8
Drum mesh is using the wrong joint id: 3

[syoyo]

FYI you are also better to use 'tydra_to_renderscene --dumpusd' and open Tydra processed USD in Blender for efficient debugging.

#148 (comment)

[AMZN-Gene]

Thanks for the pro-tip.

tydra_to_renderscene.exe C:/Users/genewalt/Downloads/toy_drummer_idle.usdz --dumpusd

The exported toy_drummer_idle.export.usda in Blender shows the drum vertex is indeed using the wrong joint "arm_lf_end_joint"

[syoyo]

Found multiple(duplicated) Skeleton Prims are exported with RenderScene conversion.

tydra_to_renderscene.exe toy_drummer_idle.usdz --dumpusd

This issue should also be fixed.

[AMZN-Gene]

Found multiple(duplicated) Skeleton Prims are exported with RenderScene conversion.

tydra_to_renderscene.exe toy_drummer_idle.usdz --dumpusd

This issue should also be fixed.

Ah yeah, RenderScene::skeletons.size() == 1 (correctly), but the exported usda ends up containing 1 skeleton per mesh.

[syoyo]

exported USDA contains multiple Skeleton Prims. You are encouraged investing where duplication happens(guess its in Tydra's USDA exporter)

And the fix and PR is also much appreciated.