Question: Luma v9 way to use color picking

[Aystein]

I did not found another channel for questions, so I created this issue as it may be interesting for other people too.

I am trying to implement object picking using the inbuilt picking module of luma.gl, (https://luma.gl/docs/api-reference/shadertools/shader-modules/gpu-picking#usage) but the documentation is not quite clear on how the actual picking process works.

The example that uses picking (instanced cubes) seems to not do anything with it (no highlight). I tried something similiar

      this.shaderInputs = new ShaderInputs({
          picking,
      });
      this.picker = new PickingManager(this.device, {
          // @ts-expect-error example
          shaderInputs: this.shaderInputs,
      });

and then after my main pass

        this.picker.clearPickState();
        const pickingPass = this.picker.beginRenderPass();
        model.draw(pickingPass);
        pickingPass.end();

        const info = this.picker.getPickInfo([mouseX, mouseY]); // mouseX, mouseY are pixel coordinates from top-left

which does not return anything and throws this warning

my vertex shader has this as last line
picking_setPickingColor(vec3(0.5, 0.5, 0.5)); // some constant color for testing purposes

and my fragment shader
picking_filterPickingColor(fragColor);

Is there a working example or a documentation page which could help me resolve this issue?

[filonik]

The instancing showcase example (which is the only example that has picking afaik) is broken for me on Windows 10 with Firefox 143.0.1 and Chrome 140.0.7339.185. Neither the WebGL nor the WebGPU version work for me. Which seems bad, since it is featured on the home page. Somehow it works on MacOS...

Overall, Luma v9 seems in a pretty rough state since adding WebGPU, which is a shame.

[ibgreen]

Overall, Luma v9 seems in a pretty rough state since adding WebGPU, which is a shame.

Yes, my apologies, the website examples in particular took a big hit with the WebGPU upgrade, and it obviously detracts from the impression.

• I have definitely not given up on luma, but my available time is intermittent right now.
• Fixes to some of the issues have already landed on master, together with additional unit tests to ensure they don't repeat.
• But in addition there have been other (build/environment type) complications preventing the website update.

However the situation is not without hope, as ChatGPT makes root causing and fixing these issues much easier than it used to be...

[filonik]

That's fair. Hopefully it didn't come across the wrong way. Luma has a tremendous amount of potential, and I am grateful for everyone that contributed to it. I wish I understood the library well enough, so that I could fix some of the problems myself and submit pull requests.

The color picking feature in particular is very cool. Not many "micro frameworks" provide that kind of functionality out of the box, since it requires quite a bit of machinery (shadertools, multiple render targets, etc.). Unfortunately, it comes at the cost of complexity.

I suppose Luma is in a bit of a similar place like Python was when Python 3 came out. Hopefully it will get through the painful transition phase soon!