Workaround for ViewNodes to receive tap events

[hshapley]

While we wait for google/filament#6633 to be approved, ViewNodes can't be interacted with because they are considered transparent objects. I attempted to find a workaround for the time being, and I wanted to post my solution here in case it helps anyone else. As you'll see, it's not a complete solution - it only works for single tap (not other gestures like dragging), and it only has one behavior for tapping the whole node (it doesn't forward the interaction to different views within itself).

Firstly, in my Fragment, when doing SceneView/session setup, I added the following:

val gestureDetector = GestureDetector(context, object : GestureDetector.SimpleOnGestureListener() {
    override fun onSingleTapUp(e: MotionEvent): Boolean {
        return onSingleTap(e)
     }
  })
arSceneView.setOnTouchListener { _, motionEvent ->
    gestureDetector.onTouchEvent(motionEvent)
}

You could override onSingleTapConfirmed instead of onSingleTapUp if you wanted. My onSingleTap method looks like this:

private fun onSingleTap(motionEvent: MotionEvent): Boolean {
    val pickNode = arSceneView.pickHitTest(motionEvent, false).node
    if (pickNode != null && pickNode is ViewNode) {
        pickNode.onTap(motionEvent, null)
        return true
    }
    return false
}

All this code does is forward single tap events to ViewNodes' onTap method. pickHitTest doesn't use Filament picking, which is the core of why this workaround works.
The last step is simply to add an onTap declaration to your ViewNode for whatever custom behavior you want to trigger on tap. Here's a simple example of a ViewNode that toggles something when it's tapped:

myViewNode.onTap = { motionEvent, _ ->
    myBoolean = !myBoolean
    myViewInTheViewNode.visibility = if (myBoolean) View.VISIBLE else View.GONE
}

I have a strong suspicion that I'm doing something wrong or non-ideally here, so I'd love to hear improvement suggestions.
Most importantly, the ability to turn gestures into View interactions within the node's View already exists, in ViewRenderable.dispatchTouchEventToView(node, motionEvent). It can even handle any MotionEvent, such as dragging a slider component. But when I tried calling that method from onTap, it didn't trigger my View's onClickListener. I'm sure there's a way to get the full functionality and I'm missing something simple.

This solution works for me for now, but if I find improvements I'll edit this post. I hope this helps!

[hshapley]

Using this approach, touch events were only going through about 50% of the time. To get it working fully, I had to manually add a collisionShape to my ViewNode. I couldn't even find a way to figure out the size of the ViewNode in meters programmatically, so I had to use hardcoded values for the collisionShape. But with that added, this approach worked. Really hoping that Filament picking for transparents gets supported soon.

[RGregat]

Hey @hshapley I tried your idea on my Project and it worked somehow. The odd part is, that on my Project with the exact same implementation of your idea the onSingleTapUp Event of the GestureDetecture is never called. Only the onDown Event is called and somtimes even twice.

By the way, if someone else want to try this approach, don't get confused with all these GestureDetecture. At least on my project I see a SceneView, Filament and AndroidView GestureDetector. The latter on is correct one!

[slobodanmilinkovic]

Hey guys, I just found this out:

ArModelNode(
    arSceneView.engine,
    PlacementMode.INSTANT,
    screenPosition = this.position,
    followHitPosition = false,
    instantAnchor = true,
).apply {
    loadModelGlbAsync(glbFileLocation = "models/ios.glb")
    name = "iAmSoHappy"
    scale = Scale(x = 0.05f, y = 0.05f, z = 0.05f)
    collisionShape = renderable?.collisionShape
    isEditable = false
    onError = { Toast.makeText(context, it.message, Toast.LENGTH_SHORT).show() }
    onTap = { event, renderable ->
        Log.d(TAG, "onTap event -> ${event.action}, renderable ${renderable?.collisionShape?.size}")
    }
}

Set collisionShape to let the engine know what to look. I have set isEditable to false to prevent scaling and... Yes, we have onTap now! We get also MotionEvent and Renderable.