fix: tooltip data "undefined" when using nearest hit testing

[michaelwallabi]

Custom tooltip handlers receive string "undefined" instead of data values in v6.4.0

Description

In vega-lite 6.4.0, custom tooltip handlers (set via view.tooltip()) receive objects where some field values are the string "undefined" instead of the actual data values. This regression was introduced as part of: #9678.

This PR includes a potential fix and some tests. This is my first time digging in to vega-lite details so very possible this isn't the right fix.

Repro

tooltip-no-repro-6.3.1.html
tooltip-repro-6.4.0.html

Expected Behavior

The tooltip handler should receive an object with actual data values:

{
  date: "2024-01-01",
  revenue: 1200,
  category: "Product A"
}

Actual Behavior (v6.4.0)

The tooltip handler receives an object where some values are the string "undefined":

{
  date: "2024-01-01",
  revenue: 1200,
  category: "undefined"
}

Key Details

The issue occurs specifically with:

• Custom tooltip handlers set via view.tooltip()
• Use of hover parameters

The default Vega tooltip rendering appears to work correctly, but custom handlers receive corrupted data.

[basbroek]

@michaelwallabi Thanks for finding this bug and creating such comprehensive test cases! As the author of the PR that seems to have introduced the bug I feel responsible and am eager to look into this together.

Although it seems to fix the problem I'm not quite sure if returning null mitigates the root cause as this part of the VG-signal \"\"+datum[\"category\"] is the same in the newLine PR (formatted in this line) as before (formatted in here).

As you point out, the problem (only) seems to occur when using hover parameters. I think this is because the signal for the voronoi layer should reference datum.datum.category instead of datum.category (see this modified version of the Vega spec of your example) but the code that generates that signal is never reached because of the shortcut in addLineBreaksToTooltip.

Do you agree? If so, any ideas for an elegant solution?

[michaelwallabi]

Thanks for responding @basbroek.

Agree, the fix here is just papering over a deeper issue. Appreicate the pointers, will take a look and see if I can come up with something better.

[michaelwallabi]

Found a better/simpler fix.

Can't say I understand the difference between datum and datum.datum but this seems consistent with the intent of the expr arg.

My understanding is that the key nuance here is that when using "nearest" tooltips a fully transparent voronoi diagram is used as a hit target for the actual marks. It's adding this voronoi layer that requires using datum.datum to get at the actual value. Is this indirection, voronoi -> actual mark, the extra lookup in the tooltip implied by the additional .datum, so without that it's looking up a value in the voronoi which doesn't exist?