areaY stacked problem with pos/neg values

[Hvass-Labs]

When data is plotted using areaY that contains both positive and negative values, it creates very confusing zig-zag lines due to the linear interpolation between pos/neg values. It took me a moment to realize what was happening when I first saw this, and I don't think the average user of my web-site will understand this. So I hope you can remove this artifact. Thanks!

Example: https://observablehq.com/d/a1c4060fcfcc75b4

PS: Your HQ Notebook system is amazing! I just wish it supported Python as well.

[Fil]

Areas are continuous, which explains the problem; so maybe try instead:

• a different mark (for example Plot.rectY(data1, {x: "x", y: "y", stroke: "z", fill: "z", interval: "3 months"}))
• keep the areaY mark, but separate positive and negative series, using an interval to make sure there are 0 values when necessary:

  Plot.areaY(data1, {
      x: "x",
      y: "y",
      fill: "z",
      z: (d) => `${Math.sign(d.y)} ${d.z}`,
      interval: "3 months"
    }),

• keep the areaY mark, separate positive and negative series by duplicating the dataset:

    Plot.areaY(
      data1.flatMap((d) => [
        { ...d, y: Math.max(0.00001, d.y), series: `positive ${d.z}` },
        { ...d, y: Math.min(-0.00001, d.y), series: `negative ${d.z}` }
      ]),
      {
        x: "x",
        y: "y",
        fill: "z",
        z: "series"
      }
    ),

[mbostock]

To add to @Fil’s answer, here is complete working code. I would probably apply the interval option to the x scale rather than the rect mark so that you get an ordinal axis, but it’s really a stylistic preference as to whether you want to draw the bars between the ticks or centered on the ticks (and hence whether you want to represent time as a continuous interval or discrete periods).

Plot.plot({
  x: {interval: "quarter", label: null},
  y: {grid: true},
  color: {legend: true},
  marks: [
    Plot.rectY(data1, {x: "x", y: "y", fill: "z", tip: true}),
    Plot.ruleY([0])
  ]
})

Another simple solution to this problem is to avoid interpolation by using the step curve instead of the default linear.

Plot.plot({
  y: {grid: true},
  color: {legend: true},
  marks: [
    Plot.areaY(data1, {x: "x", y: "y", curve: "step", fill: "z"}),
    Plot.ruleY([0])
  ]
})

[Hvass-Labs]

Thanks for your generous replies!

These are monetary amounts that sometimes switch between positive and negative, so the zig-zag lines would be rather meaningless and impossible to interpret. It is only because I understand that they are artifacts of interpolation that I can try and ignore them. But the ordinary user won't understand that and would be quite confused about the zig-zags.

But I prefer using lines as opposed to bars / steps because I think the lines look very pretty here :-)

@Fil's last suggestion is exactly what I expect to see. I have updated the Notebook to show this desired result: https://observablehq.com/d/a1c4060fcfcc75b4

Ideally this would be done automatically "behind-the-scenes" so I don't have to worry about it. It should also work with brushing / selection of data.

Would it be possible to add this to the areaY function?

Thanks!

[mbostock]

If you like @Fil’s solution, I would recommend a different implementation that is simpler and avoids the need for near-zero values (such as 0.0001). Essentially, this duplicates the stacked area so that there is a positive stacked area (growing upwards) and a negative stacked area (growing downwards), rather than alternating the two.

Plot.plot({
  y: {grid: true},
  color: {legend: true},
  marks: [
    Plot.areaY(data1, {x: "x", y: (d) => Math.max(0, d.y), fill: "z"}),
    Plot.areaY(data1, {x: "x", y: (d) => Math.min(-0, d.y), fill: "z"})
  ]
})

Personally, I prefer the rect or step curve approach to avoid interpolation in this case.

[mbostock]

Nice, yes, that’s a bug. We can make Math.min(-0, d.y) work as intended here. #1967

[Hvass-Labs]

I'm sorry I first saw your updates now. You requested the data in the PR. Please see the updated Notebook: https://observablehq.com/d/a1c4060fcfcc75b4

The hack works better on my local computer where the result is as shown above. I use a beta-build based on v0.6.11 that has the brushing feature.

Please also note that this hack messes with the stroke-color.

I wonder if a better solution might be to handle the switching between pos/neg stacked regions internally in the areaY code, instead of hacking it this way?

[Fil]

Thanks for sharing! It's very interesting to see this dataset, because it has a second peculiarity, which is that two series are labelled B (one with positive numbers, the other with negative numbers). If we sort by x, this results in weird lines such as this one:

This can be solved by grouping with a sum reducer (I think? We don't know what the data mean nor why there are two series, so it's still a wild guess), which works well with #1968:

[Hvass-Labs]

It's very interesting to see this dataset, because it has a second peculiarity, which is that two series are labelled B (one with positive numbers, the other with negative numbers).

Oops! I'm terribly sorry I made a mistake! It is such a hassle to extract this data that I accidentally used the name "B" twice. That also explains why it looked worse than the other plots I made above. I've fixed it in the Notebook and checked it should be correct now. I'll be more careful the next time. But at least you found it interesting so it wasn't a complete waste of your time :-)

[Hvass-Labs]

By the way, these are all monetary amounts that change over time. Most of them can change between pos/neg.

[Fil]

a better solution might be to handle the switching between pos/neg stacked regions internally in the areaY code

Yes, maybe in the stackY transform, where values are grouped by z, they could be further duplicated with alternating zeroes… might be a lot of code overindexing on this particular example, though 🤔

[Hvass-Labs]

Note that the hack with two areaY marks also messes with the stroke-color. The hack is still a satisfactory solution for me and I will use it for now. But from the user's perspective it appears to be an artifact / bug in the plotting code, so eventually it would be nice with a proper solution. For monetary values that can change between positive and negative over time, this would be a common problem.

My naive idea for a solution would be to pre-process the data inside areaY or the stackY transform, and separate it into pos/neg values. Perhaps it is necessary to also insert "phantom" points for zero and negative-zero when the data switches between pos/neg, but I wonder if that would mess with the brushing / selection feature. But I've never written plotting routines so I don't know exactly how all this works :-)