Fail if coords are outside of a safe range

[sfreilich]

This avoids an assert-fail crash in debug mode, and reads of uninitialized memory and possibly crashes in optimized builds.

[sfreilich]

I still don't understand exactly why this constraint is getting violated, though. But fuzztester seems to find a lot of polylines that violate it, even without getting stuck in NaN-land.

[memononen]

I think it is unrealistic that the algorithm is able to deal with max float. The issue with the large floats are that they quickly turn into infs, and then down the line either 0 or inf depending how they are used. The brittleness is in tesedgeIntersect().

To get to the bottom of the issue, I think you should check what the s/t values are after projection, as well as what the sentinel edges are InitEdgeDict().

One solution could be to clamp the input points to a safe range. We should be able to figure out the sane range from tesedgeIntersect().

[sfreilich]

For this particular example, the edge that violated that assert had:

e->Org->s,t: 1.000000,0.000000, e->Dst->s,t: 0.594775,0.000000

That is, Org->s > Dst->s and it's not looking at t.

You're right that those sentinel edges are overflowing. The projection vectors for those are:

e->Org->s,t: inf,-2.010000, e->Dst->s,t: -340282346638528859811704183484516925440.000000,-2.010000
e->Org->s,t: inf,1.010000, e->Dst->s,t: -340282346638528859811704183484516925440.000000,1.010000

The code in InitEdgeDict which handles this is:

w = (tess->bmax[0] - tess->bmin[0]) + (TESSreal)0.01;
h = (tess->bmax[1] - tess->bmin[1]) + (TESSreal)0.01;

smin = tess->bmin[0] - w;
smax = tess->bmax[0] + w;
tmin = tess->bmin[1] - h;
tmax = tess->bmax[1] + h;

That seems equivalent to:

smin = 2 * tess->bmin[0] - tess->bmax[0] - (TESSreal)0.01 ;
smax = 2 * tess->bmax[0] - tess->bmin[0] + (TESSreal)0.01;

Are you suggesting clamping the values generated for the projection vectors in tessProjectPolygon? (Clamping to [(FLOAT_MIN + 0.01) / 2.0, (FLOAT_MAX - 0.01) / 2 .0]???) Or are you suggesting clamping the input coords somewhere in tesedgeIntersect? I looked at that, but I'm a bit lost?

[sfreilich]

s is the dot-product of coords and (0, 1, 0). t is the dot-product of coords and (0, 0, 1) or (0, 0, -1). So that implies the same safe bounds for coords as for s and t. I think.

[sfreilich]

And I'd also guess that it might be better to fail in tessAddContour rather than modify values that fall out of that range.

[sfreilich]

Made an attempt at this. It changes some of the previous test-cases to exit with error due to invalid input.

[memononen]

make the per component check a helper function.

[memononen]

I had bunch of pending comments from the other day, and did not realize that, sorry :)

[sfreilich]

Updated, please take a look.

[memononen]

Few nags, should be trivial to fix. Other than that the implementation looks good.

[sfreilich]

I messed up the range in the change description, the bounds are +/- 2^23, not 2^32.