Migrate @turf/buffer to TypeScript

[mfedderly]

Instead of messing around with @turf/jsts needing types, there's actually a new entrant into the underlying library space that solves a lot of problems at once.

clipper2-ts seems pretty exciting for us. I happened to find it because of our desire to drop our jsts dependency, but it also has intersect, union, difference, xor. So perhaps we can rework the other packages to depend on this library (polyclip-ts is slow with its bignumber implementation, and we'd have to fork it if we wanted to drop that). Because it's a port of a mature C library, it already comes with a ready-made test suite.

In the current state of this PR, the test suite fails to run, and 3 of the output fixtures look weird/broken (issue#783, negative-buffer, polygon-with-holes), but I'm pretty sure this is just me not understanding how to use the library.

Here's some numbers I grabbed with the work in progress version. Dropping @turf/jsts saves us ~26 of the total turf.min.js size, and it benchmarks faster (sometimes by a lot!).

before (turf.min.js 537847 bytes)
antimeridian x 54,630 ops/sec ±1.83% (94 runs sampled)
feature-collection-points x 10,855 ops/sec ±0.56% (91 runs sampled)
geometry-collection-points x 16,381 ops/sec ±0.53% (96 runs sampled)
issue-#783 x 19,396 ops/sec ±0.53% (97 runs sampled)
issue-#801-Ecuador x 28,810 ops/sec ±0.63% (96 runs sampled)
issue-#801 x 28,521 ops/sec ±0.80% (98 runs sampled)
issue-#815 x 43,879 ops/sec ±0.62% (94 runs sampled)
issue-#900 x 9,404 ops/sec ±0.54% (99 runs sampled)
issue-#916 x 24,485 ops/sec ±0.61% (95 runs sampled)
linestring x 37,909 ops/sec ±0.72% (98 runs sampled)
multi-linestring x 7,135 ops/sec ±0.65% (98 runs sampled)
multi-point x 22,585 ops/sec ±0.43% (95 runs sampled)
multi-polygon x 13,294 ops/sec ±0.57% (99 runs sampled)
negative-buffer x 47,031 ops/sec ±1.59% (93 runs sampled)
north-latitude-points x 15,976 ops/sec ±1.05% (96 runs sampled)
north-pole x 53,893 ops/sec ±1.08% (97 runs sampled)
northern-polygon x 48,444 ops/sec ±1.27% (91 runs sampled)
point x 62,936 ops/sec ±0.55% (97 runs sampled)
polygon-with-holes x 32,376 ops/sec ±0.60% (97 runs sampled)

after (turf.min.js 333833 bytes (38% reduction) - 302407 bytes without jsbi)
antimeridian x 93,181 ops/sec ±0.39% (98 runs sampled)
feature-collection-points x 30,289 ops/sec ±0.60% (99 runs sampled)
geometry-collection-points x 47,236 ops/sec ±0.36% (98 runs sampled)
issue-#783 x 47,762 ops/sec ±0.53% (93 runs sampled)
issue-#801-Ecuador x 69,575 ops/sec ±0.31% (98 runs sampled)
issue-#801 x 67,595 ops/sec ±0.27% (99 runs sampled)
issue-#815 x 65,772 ops/sec ±0.32% (98 runs sampled)
issue-#900 x 10,281 ops/sec ±0.43% (95 runs sampled)
issue-#916 x 43,918 ops/sec ±0.46% (97 runs sampled)
issue-2522 x 4.06 ops/sec ±0.61% (15 runs sampled)
issue-2929-2 x 47,270 ops/sec ±0.28% (98 runs sampled)
issue-2929 x 46,996 ops/sec ±0.51% (100 runs sampled)
linestring x 56,923 ops/sec ±0.33% (95 runs sampled)
multi-linestring x 12,823 ops/sec ±0.44% (93 runs sampled)
multi-point x 64,107 ops/sec ±0.42% (97 runs sampled)
multi-polygon x 23,674 ops/sec ±0.25% (93 runs sampled)
negative-buffer x 78,203 ops/sec ±0.87% (96 runs sampled)
north-latitude-points x 61,523 ops/sec ±0.36% (96 runs sampled)
north-pole x 109,313 ops/sec ±0.32% (91 runs sampled)
northern-polygon x 97,372 ops/sec ±0.33% (99 runs sampled)
point x 171,341 ops/sec ±0.56% (95 runs sampled)
polygon-with-holes x 60,900 ops/sec ±0.32% (94 runs sampled)

// Note: the above benchmarks were done with native BigInt, and not the patched JSBI BigInt implementation
// The two BigInt methods barely show up in profiles, so I doubt jsbi is going to move the needle too much

Fixes #2929
Fixes #2920
Fixes #2895
Fixes #88
Fixes #2522
Fixes #2701
Fixes #2469 (Although I'm not really convinced that we create a usable polygon at the pole)

[bratter]

FWIW this would be amazing if it works out for both the buffer and the clipping operations! On the off chance if there is any support you would need for this, let me know.

[bratter]

Spent some time taking a look at just the intersect operations today. Couple of insights if you haven't got there already:

1. It appears as though the winding order clipper2 uses is counterclockwise, therefore not requiring the inversion you suggested above. It seems to be in clippper2 Delphi/C++ docs - see the "Clipping Closed Paths" section, and also seems to be true of my experimentation with turf. That will take away a some of the transformation required.
2. Expected given my understanding of the algorithm, but it seems to operate on the integer part of the numbers only. The test suite also only tests integer inputs. So at least WGS84 coords would require scaling, but probably better to do some form of projection like is done here for buffering. Makes it the same non-trivial amount of work to do the conversion here. In my (janky) exploration I just multiplied decimal degrees by some large number and it got pretty decent.
3. It is VERY fast for intersections. The two benchmarks in the suite already are on the simple side, but this recovers more than the ~10x lost in the bignumber move. See below for the benchmark results that I got, noting that it was done on a non-feature complete implementation, so will lose a decent chunk of time (e.g., no projection yet) to that.
4. There is very likely to be precision-related issues given point (2). I didn't play around too much, but did still have issues getting exact matches even to 6 d.p. Will need exploration, and maybe at least coming to a tentative conclusion on the whole precision topic given the magnitude of this particular change.
5. I really like the idea of dropping both jsts and polyclip-ts.

For reference the aforementioned benchmarks:

Current 7.3 polyclip-ts:
turf-intersect#simple x 3,256 ops/sec ±0.58% (97 runs sampled)
turf-intersect#armenia x 2,410 ops/sec ±0.48% (97 runs sampled)

My clipper replacement (but as noted above, anything final will be slower than this:
turf-intersect#simple x 216,990 ops/sec ±0.23% (98 runs sampled)
turf-intersect#armenia x 170,644 ops/sec ±0.26% (97 runs sampled)

[mfedderly]

@bratter

re: winding order, I think it still needs the reversal step because latitude is inverse of the cartesian Y axis. areaD returns positive area with the outer rings with the code as is. This stuff kinda makes my brain hurt, so I may be wrong.

re: integer vs float64, I intended to use the float64 variants before but got it wrong in the first commit. I added the D suffix today to all of the calls. My output fixtures are now visually the same (except I think clipper2 adds more points along the rounded sides with clipper's default settings). I also had to add handling for invalid winding orders as seen in polygon-with-holes's inner ring.

turf.min.js's linting does complain about the bigint syntax, which I think I'm not transitively using, but gets retained by the build instead of being treeshaken. We may have to do a major rev to be able to change that to some newer syntax (probably es2022). Perhaps when we do this, we just go ahead and document that we may roll forward and use any syntax that caniuse declares as "baseline" or another metric that we can use with babel's preset-env.

[bratter]

@mfedderly

Winding order
Think I worked it out. Indeed both winding orders are the same. The clipper docs do explicitly mention this elsewhere where they state: "Positive winding paths will be oriented in an anti-clockwise direction in Cartesian coordinates (where x coordinate values increase toward the right and y coordinate values increase upward). However, in graphics display libraries that use an inverted Y-axis, Positive winding paths will be oriented clockwise." This is the same as lng/lat.

HOWEVER - given that you are also projecting in your code the projection (which, apropos of the quote, is indeed from a graphics library :-)) it is the projection that does as you say and inverts the y and therefore requires the winding reversing that you are doing. I was playing with clipping, so was not projecting, hence experiencing the opposite.

So in summary - clipper and geojson wind the same, but clipper and a d3 geoAzimuthalEquidistant projection wind opposite. Therefore - no reversal required when using geojson, reversal is required when running through a projection.

Integer vs Float
Unfortunately Clipper2 does all its operations on ints. Even the *D versions just do some intermediate processing then delegate to the *64 versions. You can see there here for buffer where it just scales and delegates.

I think this means that the bigint syntax will be a requirement for clipper, so definitely major release, but as you point out it has been baseline for ages. Won't comment on the broader policy question.

[smallsaucepan]

I think this means that the bigint syntax will be a requirement for clipper, so definitely major release, but as you point out it has been baseline for ages

For the es5 compatible cdn version we have (I think) been able to transform and polyfill the bigint code in #2997.

And if someone is importing the individual package esm / cjs directly into a web page, any browsers not supporting bigint will be under the > 0.25% / not dead criteria.

For the node side of things, I think we're on safe ground to say we can start depending on a library that utilises ecmascript bigint. Node18 supports that at runtime, and that's the oldest node version we support.

[mfedderly]

From looking at this, I'm kind of skeptical that we can safely do lo64, hi64 without loss of precision, because Number can only contain 2^53 – 1. 🤔

[mfedderly]

Perhaps this should be a devDependency? And its a touch sketchy to only do this here because we're actually using the patched version of clipper2-ts in the @turf/buffer tests. That means that consumers would be the first folks to test this with the real BigInt syntax.

[mfedderly]

@smallsaucepan curious for your take on how you like the pnpm patch for clipper2-ts (most recent commit on this PR). I think this is probably simpler than trying to use the babel transform since it didn't work out of the box.

Edit: Its probably worth noting that I generated the out test fixtures with builtin BigInt, and then they didn't need to change with the jsbi'd clipper2-ts patch. So I guess that's a good sign.

[mfedderly]

(I know this doesn't build yet, but I wanted to check the generated test outputs and its easier to push a failing build. The necessary porting is done I think 🙏 )

[mfedderly]

@smallsaucepan @bratter I think this is ready for input from ya'll. Happy for feedback over at https://github.com/mfedderly/geoclipper2 as well.

[mfedderly]

crossProductSign64 is just a standard BigInt => JSBI conversion

[mfedderly]

isCollinear is a much more involved patch. When I profiled the bench script with native BigInt, the BigInt path barely showed up. When using jsbi it dominates the profile and speeds are about half of the speed of the jsts implementation. By just making sure we don't overflow on the multiply, we can usually get by without needing to use the jsbi path at all, and speeds return to faster-than-jsts.

I'll probably try to pull the BigInt avoiding path into the upstream eventually.

[mfedderly]

This patch only needs to exist until we can add native BigInt support, perhaps this summer with the new v8 milestone date.

[mfedderly]

I think I'm proposing just deprecating steps and exposing arcTolerance as long as this migration PR sticks for a while without creating issues. They are two different ways of accomplishing the same thing (limit the number of points along the circle), but arcTolernace is not something we can cross-calculate from steps.

[mfedderly]

All of this switching was required to match the output behavior of the original spec, specifically the nesting of Feature vs FeatureCollection. I suspect this could be made simpler by using some of the fooEach methods, but I think this is clear even though its verbose.

[mfedderly]

I've seen this done with while loops which can be fast if you are only one copy of the world off, but I tried some jsbench and the divide operations implied by the % operator here seemed to be around as fast. For that reason I like this because its less susceptible to performance if you wind up many copies of the world off.

[mfedderly]

I believe that this is the setting that people want control over (#639)

[mfedderly]

This switch is once again verbose, but each of the little details is slightly different around how the project and unproject logic is done, as well as the EndType enum.

[mfedderly]

This implies that we can skip the groupPolygonPaths, but that it is required for MultiLineStrings and MultiPolygons

[mfedderly]

Do we need to simplify MultiPolygons with only one Polygon to Polygon? If so we should remove any of that logic from bufferGeometry and put it in bufferGeometryWrapper instead.