TST: remove PANDAS_FUTURE_INFER_STRING=0 test workflow

[eicchen]

Now that 3.0 is officially out, the PANDAS_FUTURE_INFER_STRING=0 test pipeline should be retired.

It is currently failing by default due to the release.

[jorisvandenbossche]

The reason for the failures in the test build are because for IO methods that use pyarrow to read the data and convert it to a DataFrame, it is now always using the string dtype (pyarrow does that for pandas >= 3, and only checks the future.infer_string option for pandas 2.3).

A bit annoying, but those could be fixed (or skipped) to have CI green again.

For actually deprecating/removing the option (and cleaning up the pandas code and tests), I think the idea was to leave that for later (pandas 3.1?). The feedback from CoW (for which we already removed the old code on main) was that it can sometimes be easier to still be able to disable it, when debugging an issue with upgrading (cc @rhshadrach). And so therefore for the string dtype, we kept the option working for now.

[jorisvandenbossche]

Separate question of course if we need to keep the test build, if that turns out to give quite some additional work.
My feeling is that if we want to keep the option working for 3.0.x, then ideally we also have a test build. But on the other hand, in practice, the only actual test is about the correct inference in constructors, and then most other changes in the tests are just to account for the exact dtype in the expected result. That is maybe not the most important thing to keep testing.

I.e. we could just have a normal test that disables the option and verifies that the option still does what it should do, which is something we can test in all builds, and don't need a custom build for.

[eicchen]

I definitely think short term we should either patch up the tests or make the workflow non-blocking until it's fixed, so it doesn't clog up PR submissions.

Waiting to deprecate the option is the right call given the recent 3.0 release. One concern with targeted tests, if they need to validate behavior across IO methods, we'd still need conditional logic to handle both string dtype paths, which could complicate pyarrow-based functions. Could we potentially use a decorator or conditional block to make it easier to separate when it comes time to fully remove the parameter?

That said, if the scope is just verifying the option works in constructors, targeted tests seem like a solid middle ground.

[rhshadrach]

I.e. we could just have a normal test that disables the option and verifies that the option still does what it should do, which is something we can test in all builds, and don't need a custom build for.

I'd suggest we keep one build (Linux) with all of the old future options. E.g. if the copy-on-write option still worked, it would be a single build with both copy-on-write and infer_string set to False.

In a similar vein, I'd suggest 3 builds for all new future options that are not yet the default: one for windows, mac, and linux.

I also don't think we should remove options on a minor release, but don't feel so strongly here.