deprecate public export of test_utils

[alamb]

The test_utils package has a non trivial public API (e.g. it exposes rand) even though the code is fairly simple

It would be nice to make this crate more self contained and thus I think we should deprecate and eventually remove test_utils from the public API

          > > It may just be better to avoid using the arrow bench_utils at all.

+1

But it's also a good idea to merge this.

What do we think about merging this and marking all of bench_utils as deprecated (pointing out that external users should just copy the code into their repo if they want to use them) 🤔

Originally posted by @alamb in #7128 (comment)

[klion26]

@alamb I can help to implement this, does this only need to add something like below in arrow/src/util/mode.rs

#[deprecated(
    since = "54.3.2",  <-- the current version is `54.3.1`, so use `54.3.2` here, can be changed if needed
    note = "The `test_util` module is deprecated and will be removed in a future release."
)]
pub mod test_util;

If this is the right way, I will file a pr for it, and if other things need to be done, please give some hints, and I'll adopt them. thanks.

[alamb]

@alamb I can help to implement this,

Thanks @klion26 ❤

Does this only need to add something like below in arrow/src/util/mode.rs

#[deprecated(
since = "54.3.2", <-- the current version is 54.3.1, so use 54.3.2 here, can be changed if needed
note = "The test_util module is deprecated and will be removed in a future release."
)]
pub mod test_util;
If this is the right way, I will file a pr for it, and if other things need to be done, please give some hints, and I'll adopt them. thanks.

Yes I think this is the right way. The since version should be 55 in this case

• Release arrow-rs / parquet major version 55.0.0 (Apr 2025) #7084

[rluvaton]

I believe tests utils have a lot of value, I would love to have a crate for generating arrays and record batches for easy testing and benchmarking

[alamb]

I believe tests utils have a lot of value, I would love to have a crate for generating arrays and record batches for easy testing and benchmarking

Perhaps you can comment on #7375

The reason this got brought up actually was actually some tests I think you may have added in DataFusion that used the test utils. I agree utilities for generating arrays are useful. I think the question is how much benefit we get from having them reused in a library (vs the maintenance cost) compared to, for example, just copying the relevant code where it is needed.

I can see both points of view