Add options to control various aspects of Parquet metadata decoding

[etseidl]

Which issue does this PR close?

• Part of Pass options to Parquet metadata readers #8643.

Rationale for this change

This is a first attempt at an object to help control the parsing of the Parquet metadata.

What changes are included in this PR?

Adds a new MetadataOptions struct, and plumbs it down into the Thrift decoder code. The only option for now is to pass in a schema, which then causes the decoder to skip decoding the schema contained in the footer.

Also adds to the metadata bench to demonstrate the time savings from reusing the schema.

Are these changes tested?

Yes, adds a new test.

Are there any user-facing changes?

If there are user-facing changes then we may require documentation to be updated before approving the PR.

If there are any breaking changes to public APIs, please call them out.

[etseidl]

Here's an excerpt from a run of the new benchmark that shows the schema is actually skipped.

decode parquet metadata time:   [14.401 µs 14.436 µs 14.475 µs]
decode metadata with schema
                        time:   [7.1264 µs 7.1461 µs 7.1677 µs]
decode parquet metadata (wide)
                        time:   [48.440 ms 48.828 ms 49.445 ms]
decode metadata (wide) with schema
                        time:   [43.212 ms 43.452 ms 43.793 ms]

This should get even faster with the metadata index (#8714)

[etseidl]

At some point I could see moving the page index policy into the MetadataOptions and then deprecating a bunch of setters.

[etseidl]

Wondering if this should be Option<Arc<MetadataOptions>> everywhere.

[etseidl]

This may help with #5999

[mapleFU]

Does this means (1) User provided schema or (2) only (min, max, etc) columns in schema_descr be decoded?

[etseidl]

It's (1). Say you have a large number of files that share the same schema, there's no need to decode them all. Just grab the schema from the first file and use it for all the others.

[alamb]

Here is a ticket that explains the use case a bit more;

• Way to share SchemaDescriptorPtr across ParquetMetadata objects #5999

[alamb]

this API looks good to me (and actually closes an existing ticket)

[alamb]

Here is a ticket that explains the use case a bit more;

• Way to share SchemaDescriptorPtr across ParquetMetadata objects #5999

[alamb]

(I didn't approve it b/c it is still marked as a draft)

[etseidl]

(I didn't approve it b/c it is still marked as a draft)

Thanks @alamb. I'm still messing around with the API. I think I like hiding the new options object in the file reader APIs, and just exposing it for the metadata readers. The last wrinkle is figuring out a good way to share across the ParquetMetaDataReader, ParquetMetaDataPushDecoder, and MetadataParser, with an eye towards sucking the page index policy stuff into the new options object.

[etseidl]

Ok, I think this is ready now.

Right now I'm mildly against pulling in the page index policies. They are used at a higher level and I don't think it's worth the thrash to move them. Instead I want to focus on options that impact the FileMetaData parsing (skip stats, transform page encoding stats, etc), and then work in the metadata index to further accelerate the decoding.