Learning Curve of the API

[maxpoletaev]

Don't get me wrong, this is an incredible piece of software, and the platform-specific optimizations and complexity it handles are very impressive - mad respect to everyone involved in building it.

I do, however, find myself struggling with the API's learning curve to the point that it makes me feel very dumb and frustrated every time I want to read a few parquet files.

Currently, I rely exclusively on pqarrow/*_test.go files as my primary reference to understand usage patterns, which doesn't seem like the proper place to learn. The godoc, while being comprehensive, covers so many types and subpackages that it's hard to piece together a clear picture of how everything fits together.

I suspect other people might be hitting the same wall. The gap between understanding the API documentation and actually implementing something feels unusually wide here compared to other packages in the Go ecosystem.

I thought that maybe adding some foundational examples in the README or the main godoc page could make a huge difference in helping devs get over that initial hump and make the project appear more approachable.

[zeroshade]

Thanks @maxpoletaev, you're definitely right here.

We're aware of the need for improving the docs and examples here, and the issue is honestly two-fold:

1. Resources and time to dedicate to creating and writing examples
2. Myself and others are too close to the material. I actually find it difficult myself to figure out which aspects of the API specifically need more example coverage.

If you could contribute any PRs with examples, that would be amazing. Otherwise, I invite you to please add comments here describing any specific scenarios you ran into that were difficult to figure out. It would significantly help with dedicating resources to creating examples for known situations that devs have trouble with.

Either way, thanks for bringing this up! I'll see if I can find some time to work on this, and will happily review any PRs that get filed.

[zeroshade]

Just had another idea, I've been wanting to simplify the API a bit by adding generic interfaces. This would hopefully make it much easier to interact with this lib.

Again, prioritizing where to start this effort will come down to hopefully getting examples from users as to what the rough edges and difficulties in the current API.

Looking forward to any responses here!