Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test rowgroups #457

Open
nevencaplar opened this issue Feb 20, 2025 · 1 comment
Open

Test rowgroups #457

nevencaplar opened this issue Feb 20, 2025 · 1 comment
Assignees
@hombit
Milestone
Data Preview 1

Comments

@nevencaplar
Copy link
Member

nevencaplar commented Feb 20, 2025 •
edited
Loading

Benchmark how do rowgroups affect loading individual rows from the parquet file. Are there some cons or inefficiencies if we were to use rowgroups to split along index columns (for example, is hats_import slower). How best is make this grouping (in what way, which columns). Report back with results and recommendations.
@nevencaplar
Copy link
Member Author

Connected with #400

@nevencaplar nevencaplar moved this to Todo in HATS / LSDB Feb 21, 2025
@hombit hombit moved this from Todo to In Progress in HATS / LSDB Mar 19, 2025
@nevencaplar nevencaplar added this to the Data Preview 1 milestone Mar 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hombit hombit

Labels
None yet
Projects
HATS / LSDB
Status: In Progress
Milestone
Data Preview 1
Development

No branches or pull requests
2 participants
@hombit @nevencaplar