Skip to content

Conversation

@moeyensj
Copy link
Member

@moeyensj moeyensj commented Sep 11, 2025

  • 265e1b3 (Add from_kwargs benchmarks)
  • 4297dad (Add a fast path to Column._load when data is a pyarrow chunked array or standard array and the dtypes match)
    • Enhanced Column._load to skip Arrow cast when data.type == self.dtype.
    • Impact: Cuts unnecessary casts for flat columns during from_kwargs, reducing Arrow compute overhead observed in profiles.
  • 5f412fa (Implement SubTableColumn._load with fast paths to avoid casting when dtypes match)
    • Added SubTableColumn._load to normalize inputs into StructArray:
    • Accepts matching quivr.Table via to_structarray(); ensures nullable fields when needed.
    • Returns Arrow StructArray/ChunkedArray as-is if dtype matches; otherwise casts once.
    • Impact: Reduces casting/allocations for nested tables created via from_kwargs, improving constructor performance in nested workloads.

AI-generated summary of AI-generated code.

Benchmarks at 265e1b3:
Screenshot from 2025-09-11 13-44-03

Benchmarks at 4297dad:
Screenshot from 2025-09-11 13-44-34

Benchmarks at 5f412fa:
Screenshot from 2025-09-11 13-45-00

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants