Add record batch adapter to zero-copy stream record batches into csp #541
Conversation
arhamchopra
force-pushed
the
ac/record_batch_adapter
branch
from
0da8c1a to
643546c
Compare
Signed-off-by: Arham Chopra <[email protected]>
arhamchopra
force-pushed
the
ac/record_batch_adapter
branch
from
643546c to
395a316
Compare
| } | ||
| else | ||
| { | ||
| auto last_time = m_tsArray -> Value( m_numRows - 1 ); |
There was a problem hiding this comment.
Can we possibly use std::upper_bound for this logic? Same with std::find_if for the small batch case
There was a problem hiding this comment.
I don't think arrow arrays have iterators. We would need to define a forward iterator class to use the std::upper_bound and std::find_if methods.
There was a problem hiding this comment.
m_tsArray is contiguous right? Wonder if we could create a std::span of it and then we wouldn't need to define iterators ourselves
There was a problem hiding this comment.
Not sure if we can rely on data being contiguous. I would expect Arrow to expose the iterator interface.
|
|
||
| long long findFirstMatchingIndex( DateTime time ) | ||
| { | ||
| // Find the first index with time equal or greater than `time` |
There was a problem hiding this comment.
Same idea, could we use std::lower_bound here rather than implementing our own binary search?
| { | ||
| case ::arrow::TimeUnit::SECOND: | ||
| { | ||
| m_multiplier = 1000000000; |
There was a problem hiding this comment.
These constants are defined here
Line 20 in d22137c
No description provided.