Skip to content

Add bitwise ops on BooleanBufferBuilder and MutableBuffer that mutate directly the buffer #8618

@rluvaton

Description

@rluvaton

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I want to change the code in datafusion to use a bitmask rather than slice of booleans for tracking equal group by values so I can do bitwise operations.

And to allow me to nudge the compiler to auto-vectorize code.

Also it will allow me to avoid a lot of copies in combining multiple bitmasks

If I wanted to AND a vector of BooleanBuffer into 1, I would have to do the code below which creates a lot of intermediate copies (one for each AND)

fn and_all_before(buffers: Vec<BooleanBuffer>) -> BooleanBuffer {
    assert!(!buffers.is_empty());

    let mut buffers_iter = buffers.into_iter();
    let mut inter = buffers_iter.next().unwrap();
    for buffer in buffers_iter {
        inter = (&inter) & (&buffer);
    }

    inter
}

If I could use BooleanBufferBuilder I could only have single copy:

fn and_all(buffers: Vec<BooleanBuffer>) -> BooleanBuffer {
    assert!(!buffers.is_empty());
    let mut acc = BooleanBufferBuilder::new(buffers[0].len());
    acc.append_buffer(&buffers[0]);
    for buffer in &buffers[1..] {
        acc &= buffer;
    }

    acc.finish()
}

Related issue:

Describe the solution you'd like
Adding AND/NOT/XOR/OR operations to BooleanBufferBuilder and MutableBuffer

I would like to be able to not only do the operation on an entire BooleanBufferBuilder and MutableBuffer but also on a slice of it (only AND between 8 and 64 bit from the mutable buffer)

Describe alternatives you've considered
None

Additional context
None

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementAny new improvement worthy of a entry in the changelog

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions