Support for preallocated internal buffers

[wlkrm]

Because of its zero-copy properties, flatbuffers might be a good fit for real-time systems.
For real-time systems, memory allocations in critical sections, such as control loops, should be avoided.
Currently in the Rust implementation, I can only preallocate the "data" buffer used for serialization.
This only avoids some allocations at runtime: Some internal vecs of the FlatBufferBuilder are allocated on-demand, which will lead to additional allocations at runtime. It would be useful to be able to preallocate some user-definable space for the internal buffers optionally, as it is already possible for the "data" buffer using the FlatBufferBuilder::with_capacity(usize) function.

Example

// Schema
namespace users;

table User {
  name:string;
}

root_type User;

let mut buffer = FlatBufferBuilder::with_capacity(1024);
buffer.reset();

loop {
        let name = buffer.create_string("testt");
        let user_args = UserArgs { name: Some(name) };
        let test = User::create(&mut buffer, &user_args);
        buffer.finish(test, None);
        let _ = buffer.finished_data();
 }

Proposal

My suggestion would be to simply add a second initializer function besides with_capacity(), which also allocated internal capacity, e.g., FlatBufferBuilder::with_internal_capacity(1024 /* data buffer's len */, 16 /* internal vecs' len */) (Of course with a different len for every vec in the real implementation).
Of course, the user would have to make a sophisticated decision on how much to preallocate, but this is already the case for the "data" buffer size.

What do you think?

See master...wlkrm:flatbuffers:master for the implementation

---
 rust/flatbuffers/src/builder.rs | 41 +++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/rust/flatbuffers/src/builder.rs b/rust/flatbuffers/src/builder.rs
index d8a6e81d..62d239f0 100644
--- a/rust/flatbuffers/src/builder.rs
+++ b/rust/flatbuffers/src/builder.rs
@@ -160,6 +160,26 @@ impl<'fbb> FlatBufferBuilder<'fbb, DefaultAllocator> {
     pub fn with_capacity(size: usize) -> Self {
         Self::from_vec(vec![0; size])
     }
+    /// Create a FlatBufferBuilder that is ready for writing, with a
+    /// ready-to-use capacity of the provided size and ready-to-use capacity for internal vecs.
+    ///
+    /// The maximum valid value is `FLATBUFFERS_MAX_BUFFER_SIZE`.
+    pub fn with_internal_capacity(size: usize, internal_vecs_size: usize) -> Self {
+        Self::from_vec_with_internal_capacity(vec![0; size], internal_vecs_size)
+    }
+    /// Create a FlatBufferBuilder that is ready for writing, reusing
+    /// an existing vector and ready-to-use capacity for internal vecs.
+    pub fn from_vec_with_internal_capacity(buffer: Vec<u8>, internal_vecs_size: usize) -> Self {
+        // we need to check the size here because we create the backing buffer
+        // directly, bypassing the typical way of using grow_allocator:
+        assert!(
+            buffer.len() <= FLATBUFFERS_MAX_BUFFER_SIZE,
+            "cannot initialize buffer bigger than 2 gigabytes"
+        );
+        let allocator = DefaultAllocator::from_vec(buffer);
+        Self::new_in_with_internal_capacity(allocator, internal_vecs_size)
+    }
+
     /// Create a FlatBufferBuilder that is ready for writing, reusing
     /// an existing vector.
     pub fn from_vec(buffer: Vec<u8>) -> Self {
@@ -203,6 +223,27 @@ impl<'fbb, A: Allocator> FlatBufferBuilder<'fbb, A> {
         }
     }
 
+    /// Create a [`FlatBufferBuilder`] that is ready for writing with a custom [`Allocator`] with preallocated internal vecs.
+    pub fn new_in_with_internal_capacity(allocator: A, internal_capacity: usize) -> Self {
+        let head = ReverseIndex::end();
+        FlatBufferBuilder {
+            allocator,
+            head,
+
+            field_locs: Vec::with_capacity(internal_capacity),
+            written_vtable_revpos: Vec::with_capacity(internal_capacity),
+
+            nested: false,
+            finished: false,
+
+            min_align: 0,
+            force_defaults: false,
+            strings_pool: Vec::with_capacity(internal_capacity),
+
+            _phantom: PhantomData,
+        }
+    }
+
     /// Destroy the [`FlatBufferBuilder`], returning its [`Allocator`] and the index
     /// into it that represents the start of valid data.
     pub fn collapse_in(self) -> (A, usize) {
-- 
2.47.3