In C API, fix or explain memory management for TFHE data

[rdlopes]

Describe the bug

Creating and destroying TFHE data can lead to surprising crashes.

This includes:

• Config
• ConfigBuilder
• DynamicBuffer
• All keys

To Reproduce

Key generation

#include <assert.h>
#include <stdio.h>
#include "tfhe.h"

int main(void)
{
    ConfigBuilder* builder;
    Config* config;

    assert(config_builder_default(&builder) == 0);
    assert(config_builder_build(builder, &config) == 0);

    ClientKey* client_key = NULL;
    ServerKey* server_key = NULL;
    assert(generate_keys(config, &client_key, &server_key) == 0);

    client_key_destroy(client_key);
    server_key_destroy(server_key);
    config_destroy(config);
    config_builder_destroy(builder);

    return EXIT_SUCCESS;
}

Crashes with

tfhe_c_api(67203,0x20a29a0c0) malloc: Double free of object 0x12a605980
tfhe_c_api(67203,0x20a29a0c0) malloc: *** set a breakpoint in malloc_error_break to debug

Generation of multiple keys

#include <assert.h>
#include <stdio.h>
#include "tfhe.h"

int main(void)
{
    ConfigBuilder* builder;
    Config* config;

    assert(config_builder_default(&builder) == 0);
    assert(config_builder_build(builder, &config) == 0);

    ClientKey* client_key = NULL;
    ServerKey* server_key = NULL;
    assert(generate_keys(config, &client_key, &server_key) == 0);

    ClientKey* second_client_key = NULL;
    ServerKey* second_server_key = NULL;
    assert(generate_keys(config, &second_client_key, &second_server_key) == 0);

    client_key_destroy(client_key);
    server_key_destroy(server_key);
    client_key_destroy(second_client_key);
    server_key_destroy(second_server_key);
    config_destroy(config);
    config_builder_destroy(builder);

    return EXIT_SUCCESS;
}

Crashes with surprising

thread '<unnamed>' panicked at tfhe/src/high_level_api/keys/inner.rs:237:9:
This API only supports parameters for which the MessageModulus is 2 or 4 (1 or 2 bits per block)
stack backtrace:
   0:        0x1028ea788 - std::backtrace_rs::backtrace::libunwind::trace::hc02444a115fdd541
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
...

Expected behaviour

1. Memory management should be done atomically for TFHE data, or we should have documentation for understanding why and when we should use the destroy methods.
2. Key management could be better explained so we know what data can be used, reused, or discarded.

Evidence

Crash logs are provided along with the code.

Here are the crash logs for multiple keys generation (RUST_BACKTRACE=full)

thread '<unnamed>' panicked at tfhe/src/high_level_api/keys/inner.rs:237:9:
This API only supports parameters for which the MessageModulus is 2 or 4 (1 or 2 bits per block)
stack backtrace:
   0:        0x1028ea788 - std::backtrace_rs::backtrace::libunwind::trace::hc02444a115fdd541
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
   1:        0x1028ea788 - std::backtrace_rs::backtrace::trace_unsynchronized::h9f3802572facb192
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
   2:        0x1028ea788 - std::sys::backtrace::_print_fmt::h7dac7a31decff96a
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/sys/backtrace.rs:66:9
   3:        0x1028ea788 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h75d95a49293b41b7
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/sys/backtrace.rs:39:26
   4:        0x102929700 - core::fmt::rt::Argument::fmt::hca6d11d2f60109ea
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/core/src/fmt/rt.rs:179:76
   5:        0x102929700 - core::fmt::write::h0e38a28a73a97af3
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/core/src/fmt/mod.rs:1481:25
   6:        0x1028e0570 - std::io::default_write_fmt::h82aa6e182ce4f95f
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/io/mod.rs:639:11
   7:        0x1028e0570 - std::io::Write::write_fmt::hd4ec7e5fb7a13536
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/io/mod.rs:1914:13
   8:        0x1028ea63c - std::sys::backtrace::BacktraceLock::print::h9fe48addc5e7235f
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/sys/backtrace.rs:42:9
   9:        0x1028ede74 - std::panicking::default_hook::{{closure}}::h16007dca0cdce0c0
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/panicking.rs:300:22
  10:        0x1028edcc4 - std::panicking::default_hook::h561cba65d1918c94
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/panicking.rs:327:9
  11:        0x1028ee9ec - std::panicking::rust_panic_with_hook::hacd2c52cc44f1e21
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/panicking.rs:833:13
  12:        0x1028ee5f0 - std::panicking::begin_panic_handler::{{closure}}::h0b705c4ccbc2a495
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/panicking.rs:699:13
  13:        0x1028eac4c - std::sys::backtrace::__rust_end_short_backtrace::h456a671109989e15
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/sys/backtrace.rs:168:18
  14:        0x1028ee2c0 - __rustc[93dcfddc367fd01c]::rust_begin_unwind
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/std/src/panicking.rs:697:5
  15:        0x102985438 - core::panicking::panic_fmt::ha88b2d8c464fb33f
                               at /rustc/cb31a009e3e735ab08613cec2d8a5a754e65596f/library/core/src/panicking.rs:75:14
  16:        0x102513f04 - <tfhe::high_level_api::keys::inner::IntegerClientKey as core::convert::From<tfhe::high_level_api::keys::inner::IntegerConfig>>::from::hc4be53ec5a521677
  17:        0x101506570 - tfhe::c_api::utils::catch_panic::hb3aebf3f44ebc8fe
  18:        0x102648e28 - _generate_keys
  19:        0x100dfd0f8 - <unknown>
                               at /Users/ruilopes/Developer/rdlopes/tfhe-c-api/main.c:19:5
tfhe_c_api(68123,0x20a29a0c0) malloc: Double free of object 0x143e05980
tfhe_c_api(68123,0x20a29a0c0) malloc: *** set a breakpoint in malloc_error_break to debug

Configuration(please complete the following information):

• OS: Apple M3 Max running on 96 Go RAM and OS Sequoia 15.6.1

Additional context
C API has been generated with the provided Makefile and here is the
CMakeLists.txt

[tmontaigu]

All data created by tfhe-rs has to be manually destroyed

Config and ConfigBuilder are the 2 exceptions to this, in that functions that use them destroy them

This is the case for config_builder_build which in the tfhe.h has a small comment

/**
 * Takes ownership of the builder
 */
int config_builder_build(struct ConfigBuilder *builder, struct Config **result);

Which means that after config_builder_build the builder is destroyed, so you do not need to destroy it AND, you cannot use it anymore.

And its the same for generate_keys which takes ownership of the input config and deletes it. Here the comment about that is completely missing.
client_key_generate has the proper documentation`

/**
 * Generates a client key with the given config
 *
 * This function takes ownership of the config,
 * thus the given config pointer should not be used/freed after.
 */
int client_key_generate(struct Config *config, struct ClientKey **result_client_key);

So in your first and second example, you have to remove calls to

   config_destroy(config);
    config_builder_destroy(builder);

and also in your second example, you need to generate a 2nd config for the 2nd pair of keys

    ConfigBuilder* builder;
    Config* config;

    assert(config_builder_default(&builder) == 0);
    assert(config_builder_build(builder, &config) == 0);


    ClientKey* client_key = NULL;
    ServerKey* server_key = NULL;
    assert(generate_keys(config, &client_key, &server_key) == 0);

    assert(config_builder_default(&builder) == 0);
    assert(config_builder_build(builder, &config) == 0);
    ClientKey* second_client_key = NULL;
    ServerKey* second_server_key = NULL;
    assert(generate_keys(config, &second_client_key, &second_server_key) == 0);

[rdlopes]

All clear, thanks @tmontaigu.