Use of sqlite_orm in library causes sqlite3_mutex_enter to deference NULL and cause segmentation fault

[tpanajott]

First of, hi and thank you for this project. It makes working against SQLite databases a lot easier.

It seems like I've managed to run into a multi-threaded issue. I am creating the storage as per below:

namespace database_manager {

// A setting as represented in the database
struct SettingHolder {
  int id = 0;
  std::string name;
  std::string value;
};

// A room as presented in the database
struct Room {
  int id = 0;
  std::string friendly_name;
  int display_order = 0;
};

static inline auto get_db() {
  auto db = sqlite_orm::make_storage("/data/nspanelmanager_db.sqlite3",
                                     sqlite_orm::make_table("web_settings",
                                                            sqlite_orm::make_column("id", &SettingHolder::id, sqlite_orm::primary_key()),
                                                            sqlite_orm::make_column("name", &SettingHolder::name),
                                                            sqlite_orm::make_column("value", &SettingHolder::value)),
                                     sqlite_orm::make_table("web_room",
                                                            sqlite_orm::make_column("id", &Room::id, sqlite_orm::primary_key()),
                                                            sqlite_orm::make_column("friendly_name", &Room::friendly_name),
                                                            sqlite_orm::make_column("displayOrder", &Room::display_order)));
  return db;
}

}; // namespace database_manager

When in the main thread I can simply do something like this:

  auto db_room = database_manager::get_db().get<database_manager::Room>(2);
  SPDLOG_DEBUG("Got room {}::{}.", db_room.id, db_room.friendly_name);

  auto db_setting = database_manager::get_db().get<database_manager::SettingHolder>(2);
  SPDLOG_DEBUG("Got value {}", db_setting.value);

and that works absolutely fine. However, If I then do the exact same thing in another thread it will simple crash as per the backtrace:

#0  0x0000000000000000 in ?? ()
#1  0x00007ffff41f92dd in sqlite3_mutex_enter (p=0x555555f67988) at /MQTTManager/conan_cache/b/sqlit61cfc359e99c8/b/src/sqlite3.c:28751
#2  0x00007ffff42365a6 in vdbeUnbind (p=0x555555f87a48, i=0) at /MQTTManager/conan_cache/b/sqlit61cfc359e99c8/b/src/sqlite3.c:91438
#3  0x00007ffff4236996 in sqlite3_bind_int64 (pStmt=0x555555f87a48, i=1, iValue=2) at /MQTTManager/conan_cache/b/sqlit61cfc359e99c8/b/src/sqlite3.c:91549
#4  0x00007ffff4236963 in sqlite3_bind_int (p=0x555555f87a48, i=1, iValue=2) at /MQTTManager/conan_cache/b/sqlit61cfc359e99c8/b/src/sqlite3.c:91544
#5  0x00007ffff418b7ea in sqlite_orm::statement_binder<unsigned short, void>::bind (this=0x7fffffffb3eb, stmt=0x555555f87a48, index=1, 
    value=@0x7fffffffb620: 2) at /MQTTManager/conan_cache/sqlite1767d63f3621/p/include/sqlite_orm/sqlite_orm.h:9961
#6  0x00007ffff418985a in sqlite_orm::statement_binder<unsigned short, void>::bind (this=0x7fffffffb3eb, stmt=0x555555f87a48, index=1, 
    value=@0x7fffffffb620: 2) at /MQTTManager/conan_cache/sqlite1767d63f3621/p/include/sqlite_orm/sqlite_orm.h:9950
#7  0x00007ffff4186896 in sqlite_orm::internal::conditional_binder::operator()<unsigned short, true> (this=0x7fffffffb590, t=@0x7fffffffb620: 2)
    at /MQTTManager/conan_cache/sqlite1767d63f3621/p/include/sqlite_orm/sqlite_orm.h:10162
#8  0x00007ffff4182e0f in sqlite_orm::internal::ast_iterator<unsigned short, void>::operator()<sqlite_orm::internal::conditional_binder> (
    this=0x7fffffffb44f, t=@0x7fffffffb620: 2, lambda=...) at /MQTTManager/conan_cache/sqlite1767d63f3621/p/include/sqlite_orm/sqlite_orm.h:15271
#9  0x00007ffff417ea3f in sqlite_orm::internal::iterate_ast<unsigned short, sqlite_orm::internal::conditional_binder&> (t=@0x7fffffffb620: 2, lambda=...)
    at /MQTTManager/conan_cache/sqlite1767d63f3621/p/include/sqlite_orm/sqlite_orm.h:15281
#10 0x00007ffff4179be0 in sqlite_orm::internal::ast_iterator<std::tuple<unsigned short>, void>::operator()<sqlite_orm::internal::conditional_binder>(std::tuple<unsigned short> const&, sqlite_orm::internal::conditional_binder&) const::{lambda(auto:1&)#1}::operator()<unsigned short const>(sqlite_orm::internal::conditional_binder&) const (__closure=0x7fffffffb4e8, v=@0x7fffffffb620: 2)

It seems that calling from a different thread causes sqlite3_mutex_enter to want to dereference a mutex that isn't set? I've looked though the issues on here that I found that talked about multi-threading and I've also tried reading up on how to do multi-threading with only SQLite3 but for the life of me I can't figure it out. I've tried both version 1.9 and version 1.8.2 from conan.

Thankful for any help!

[fnc12]

Hi. Thanks for such a warm feedback!

This issue looks as usual multi-threading issue. Storage doesn't handle multi-threading for, it just passed the calls to SQLite engine. Of course there is a small responsibility area of storage itself which probably has to be covered with multi-threading primitives to handle cases like this but it will reduce performance in single-threaded apps. Quick advice: try to call open_forever to keep sqlite3* handle the same if it is suitable for your case. Otherwise you better handle multi-threading by yourself

[trueqbit]

What you are reporting here seems really odd.

Can you check the result of sqlite_orm::threadsafe(), which reports on the compile-time selection of the concurrency mode?

SQLite3 itself supports multiple concurrency modes, however it is threadsafe by default.

There is a whole range of information available to give you a complete picture of multithreading and multiple access to a single database in SQLite:
FAQ about thread safety
FAQ about accessing a database from multiple processes
Using SQLite In Multi-Threaded Applications
Test To See If The Library Is Threadsafe
Multiple connections are always threadsafe unless completely turned off
Compile-time flags

Especially because you are using a fresh sqlite_orm 'storage' instance on every call there should be no problem, as I understand it.

Like @fnc12 I also suggest that you create the sqlite_orm 'storage' instance only once (i.e. a singleton), and use open_forever(). While sqlite_orm's connection reference count is atomic, establishing and closing the connection is not threadsafe.

[tpanajott]

I'll do so an report back. It's just very strange to me that if I run the code in main before running it in a sub class it works.

I'll see if sqlite forums can help and report back.

[tpanajott]

I've opened a thread on the SQlite forum but not heard anything back as of yet. In the mean time I've been doing some more digging, including stepping through each line of the code called and it seems that the .mutex object is simply reset to the default 0x0 which seems VERY strange to me. I also figured out the sqlite3GlobalConfig is #defined to sqlite3Config within the sqlite3.c file so that I can observe the variable. But, given the screenshot below I find this very strange and I'm actually at a loss of what to do. Any ideas? Shortly after this sqlite3_mutex_enter is called and simply crashes because it tries to call sqlite3GlobalConfig.mutex.xMutexEnter which is set to 0x0 so that obviously fails.

Backtrace:

[codinghaven]

I too was experiencing the same issue recently, I'm on Apple Silicon M3 Max chip, the issue was happening on sqlite v3.43.2, however when I pointed my lib to v3.48 (latest version) the error disappeared, threadsafe() was returning 2 and even tried keeping a single connection, using :memory: option for the db name, nothing worked until I installed the latest version.

[tpanajott]

Thanks for your suggestion @codinghaven, unfortunetly it seems that it didn't solve my problem. It still resets the sqlite3Config.Mutex config when calling sqlite3_prepare_v2 which just seems really strange to me as it doesn't touch those. As soon as I step from sqlite_orm::internal::prepare_stmt and step into sqlite3_prepare_v2 the mutexes are simply gone.

Even if I run watch sqlite3Config.mutex or watch sqlite3Config.mutex.xMutexEnter in gdb to watch for when the variable changes it doesn't catch it. And at this point in the program there is only the main thread running so there shouldn't be any other thread that I'm not seeing that would change it.

The thing is also that earlier in the program it run sqlite3_prepare_v2 just fine when running a PRAGMA foreign_keys = 1 for example though that time doesn't go through sqlite_orm::internal::prepare_stmt. The next time sqlite_prepare_v2 is run is when trying to run SELECT "id", "friendly_name", "display_order" FROM "web_room" WHERE "id" = ? via sqlite_orm::internal::prepare_stmt and that's when it goes wrong.

The strange thing is that when it runs the first command PRAGMA foreign_keys = 1 the address for sqlite3Config is 0x555555f51b40 and when it later runs the sqlite3_prepare_v2 just fine when running a PRAGMA foreign_keys = 1 command the address is 0x7ffff5daef20 so somewhere along the lines the sqlite3Config has been replace and it seems that when that is done, the default mutexes aren't set.

[tpanajott]

I've set up gdb to break in sqlite3_config and sqlite3_prepare_v2 and this is what I can see.

1. sqlite3_config is called and setup for the first time and mutexes are set. Address of sqlite3Config: 0x555555f51b40.
2. sqlite3_prepare_v2 is called for command PRAGMA foreign_keys = 1, address for sqlite3Config is still the same.
3. sqlite3_prepare_v2 is run with command SELECT "id", "friendly_name", "displayOrder" FROM "web_room" WHERE "id" = ? but this time the address to sqlite3Config is 0x7ffff5daef80. So, somewhere along the lines the sqlite3Config object has been replaced and this time there is no mutexes set either.

From what I've been able to see in sqlite3.c file the only place where the sqlite3Config object is manipulated should be within the sqlite3_config function, but as seen above, that is never called and yet the object has changed/been replace between step 2 and 3.