Skip to content

Commit efaefe5

Browse files
committed
update zstd_vfs with VFS logging
1 parent d248a12 commit efaefe5

File tree

3 files changed

+7
-5
lines changed

3 files changed

+7
-5
lines changed

CMakeLists.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ endif()
1818
FetchContent_Declare(
1919
sqlite_zstd_vfs
2020
GIT_REPOSITORY https://github.com/mlin/sqlite_zstd_vfs.git
21-
GIT_TAG 25e962e
21+
GIT_TAG 2e9c7db
2222
)
2323
FetchContent_MakeAvailable(sqlite_zstd_vfs)
2424
FetchContent_MakeAvailable(sqlitecpp)

docs/guide_db.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -205,7 +205,7 @@ The **GenomicSQLite Open** routine and the `genomicsqlite` shell also accept htt
205205
Under the hood, the extension uses [libcurl](https://curl.se/libcurl/) to send web requests for necessary portions of the database file as queries proceed, with adaptive batching & prefetching to balance the number and size of these requests. This works well for point lookups and queries that scan largely-contiguous slices of tables and indexes (and a modest number thereof). It's less suitable for big multi-way joins and other aggressively random access patterns; in such cases, it'd be better to download the database file upfront to open locally.
206206

207207
* Reading large databases over the web, budget an additional ~600MiB of memory for HTTP prefetch buffers.
208-
* The HTTP driver writes log messages to standard error when requests fail or had to be retried, which can be disabled by setting configuration web_log = 0 or environment SQLITE_WEB_LOG=0; or increased up to 5 to log every request and other details.
208+
* The HTTP driver writes log messages to standard error when requests fail or had to be retried, which can be disabled by setting configuration vfs_log = 0 or environment SQLITE_VFS_LOG=0; or increased up to 5 for extensive debug logging.
209209
* To disable TLS certificate and hostname verification, set web_insecure = true in the GenomicSQLite configuration, or SQLITE_WEB_INSECURE=1 in the environment.
210210
* The above-described `genomicsqlite DB_FILENAME --compact` optimizes a database for web access by making the request pattern more contiguous.
211211

src/genomicsqlite.cc

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ std::string GenomicSQLiteDefaultConfigJSON() {
6868
"zstd_level": 6,
6969
"inner_page_KiB": 16,
7070
"outer_page_KiB": 32,
71-
"web_log": 2,
71+
"vfs_log": -1,
7272
"web_insecure": false,
7373
"web_dbi_url": "",
7474
"web_nodbi": false,
@@ -173,9 +173,11 @@ string GenomicSQLiteURI(const string &dbfile, const string &config_json = "") {
173173
bool web = dbfile.substr(0, 5) == "http:" || dbfile.substr(0, 6) == "https:";
174174
ostringstream uri;
175175
uri << "file:" << (web ? "/__web__" : SQLiteNested::urlencode(dbfile, true)) << "?vfs=zstd";
176+
if (cfg.GetInt("$.vfs_log") >= 0) {
177+
uri << "&vfs_log=" << cfg.GetInt("$.vfs_log");
178+
}
176179
if (web) {
177-
uri << "&mode=ro&immutable=1&web_url=" << SQLiteNested::urlencode(dbfile)
178-
<< "&web_log=" << cfg.GetInt("$.web_log");
180+
uri << "&mode=ro&immutable=1&web_url=" << SQLiteNested::urlencode(dbfile);
179181
if (cfg.GetBool("$.web_insecure")) {
180182
uri << "&web_insecure=1";
181183
}

0 commit comments

Comments
 (0)