Description
What happens?
I'm using DuckDB Wasm 1.29.0 through JavaScript in the browser (Chrome).
I create some tables in the "memory" database and populate them with data from various sources. Now I'd like to download the in-memory db to a single file (i.e. preferrably not as single csv, json or parquet tables).
There seems to be the possibility to download the db as a duck or sqlite file, however I struggle to succeed with this.
Here's what I tried for sqlite:
async exportSQLiteFile() {
const fileName = "my.db";
const aliasName = "mydb";
await this.#db.registerEmptyFileBuffer(fileName);
await this.#conn.query(`ATTACH '${fileName}' AS ${aliasName} (READ_WRITE, TYPE SQLITE);`);
// Copy the in-memory database to the new database file
await this.#conn.query(`COPY FROM DATABASE memory TO ${aliasName};`);
await this.#conn.query(`DETACH ${aliasName};`);
const buffer = await this.#db.copyFileToBuffer(fileName);
fileIo.downloadFile(buffer, fileName);
await this.#db.dropFile(fileName);
}
The COPY process seems to work. When I change to mydb, the tables and data from the memory db are there. However, the downloaded file "my.db" is 1 byte in size...
I then tried to download a duckdb file:
async exportDuckFile() {
const fileName = "my.db";
const aliasName = "mydb";
await this.#db.registerEmptyFileBuffer(fileName);
await this.#conn.query(`ATTACH '${fileName}' AS ${aliasName} (READ_WRITE, TYPE DUCKDB);`);
// Copy the in-memory database to the new database file
await this.#conn.query(`COPY FROM DATABASE memory TO ${aliasName};`);
await this.#conn.query(`DETACH ${aliasName};`);
const buffer = await this.#db.copyFileToBuffer(fileName);
fileIo.downloadFile(buffer, fileName);
await this.#db.dropFile(fileName);
}
This results in an "Uncaught (in promise) Error" when using copyFileToBuffer.
The only thing that has worked for me so far, is exporting the tables to single parquet-files like this:
async exportParquetFiles(dbName) {
// Note: This is a custom function for getting the names of existing tables.
// Not to be confused with conn.getTableNames from the API, which does something else.
const tableNames = await this.getTableNames(dbName);
for (const tableName of tableNames) {
const fileName = `${tableName}.parquet`;
await this.#db.registerEmptyFileBuffer(fileName);
await this.#conn.query(`COPY (SELECT * FROM ${tableName}) TO '${fileName}' (FORMAT PARQUET);`);
const buffer = await this.#db.copyFileToBuffer(fileName);
fileIo.downloadFile(buffer, fileName);
await this.#db.dropFile(fileName);
}
}
This still trows a warning for every exported file: "Buffering missing file: *.parquet"
I'm not sure if this is a problem or what needs to be done in order to not get this warning.
What am I missing in the first 2 examples, how is it possible to export the in.memory db to a duckdb or sqlite file?
To Reproduce
See code examples and description above.
Browser/Environment:
Chrome 134.0.6998.178
Device:
Win11 Laptop, i7-8850-H CPU
DuckDB-Wasm Version:
1.29.0
DuckDB-Wasm Deployment:
locally via VSCode Live Server
Full Name:
Andy
Affiliation:
none