Description
What happens?
I’m using duckdb-wasm
with the spatial
extension loaded, and repeatedly running a query of the form:
INSERT INTO segments (id, geom, a, b, recorded_at_z)
SELECT id, geom, a, b, z FROM ST_Read('previously-registered-file.geojsonl')
ON CONFLICT (id) DO UPDATE SET
geom = EXCLUDED.geom,
recorded_at_z = EXCLUDED.recorded_at_z
WHERE segments.recorded_at_z < EXCLUDED.recorded_at_z;
I’m frequently seeing errors from within duckdb-wasm
like the following:
- TransactionContext Error: Failed to commit: Trying to read past end of buffer
- TransactionContext Error: Conflict on tuple deletion!
- RuntimeError: memory access out of bounds
- TransactionContext Error: Cannot perform IO in in-memory database — CreateBlock!
the purpose is to ingest a large amount of geospatial data into a queryable form as it loads incrementally from map vector tiles. Each geojson file often contains hundreds (up to a few thousand) of features, and each feature becomes its own record in duckdb.
I’m not sure whether these errors arise as a result of one issue or multiple, but I am seeing all of these errors from the same query (when run with different data)
To Reproduce
I “recorded” the registered files and the executed queries executed for a single session that displays several of these errors in order to create a reliable minimal reproduction.
Here,
files.json
contains content that was “registered” as files to readqueries.json
contains queries that were run that produced the error(s)- the included
index.js
loads duckdb, registers each of the files, and runs the recorded queries in sequence. This is a simple entrypoint that “replays” the sequence of queries that leads to the errors.
Running index.js
reliably produces the errors I saw earlier; the set of conditions leading to the errors appears to be deterministic.
Browser/Environment:
Chrome 136
Device:
macOS 15.5 (Darwin 24.5.0 arm64 arm)
DuckDB-Wasm Version:
1.29.1-dev132.0
DuckDB-Wasm Deployment:
bundled
Full Name:
Luke Taylor
Affiliation:
N/A; using for my own purposes for personal projects