Open
Description
What happens?
Currently, duckdb-wasm is using Apache Arrow v17.0.0 as a dependency, but the latest version available is v19.x.x.
In package.json:
"dependencies": {
"apache-arrow": "^17.0.0"
}
I'd like to propose updating the dependency to the latest version (v19.x.x) to access the latest features, better compatibility with other projects, and security/performance improvements.
To Reproduce
- Check package.json in the packages/duckdb-wasm directory
- Notice the apache-arrow dependency is pinned to ^17.0.0
- Compare with the latest version available on npm (v19.x.x)
Based on previous Arrow updates (like the upgrade to v16 in PR #1769 and v17 in PR #1804), there might be some compatibility issues to address:
- Issue
castTimestampToDate
does not work since arrow upgrade to v16 #1801 showed that thecastTimestampToDate
option stopped working correctly after upgrading to Arrow v16 - Issue Throw an error with mismatched Apache Arrow version #1761 highlighted problems with mismatched Apache Arrow versions
If approved, I'd be willing to create a PR with the following testing strategy:
- Update the dependency version in package.json
- Run the existing test suite to identify potential issues
- Pay special attention to timestamp handling and date conversions
- Test both browser and Node.js environments
Questions:
- Are there any known blockers for upgrading to Arrow v19?
- Should we consider first testing with a specific patch version rather than using the caret syntax?
- Are there any specific parts of the codebase that would need careful review during this upgrade?
Browser/Environment:
Chrome v136.0.7103.93, Node.js v23.11.0
Device:
PC, Windows 11, WSL2
DuckDB-Wasm Version:
1.11.0
DuckDB-Wasm Deployment:
Local development environment
Full Name:
Yui Matsumura
Affiliation:
Yuiseki Inc.
Metadata
Metadata
Assignees
Labels
No labels