Skip to content

Inserting data via Arrow fails in DuckDB 1.1.0 #120

@marvold-mw

Description

@marvold-mw

Using the following code (basically the sample code from https://duckdb.org/docs/api/nodejs/overview.html#inserting-data-via-arrow), DuckDB 1.1.0 failed to register an Arrow buffer via the Node bindings:

const arrow = require("apache-arrow");
const duckdb = require("duckdb");
const db = new duckdb.Database(":memory:");

const jsonData = [
  { userId: 1, id: 1, title: "delectus aut autem", completed: false },
  {
    userId: 1,
    id: 2,
    title: "quis ut nam facilis et officia qui",
    completed: false,
  },
];

// note; doesn't work on Windows yet
db.exec(`INSTALL arrow; LOAD arrow;`, (err) => {
  if (err) {
    console.warn(err);
    return;
  }

  const arrowTable = arrow.tableFromJSON(jsonData);
  db.register_buffer(
    "jsonDataTable",
    [arrow.tableToIPC(arrowTable)],
    true,
    (err, res) => {
      if (err) {
        console.warn(err);
        return;
      }

      // `SELECT * FROM jsonDataTable` would return the entries in `jsonData`
    }
  );
});
$ node ./test.js
[Error: Binder Error: No function matches the given name and argument types 'scan_arrow_ipc(STRUCT(ptr BIGINT, size INTEGER)[])'. You might need to add explicit type casts.
	Candidate functions:
	scan_arrow_ipc(STRUCT(ptr UBIGINT, size UBIGINT)[])

LINE 1: ...RY VIEW jsonDataTable AS SELECT * FROM scan_arrow_ipc([{'ptr': 5099265536, 'si...
                                                  ^] {
  errno: -1,
  code: 'DUCKDB_NODEJS_ERROR',
  errorType: 'Binder'
}

This code failed with NPM packages apache-arrow 17.0.0 and duckdb 1.1.0, but succeeded with apache-arrow 17.0.0 and duckdb 1.0.0.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions