ArrowArrayInitFromSchema(): Error parsing schema->format: Expected decimal bitwidth of 128 or 256 but found 64

[mskyttner]

Connecting using adbi and adbcflightsql against a Arrow Flight SQL Server and reading data from a decimal column ("lineitem" table) complains with the error message "ArrowArrayInitFromSchema(): Error parsing schema->format: Expected decimal bitwidth of 128 or 256 but found 64".

Replicate by running two Flight SQL Servers, using container images from voltrondata/sqlflite (older) and gizmodata/gizmosql (newer):

# first run this arrow flight sql server (arrow v 17 and duckdb v 1.1.1)
docker run --name sqlflite --detach --rm --tty --init --publish 31336:31337   --env TLS_ENABLED="0" --env SQLFLITE_PASSWORD="sqlflite_password"   --env PRINT_QUERIES="1" --env INIT_SQL_COMMANDS="SET threads = 1; SET memory_limit = '1GB';"   --pull missing voltrondata/sqlflite:latest

# then run the new "version" of this "gizmosql" (arrow v 20 and duckdb v 1.3)
docker run --name gizmosql --detach --rm --tty --init --publish 31337:31337   --env TLS_ENABLED="0" --env GIZMOSQL_PASSWORD="sqlflite_password"   --env PRINT_QUERIES="1" --env INIT_SQL_COMMANDS="SET threads = 1; SET memory_limit = '1GB';"   --pull missing gizmodata/gizmosql:latest

Then from R make a connection to the newer server:

#install.packages("adbi")
#install.packages("adbcflightsql", repos = "https://community.r-multiverse.org")

con <- adbi::dbConnect(adbi::adbi("adbcflightsql"),
  uri = "grpc://127.0.0.1:31337/",
  username = "sqlflite_username",
  password = "sqlflite_password"
)

con |> DBI::dbGetQuery("select #5 from lineitem;")

# we get an error ! ArrowArrayInitFromSchema(): Error parsing schema->format: Expected decimal bitwidth of 128 or 256 but found 64


DBI::dbDisconnect(con)

Note that this works against the other (older) server

# it works against the older version (voltrondata/sqlflite)

con <- adbi::dbConnect(adbi::adbi("adbcflightsql"),
  uri = "grpc://127.0.0.1:31336/",
  username = "sqlflite_username",
  password = "sqlflite_password"
)

con |> DBI::dbGetQuery("select #5 from lineitem;")

DBI::dbDisconnect(con)

The error traceback:

Error in `nanoarrow::nanoarrow_array_init()`:
! ArrowArrayInitFromSchema(): Error parsing schema->format: Expected decimal bitwidth of 128 or 256 but found 64
Hide Traceback
    ▆
 1. ├─DBI::dbGetQuery(con, "select #5 from lineitem;")
 2. └─DBI::dbGetQuery(con, "select #5 from lineitem;")
 3.   └─DBI (local) .local(conn, statement, ...)
 4.     ├─DBI::dbFetch(rs, n = n, ...)
 5.     └─adbi::dbFetch(rs, n = n, ...)
 6.       └─adbi:::get_data_batch(res, "rest")
 7.         └─adbi:::arrow_ptype(x)
 8.           └─nanoarrow::nanoarrow_array_init(...)

[mskyttner]

Using the latest devel versions of the adb(i|c) packages I don't see this issue anymore, so perhaps it could be closed?

I tried this with against a gizmosql Flight SQL server (https://github.com/gizmodata/gizmosql/pkgs/container/gizmosql/574482320?tag=v1.12.10-slim) and it now works well:

pak::pak("github::r-dbi/adbi")
pak::pak("apache/arrow-adbc/r/adbcflightsql")
pak::pak("apache/arrow-adbc/r/adbcdrivermanager")

library(adbi)
library(adbcflightsql)
library(adbcdrivermanager)

con <- 
  DBI::dbConnect(
    drv = adbi::adbi(driver = adbcflightsql::adbcflightsql()),
    uri = "grpc://some.ip:31337",
    username = "some_user",
    password = "some_pass",
    bigint = "integer64"
  )

# works and returns a column with (duckdb) bigints as R bit64::integer64 values
con |> dplyr::tbl("some_table") |> head()