Skip to content

Conversation

@nunogoncalves03
Copy link
Contributor

@nunogoncalves03 nunogoncalves03 commented Dec 14, 2025

Description

This PR updates JSON functions to correctly reject BLOB values, aligning their behavior with SQLite. It also fixes handling of text-based JSON passed as BLOB arguments to json(), which previously did not work as expected.

Previously, JSON functions (json_object, json_set, json_insert, json_replace, and their jsonb variants) attempted to convert BLOB values into JSON instead of rejecting them with an error. This differed from SQLite's behavior, which returns the error JSON cannot hold BLOB values. In addition, these functions were silently converting errors to NULL in the executor rather than propagating them to the user.

Additionally, json() previously returned incorrect results when given BLOBs containing JSON text. The root cause was that the function assumed all BLOBs were in binary JSONB format and attempted to parse them as such, leading to incorrect results when the BLOB actually contained textual JSON.

With these changes applied, JSON functions are matching SQLite's behavior:

turso> CREATE TABLE t(a);
turso> INSERT INTO t VALUES (x'74727565'); -- ASCII "true”;
turso> SELECT json_object('a', a) FROM t;
  × Runtime error: JSON cannot hold BLOB values

turso> SELECT json_array(a) FROM t;
  × Runtime error: JSON cannot hold BLOB values

turso> SELECT json_set('{"x":1}', '$.a', a) FROM t;
  × Runtime error: JSON cannot hold BLOB values

turso> SELECT json_insert('{"x":1}', '$.a', a) FROM t;
  × Runtime error: JSON cannot hold BLOB values

turso> SELECT json_replace('{"a":1}', '$.a', a) FROM t;
  × Runtime error: JSON cannot hold BLOB values

turso> SELECT json_quote(a) FROM t;
  × Runtime error: JSON cannot hold BLOB values
turso> SELECT json(x'2268656c6c6f22'); -- ASCII "\"hello\"”;
┌──────────────────────────┐
│ json (X'2268656c6c6f22') │
├──────────────────────────┤
│ "hello"                  │
└──────────────────────────┘

Motivation and context

This PR fixes the issues identified in #4195 to align with SQLite’s behavior.

Currently, Turso handles BLOB inputs inconsistently across JSON functions:

turso> CREATE TABLE t(a);
turso> INSERT INTO t VALUES (x'74727565'); -- ASCII "true”;
turso> SELECT json_object('a', a) FROM t;
┌────────────────────────┐
│ json_object ('a', t.a) │
├────────────────────────┤
│ {"a":true}             │
└────────────────────────┘
turso> SELECT json_array(a) FROM t;
  × Runtime error: JSON cannot hold BLOB values

turso> SELECT json_set('{"x":1}', '$.a', a) FROM t;
┌──────────────────────────────────┐
│ json_set ('{"x":1}', '$.a', t.a) │
├──────────────────────────────────┤
│ {"x":1,"a":true}                 │
└──────────────────────────────────┘
turso> SELECT json_insert('{"x":1}', '$.a', a) FROM t;
┌─────────────────────────────────────┐
│ json_insert ('{"x":1}', '$.a', t.a) │
├─────────────────────────────────────┤
│ {"x":1,"a":true}                    │
└─────────────────────────────────────┘
turso> SELECT json_replace('{"a":1}', '$.a', a) FROM t;
┌──────────────────────────────────────┐
│ json_replace ('{"a":1}', '$.a', t.a) │
├──────────────────────────────────────┤
│ {"a":true}                           │
└──────────────────────────────────────┘
turso> SELECT json_quote(a) FROM t;
  × Runtime error: JSON cannot hold BLOB values
turso> SELECT json(x'2268656c6c6f22'); -- ASCII "\"hello\"”;
┌──────────────────────────┐
│ json (X'2268656c6c6f22') │
├──────────────────────────┤
│ false                    │
└──────────────────────────┘

Expected behavior:

sqlite> CREATE TABLE t(a);
sqlite> INSERT INTO t VALUES (x'74727565'); -- ASCII "true”;
sqlite> SELECT json_object('a', a) FROM t;
Runtime error: JSON cannot hold BLOB values
sqlite> SELECT json_array(a) FROM t;
Runtime error: JSON cannot hold BLOB values
sqlite> SELECT json_set('{"x":1}', '$.a', a) FROM t;
Runtime error: JSON cannot hold BLOB values
sqlite> SELECT json_insert('{"x":1}', '$.a', a) FROM t;
Runtime error: JSON cannot hold BLOB values
sqlite> SELECT json_replace('{"a":1}', '$.a', a) FROM t;
Runtime error: JSON cannot hold BLOB values
sqlite> SELECT json_quote(a) FROM t;
Runtime error: JSON cannot hold BLOB values
sqlite> SELECT json(x'2268656c6c6f22'); -- ASCII "\"hello\"”;
"hello"

Fixes #4195.

AI Disclosure

This PR was developed with assistance from GitHub Copilot (Claude Sonnet 4.5). The AI helped identify the root cause and assisted in writing the tests.

Copy link

@turso-bot turso-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please review @pedrocarlo

@nunogoncalves03
Copy link
Contributor Author

nunogoncalves03 commented Dec 15, 2025

Messed up the rebase, fixing it now 🙃

Edit: Done!

@pedrocarlo
Copy link
Contributor

@jussisaurio or @PThorpe92 need someone to approve the workflow here to run the test

@nunogoncalves03
Copy link
Contributor Author

Fixed the failing clippy job, could we rerun the pipeline?


do_execsql_test_in_memory_error_content json_object_blob_error {
CREATE TABLE t(a);
INSERT INTO t VALUES (x'74727565'); -- ASCII "true";
Copy link
Contributor

@Pavan-Nambi Pavan-Nambi Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could u please add more tests? specifically like using jsonb? are we supposed to reject them too?

json_funcs(jsonb())

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are absolutely right, thanks for catching that! I have pushed a commit adding more tests for the jsonb functions and for json/jsonb interop. Let me know if these tests cover what you were expecting.

Also, while adding these tests, I discovered a bug: Turso wasn’t unescaping strings when converting jsonb to db/sql values, whereas SQLite does.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

json() (and possibly all json functions) loses first character when parsing blob

3 participants