JSON export is empty when there is blob type

[cfahlgren1]

What happens?

For example parquet for this dataset.

that has this schema:

0	id	VARCHAR	false	null	false
1	prompt	VARCHAR	false	null	false
2	chosen	STRUCT(bytes BLOB, path VARCHAR)	false	null	false
3	rejected	STRUCT(bytes BLOB, path VARCHAR)	false	null	false
4	chosen_model	VARCHAR	false	null	false
5	rejected_model	VARCHAR	false	null	false
6	evolution	VARCHAR	false	null	false
7	category	VARCHAR	false	null	false
8	sub_category	VARCHAR	false	null	false

it seems to download 5 empty rows when doing JSON export.

copy (select * from dataset limit 5) to 'output.json';

however the CSV and Parquet command works properly.

copy (select * from dataset limit 5) to 'output.csv';

To Reproduce

You can reproduce here with these queries in DuckDB Shell

https://shell.duckdb.org/#queries=v0,create-view-dataset-as-select-*-from-read_parquet('https%3A%2F%2Fhuggingface.co%2Fdatasets%2Fdata%20is%20better%20together%2Fopen%20image%20preferences%20v1%20binarized%2Fresolve%2Fmain%2Fdata%2Ftrain%2000001%20of%2000005.parquet')~,copy-(select-*-from-dataset-limit-5)-to-'output.csv'~,copy-(select-*-from-dataset-limit-5)-to-'output.json'~,.files-download-output.json,.files-download-output.csv

Browser/Environment:

Chrome Version 136

Device:

Mac

DuckDB-Wasm Version:

@duckdb/duckdb-wasm@1.29.1-dev141.0 (latest DuckDB Shell)

DuckDB-Wasm Deployment:

shell.duckdb.org

Full Name:

Caleb Fahlgren

Affiliation:

Hugging Face

[carlopi]

I can reproduce, but the same appears to be there in duckdb 1.2.2 CLI:

% duckdb
v1.2.2 7c039464e4
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
D create view dataset as select * from read_parquet('https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1-binarized/resolve/main/data/train-00001-of-00005.parquet');
D copy (select * from dataset limit 5) to 'output.json';
100% ▕████████████████████████████████████████████████████████████▏ 
D FROM 'output.json';
┌────────┐
│  json  │
│  json  │
├────────┤
│ 0 rows │
└────────┘
D 
D 

1.2.2 works correctly on CSV, so I would assume it's a duckdb issue.

Trying on a recent nightly build (native):

% duckdb-latest
v1.3.0-dev3269 e144689926
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
D create view dataset as select * from read_parquet('https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1-binarized/resolve/main/data/train-00001-of-00005.parquet');
D copy (select * from dataset limit 5) to 'output.json';
100% ▕████████████████████████████████████████████████████████████▏ 
D FROM 'output.json';
┌─────────────────┬──────────────────────┬──────────────────────┬──────────────────────┬──────────────────────────────────────┬────────────────────────────────────────┬────────────┬──────────────┬─────────────────┐
│       id        │        prompt        │        chosen        │       rejected       │             chosen_model             │             rejected_model             │ evolution  │   category   │  sub_category   │
│     varchar     │       varchar        │ struct(bytes varch…  │ struct(bytes varch…  │               varchar                │                varchar                 │  varchar   │   varchar    │     varchar     │
├─────────────────┼──────────────────────┼──────────────────────┼──────────────────────┼──────────────────────────────────────┼────────────────────────────────────────┼────────────┼──────────────┼─────────────────┤
│ 947-quality     │ autumn scene, a cu…  │ {'bytes': '\\xFF\\…  │ {'bytes': '\\xFF\\…  │ stabilityai/stable-diffusion-3.5-l…  │ black-forest-labs/FLUX.1-dev           │ quality    │ Illustration │ Manga           │
│ 3008-simplified │ A medieval brick c…  │ {'bytes': '\\xFF\\…  │ {'bytes': '\\xFF\\…  │ stabilityai/stable-diffusion-3.5-l…  │ black-forest-labs/FLUX.1-dev           │ simplified │ Digital art  │ highly detailed │
│ 1503-simplified │ Neonpunk Jedi duel…  │ {'bytes': '\\xFF\\…  │ {'bytes': '\\xFF\\…  │ black-forest-labs/FLUX.1-dev         │ stabilityai/stable-diffusion-3.5-large │ simplified │ Neonpunk     │ crisp           │
│ 2138-quality    │ organic cyborg, fu…  │ {'bytes': '\\xFF\\…  │ {'bytes': '\\xFF\\…  │ black-forest-labs/FLUX.1-dev         │ stabilityai/stable-diffusion-3.5-large │ quality    │ Illustration │ Technical       │
│ 295-simplified  │ Pixel art grand pi…  │ {'bytes': '\\xFF\\…  │ {'bytes': '\\xFF\\…  │ black-forest-labs/FLUX.1-dev         │ stabilityai/stable-diffusion-3.5-large │ simplified │ Pixel art    │ blocky          │
└─────────────────┴──────────────────────┴──────────────────────┴──────────────────────┴──────────────────────────────────────┴────────────────────────────────────────┴────────────┴──────────────┴─────────────────┘
D 

works, so I think this will be solved properly with the 1.3.0 release (end of May).

I will double check on duckdb-wasm + duckdb main, but I would assume it has been solved at some point in duckdb/duckdb.

[cfahlgren1]

@carlopi thanks for the quick response. Sounds good!

[carlopi]

It should be working as intended