csv deliminator support "\u0001"

[TATOAO]

Currently, it will raise an error "delimiter must be a 1-character string"

But it might be an often case that the csv is using \u0001 as the delimiter

I used this method to change deliminator:

Open file.
Press zO to open the sheet's options.
Navigate to option csv-delimiter.
Press l to go to the value column (usually, second to the right).
Press e to edit the value.
Write | and press Enter.
Press q to go back to your original sheet.
Press Ctrl-r to reload the sheet.

From https://stackoverflow.com/questions/69359412/how-do-i-open-a-file-in-visidata-with-a-specific-separator

[midichef]

You can do it, but it requires entering the delimiter character differently. To use \u0001 aka \x01 as the delimiter, assign to csv_delimiter using e for edit as you did in the options sheet, then ^V then ^A.

Or set the delimiter when you start vd at the command line (in bash or zsh): vd -f csv --csv-delimiter=$'\x01'

In general, instead of csv and --csv-delimiter, I'd recommend using the tsv filetype and the --delimiter argument. tsv allows delimiters of any length, so you can use a string abc as a separator: --delimiter=abc. It even allows \u0000 aka \x00 as the delimiter, by supplying an empty string: --delimiter=. The csv implementation in visidata can't do that. EDITED to add: the implementation can't use multi-character strings, but it can use \x00 as the delimiter.

[TATOAO]

You can do it, but it requires entering the delimiter character differently. To use \u0001 aka \x01 as the delimiter, assign to csv_delimiter using e for edit as you did in the options sheet, then ^V then ^A.

Or set the delimiter when you start vd at the command line (in bash or zsh): vd -f csv --csv-delimiter=$'\x01'

In general, instead of csv and --csv-delimiter, I'd recommend using the tsv filetype and the --delimiter argument. tsv allows delimiters of any length, so you can use a string abc as a separator: --delimiter=abc. It even allows \u0000 aka \x00 as the delimiter, by supplying an empty string: --delimiter=. The csv implementation in visidata can't do that.

Thanks, using ^V then ^A to replace \u0001 works perfectly!

vd -f csv --csv-delimiter=$'\x01' the_csv_file.csv
also works, but it breaks the usage of vd as a file explorer.

So my point is, actually, shouldn't we consider removing the error check for "delimiter must be a 1-character string"?

[saulpw]

This restriction is from the Python csv library itself: python/cpython@df98a47#diff-38fcce6bb475616052f5c9a0973eefd49489a4dff719f30e407534258e2a3ec3L260-L273

[midichef]

vd -f csv --csv-delimiter=$'\x01' the_csv_file.csv
also works, but it breaks the usage of vd as a file explorer.

In what way does it cause a problem for using vd as a file explorer?

Do you mean that it breaks parsing of standard .csv files that you want to open from within vd? If so, try adding -n (aka --nonglobal): vd -n --csv-delimiter=$'\x01' the_csv_file.csv
That will limit the CLI csv_delimiter change so it applies only to the_csv_file.csv, not any sheets you open later.

Also, two more ways of editing the csv_delimiter sheet option from the options sheet (O), instead of e (edit-cell):

1. ^O (sysedit-cell) and enter the characters using an editor like vi.
2. Make the new value using an expression: = then '\u0001' then Enter. Copy the new expression cell with zy and paste it into the csv_delimiter global_value cell using zp. Using this method for \u0000 I found that csv can indeed use the ASCII NUL as a delimiter.