feature: highlight patterns like vim's hlsearch

[midichef]

This is a draft of a feature to highlight substrings within a column, or in every column. It looks like this, highlighting \d+ in the SKU column, and highlighting cat everywhere.

I'm floating this to get feedback. By default, highlighting is used after every search with / and ? and g/ and g?. options.highlight can be toggled to turn highlighting off. highlight-sheet will do the same highlighting as g/ and highlight-col will do the same as /, while keeping the cursor in place.

In the screenshot you can see that partial offscreen matches are highlighted for strings that don't fully fit in the cell, like where Cat is shown as C…. (But partial offscreen data will not be highlighted for collections like tuples, lists, and dictionaries. That treatment of offscreen contents is the same as the current behavior for /.)

Remaining work

1. Should | set the highlight pattern like / does? I'm not sure.
2. When a column has a highlight pattern, the sheet pattern is not used for that column. In the last line, in the SKU column, you can see that the column \d+ pattern is lit, but the sheet pattern cat is not.
3. Undo of search-* does not undo highlighting. (undo of highlight-* works properly, but search-* is in nonLogged so it is more complicated)
4. Ellipses at the edge of a column are sometimes absent, when they are adjacent to an offscreen highlighted string.
5. Matches that are partially offscreen might have off-by-1 errors in highlighting for corner cases of full-width characters and very narrow columns.

[midichef]

@daviewales @frosencrantz You two have contributed to past discussions and work on visidata's colorizers. Like those, pattern highlighting is a visual cue. Perhaps you have some feedback?

[daviewales]

Is there a way to clear the highlighting? For example, vim-sensible maps this to Ctrl+L:
https://github.com/tpope/vim-sensible/blob/0ce2d843d6f588bb0c8c7eec6449171615dc56d9/plugin/sensible.vim#L58

[midichef]

There's no command/binding to clear it now. The only way to clear it is to set a new nonexistent pattern. Having one is a good suggestion. Ctrl+L is an interesting direction. Maybe zCtrl+L?

On thinking about it more, allowing multiple simultaneous highlight patterns (one for each column, plus one for the entire sheet) is too complicated. Instead there should only be one at a time. That's what users expect from vim and less.

[frosencrantz]

Thank you for doing this. I think this is great.

Having only one highlighting format at a time is reasonable. Having multiple different highlights can get too complicated. Would the highlighting be per column or per sheet depending on the last type of search performed? Or would it be useful for it to control that by option? At some point it might be interesting to have a way do some sort of syntax highlight like vim and other tools can do, but that is a separate item.

I think the special cases where the matches are at the edge of the cell or screen should eventually be fixed.

[daviewales]

I agree that it makes sense just to have one highlight type at a time. In Vim at least, the highlight is based on the most recent search. I'd likely expect similar in Visidata. For example, I know that when I jump to the next match, it will be the next highlighted item.

[midichef]

@frosencrantz I've pushed a change. The last search sets the highlight type. So / does per-column, g/ does throughout-sheet.

Ctrl+L now clears the highlighting as part of redraw. It's an intuitive binding, thank you for suggesting it @daviewales.

For | and \, selecting rows by regex, my feeling is they should not change highlights. They already give visual feedback by changing selections. And someone who does want highlighting, they can just follow | up with / Up Enter.

So the remaining issues now are just 3, 4, and 5 from my initial list: fixing undo for / and variants, and fixing display problems at edges of columns.

[saulpw]

Always use sheet.options (self.options in this case) when possible. Then the option can be set per-sheet.

[saulpw]

This should be on Sheet, not BaseSheet.

[saulpw]

This is a neat feature. My main concern is that it has to be a core feature. If it could be segmented into a self-contained file in features/ that would make it a lot more compelling. I don't see how that can be though, without adding more complicated hooks. So let's keep it simple and minimal and if people really like the feature we can take it.