Merge pull request #1040 from saulpw/v2.5rc

v2.5 Release

Author: anjakefala

.circleci/config.yml

@@ -4,6 +4,9 @@ workflows:
   test:
     jobs:
       - test-3.6
+      - test-3.7
+      - test-3.8
+      - test-3.9
 jobs:
   test-3.6: &test-template
     working_directory: ~/visidata
@@ -25,3 +28,18 @@ jobs:
       - run:
           command: |
             dev/test.sh
+
+  test-3.7:
+    <<: *test-template
+    docker:
+      - image: circleci/python:3.7
+
+  test-3.8:
+    <<: *test-template
+    docker:
+      - image: circleci/python:3.8
+
+  test-3.9:
+    <<: *test-template
+    docker:
+      - image: circleci/python:3.9

.envrc

@@ -0,0 +1,7 @@
+if has guix; then
+    use guix -m requirements.scm
+fi
+
+if has python3; then
+    layout python python3
+fi

.gitmodules

@@ -4,7 +4,4 @@
 [submodule "deps/pyxlsb"]
 	path = deps/pyxlsb
 	url = https://github.com/saulpw/pyxlsb.git
-[submodule "deps/sh"]
-	path = deps/sh
-	url = https://github.com/saulpw/sh.git
 

CHANGELOG.md

@@ -1,5 +1,107 @@
 # VisiData version history
 
+# v2.5 (2021-07-08)
+
+- [social] #visidata has moved off of freenode to libera.chat
+- [deps] required pandas version for the pandas loader has been bumped to at least 1.0.5
+- [caa] new PR submitters required to sign CAA
+
+## Features
+
+- [cli] when no arguments on commandline, open currentDirSheet (previously vdmenu); -f opens empty sheet of that filetype
+- [clipboard] bind `x` family to `cut-*` (thanks @geekscrapy #895)
+- [date] add specialized comparators for `datetime.date` (thanks @aborruso #975)
+    - visidata.date now compares to datetime.date (previously raised exception)
+        - identical dates compare equal even if intra-day times are different
+        - this does not work for incompletely specified visidata.date; e.g.
+            `visidata.date(2016, 10, 29, 4, 0, 0) != visidata.date(2016, 10, 29)`
+- [DirSheet] add y/gy to copy file(s) to given directory
+- [loaders vds] save non-jsonable cells as string (thanks @pacien #1011)
+- [loaders zstd] support loading zstd-compressed files (thanks @lxcode #971)
+- [movement] bind `Ctrl+Left/Right` to `go-left`/`right-page` (thanks @davidwales #1002)
+- [options] save to foo.visidatarc from OptionsSheet (thanks @njthomas #958)
+- [sqlite] RENAME and DROP tables from SqliteIndexSheet
+- [unfurl] add `options.unfurl_empty` to include row for empty list/dict (thanks @frosencrantz #898)
+- [quitguard] confirm quit/reload only if sheet modified (references #955, #844, #483; thanks @jvns, @frosencrantz)
+
+## Improvements
+
+- [addRow] advance cursor if row inserted before cursor
+- [archive] add .lzma as alias for .xz
+- [clipboard] gzp pastes None if nothing on clipboard
+- [clipboard] make syspaste async
+- [clipboard] bind `zP` to syspaste-cells and gzP to syspaste-cells-selected (thanks @jvns and frosencrantz #983, #990)
+- [cliptext] better support for combining and variant chars (thanks @lxcode #758 #1034)
+- [colors] reduce color swatch size to remove flashing (thanks @frosencrantz #946)
+- [encoding] specify encoding explicitly for all Path.open_text (thanks @pacien #1016)
+- [error] exceptionCaught(status=False) to add to status history, but not post to status (thanks @frosencrantz #982)
+- [freqtbl] copy fmtstr from source col to aggcol (thanks @geekscrapy #1003)
+- [help] ENTER/exec-command to execute command on undersheet (thanks @geekscrapy #1011)
+- [help] add `all_bindings` hidden column (thanks @frosencrantz #896)
+- [inputs] put reused input at end of lastInputs (thanks @geekscrapy #1033)
+- [loaders json] streamify save to .json
+- [loaders npy] add `npy_allow_pickle` option, default False
+- [loaders tsv] increase bufsize to improve loader performance by 10%
+- [path] all Path.open track Progress via read/filesize (thanks @jspatz #987)
+- [path] add Progress for opening compressed files
+- [path] implement line-seek operations (thanks @pacien #1010)
+- [regex expand] deprecate `options.expand_col_scanrows`; standardize on `options.default_sample_size` (thanks @jsvine)
+- [regex] "match regex" to "capture regex" (thanks @geekscrapy #1032)
+- [shell] `addcol-shell` pass command to $SHELL (thanks @juston2004 #1023)
+- [shortcut] allow shortcut for jump-sheet to be settable
+- [splitwin] push sheet in empty pane iff splitwin
+- [stdin] use cli --encoding option for piped data (thanks @pacien #1018)
+- [undo] remove undo for reload (replaced with quitguard+confirm)
+- [quit] add Shift+Q/quit-sheet-free to quit and free associated memory (thanks @cwarden)
+
+## Display
+- [canvas] add `options.disp_canvas_charset` to change displayed chars (thanks @albert-ying #963)
+- [canvas] use sheet specific options for draw
+- [disp] format list/dict as [n]/{n} only for anytype
+- [save] iterdispvals(format=True) convert None to empty string
+
+## Bugfixes
+
+- [batch] ensure quitguard is off during batch mode
+- [canvas[ fix error on dive into cursor including y-axis
+- [cli] have an actual error if there is a missing argument for final option
+- [cli] do nothing (no error) if no sources given
+- [clipboard] fix zy/gzp regression (thanks @sfranky #961)
+- [clipboard] syscopy-cell do not include column name
+- [cmdlog] fix bug where customising replayable options in Options Sheet led to issues opening metasheets (thanks @jsvine #952)
+- [cmdlog] fix bug where cmdlog records new sheet name, instead of old sheet name for `rename-sheet` (thanks @aborruso #979)
+- [color] track precedence so colorizers apply over `color_current_row`
+- [color] determine color availability with `init_pair`
+- [color] do not break on nonsense color
+- [column] getitemdeep/setitemdeep get/set dotted item key if exists (thanks @frosencrantz #991)
+- [column] fix bug where hard crash occurs when cursor on cell of SheetsSheet is on cursorDisplay (thanks @frosencrantz #1029)
+- [curses] add default `vd.tstp_signal` for non-cli users
+- [execCommand] warn gracefully if bound command longname does not exist
+- [expr] setValuesFromExpr do not stop processing on exception
+- [join] fix when keys have different names (thanks @aborruso #964)
+- [join] fix for rowdefs without bool (like pandas)
+- [join] fix multiple extend (thanks @cwarden for reporting)
+- [loaders fixed] fix editing in final column for fixed-width load (thanks @mwayne #974)
+- [loaders geojson] do not abort plot if rows have errors
+- [loaders html] add columns even if not in first row
+- [loaders pandas] fix column rename
+- [loaders rec json] fix adding new columns for json and rec loaders (thanks @ajkerrigan #959)
+- [loaders postgresql] add postgresql scheme (fixes #966) (thanks @zormit #967)
+- [loaders sqlite] fix saving deleted cells (thanks @mattenklicker #969)
+- [loaders vds] save SettableColumn as Column (thanks @pacien #1012)
+- [loaders zip] fix extract-selected-to
+- [open] fix regression where opening blank sheets of type tsv, csv, txt, etc was not working
+- [plugins] fix stdout/error from plugins installation message (was in bytes, changed to str)
+- [quit] remove sheets from **Sheets Sheet** upon quit
+- [save-col] fix inputPath error (thanks @savulchik #962)
+- [shell] fix `options.dir_hidden`; also apply to dirs when `dir_recurse`
+- [textsheet] fix reload after `^O` sysopen
+
+## vdplus
+
+- moved clickhouse, vsh, vgit, windows to vdplus
+
+
 # v2.4 (2021-04-11)
 
 - [splitwindow] stabilize sheet stack associations

CONTRIBUTING.md

@@ -41,19 +41,21 @@ VisiData is designed to be extensible, and most feature requests can be implemen
 If this would require changes to the VisiData core, and a reasonable design is approved, then the issue can stay open until the core changes have been made.
 Otherwise, in the spirit of Marie Kondo, the issue will be closed without prejudice.
 
-Feature requests with some amount of working Python code are more likely to get attention.
+Feature requeste with some amount of working Python code are more likely to get attention.
 Design proposals with concrete use cases are very welcome.
 
 ## Writing a well constructed bug report
 
 If you encounter any bugs or have any problems, please [create an issue on GitHub](https://github.com/saulpw/visidata/issues).
 
 A great bug report will include:
+
     - a stacktrace, if there is an unexpected error; the most recent full stack traces can be viewed with `Ctrl+E` (then saved with `Ctrl+S`)
     - a [.vd](http://visidata.org/docs/save-restore/) and sample dataset that reproduces the issue
     - a .png/.gif (esp. for user interface changes)
 
 Some examples of great bug reports:
+
     - [#350 by @chocolateboy](https://github.com/saulpw/visidata/issues/350)
     - [#340 by @Mikee-3000](https://github.com/saulpw/visidata/issues/340)
 
@@ -69,7 +71,7 @@ VisiData has two main branches:
   - [stable](https://github.com/saulpw/visidata/tree/stable) has the last known good version of VisiData (what is in pypi/brew/apt).
   - [develop](https://github.com/saulpw/visidata/tree/develop) has the most up-to-date version of VisiData (which will eventually be merged to stable).
 
-All pull requests should be submitted against `develop`.
+All pull requests should be submitted against `develop`. Submitters will need to e-sign a Copyright Assignment Agreement (CAA) before a pull request will be accepted.
 
 # Open Source License and Copyright
 

README.md

@@ -1,5 +1,5 @@
 
-# VisiData v2.4 [![twitter @VisiData][1.1]][1] [![CircleCI](https://circleci.com/gh/saulpw/visidata/tree/stable.svg?style=svg)](https://circleci.com/gh/saulpw/visidata/tree/stable) [![Gitpod ready-to-code](https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/saulpw/visidata)
+# VisiData v2.5 [![twitter @VisiData][1.1]][1] [![CircleCI](https://circleci.com/gh/saulpw/visidata/tree/stable.svg?style=svg)](https://circleci.com/gh/saulpw/visidata/tree/stable) [![Gitpod ready-to-code](https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/saulpw/visidata)
 
 A terminal interface for exploring and arranging tabular data.
 

deps/sh

@@ -9,7 +9,7 @@
   - [scroll cell] zh/zl Left hand side of a cell with content hidden doesn't show ellipsis #751
 
 2. Persistence
-  - input history  #736 #468
+  + input history  #736 #468
   - key indexes for better joining and lookups
 
 3. Interface discoverability for commands  #247 #742
@@ -19,9 +19,9 @@
   - possibly add popup modals
 
 4. more expressive expressions
-  - Memory sheet; can give things names and use in expressions.  #392
+  + Memory sheet; can give things names and use in expressions.  #392
   - access column values from execstr #655
-  - shortcut name for current column #659
+  + shortcut name for current column #659
 
 5. better asynchronicity
   - async thread pool
@@ -57,4 +57,4 @@ c) intra-cell coloring (for search results, markup)
 
   - #480: HTTP API pagination loader
   - #465: Ability to load from the contents of a cell.
-  - #505: [html] Provide way to access non-table elements
+  + #505: [html] Provide way to access non-table elements

dev/ROADMAP

@@ -115,7 +115,7 @@ git push --tags
 
 16. Upload new motd
 
-17. Update the website by pushing to master.
+17. Update the website by pushing to master. Update with new manpage. Update redirect to point to new manpage.
 
 18. Comb through issues and close the ones that have been solved, referencing the version number
 

dev/checklists/release.md

@@ -1,3 +1,9 @@
+visidata (2.4-1) unstable; urgency=low
+
+  * Update package to 2.4
+
+ -- Anja Boskovic <[email protected]>  Mon, 12 Apr 2021 20:49:47 -0700
+
 visidata (2.3-1) unstable; urgency=low
 
   * Update package to 2.3