Releasing v1.0.9

[ndonfris]

releasing v1.0.9 feat.separating-symbol-parsing

Hello there!

I've been working on cleaning up the code base here, and improving the
maintainability of this project for a while now. Most of my work has been
documented in the history of the branch, feat.separating-symbol-parsing.

Apologies for the large amount of changes in this branch. This branch should be
a pretty stable experience in its current state, and will continue improving
until it is merged. Feel free to fork the repo and set up a local development
branch that tracks this branch, if you are interested in contributing directly
to it.

Contributions to master are also welcome. However, if you are attempting to
work on a feature mentioned below, you will likely need some of the newer
features to successfully implement them.

Below are some of the things I've been working toward finishing in this branch.
Any help, ideas, or feedback would be greatly appreciated.

General Improvements

• Startup speed time has increased

• Parsing out nodes that are a definition are significantly easier to work with.

  Over the history of this project, finding where functions or variables are
  defined was a bit janky and required lots of testing to work with.

  Since tree-sitter-fish doesn't include token definitions for the nodes we are
  interested in, we have to rely on the context around a SyntaxNode to
  determine it's type.

  The newer src/parsing/*.ts files allow us to keep the previous logic we used
  to determine if a SyntaxNode was a definition by primarily looking up through its parent nodes,
  while also only moving down the tree to only focus nodes that are the
  node.type or node.firstNamedChild.text for commands that we specify.

  We also try to define what possible flags can be seen on in each command that we process
  (see SetOptions). This allows us to define shape of a command.

  Lastly, we still export the isVariableDefinitionName, isAliasDefinitionName,
  and isFunctionDefinitionName from the src/parsing/*.ts files, and the
  again from the src/utils/node-types.ts files, so that we can import all
  node-types from the same location.

• Documentation included on FishSymbol items are improved, and Hover results
  include more information. This is primarily done by the src/snippets/*.json files.

• Autoloaded environment variables are now sourced during the server's startup
  process. This allows us to use the values from these variables in multiple
  places throughout the server. A particularly nice example is in CodeAction's
  that previously used an absolute file path in it's title, now uses the variable
  corresponding to the workspace it is in to shorten the path.

• Tons of files that were no longer used were removed.

  This is in preparation for the eventual 1.1.0 release, where the project
  structure will not need to be changed within the foreseeable future. (I hope)

  1.1.0 will include a rewrite of the src/utils/completions/*.ts files

TODO

• Add better workspace support for non auto-loaded/small fish workspaces.

  THE server FILE LIKELY NEEDS workspace handlers.
  If you are familiar with this feature, please don't hesitate to submit a PR
  to this branch. My attempts thus far have been unsuccessful :/

  You should also be able to have a relatively stable experience
  using this branch, if you are interested in helping test the upcoming
  features.

  • Supporting /tmp/*.fish buffers from things like edit_command_buffer/alt-e

  • Supporting fisher workspaces (consider a workspace like ~/repo/fisher-plugin, as its own workspace without $fish_lsp_all_indexed_paths)

    NOTE: fish_lsp_single_workspace_support was an attempt at specifying that
    the user has opted into this feature

  • Supporting sourced files outside of workspace, and adding them to the
    workspace

• implement a procedure to compare argparse definition flags to completion complete
  flags that would match. Allowing our generate completion for argparse command code-actions to
  skip already defined flags in a loaded file.

• Add some more user configuration, options like:

  • fish_lsp_indent_command to specify the fish_indent command & it's flags
    for formatting a LspDocument.

    • This feature would allow users who write their fish scripts with ;
      symbols to format their code using the command: fish_indent --only-indent.
      fish_indent behavior was changed from version [email protected] to
      [email protected]

  • potentially remove fish_lsp_single_workspace_support since it
    doesn't work as intended.

  • support initializationOptions

    • deprecated env variables are supported
    • add documentation for using the initializationOptions

Completed

• Add $__fish_** auto-loaded environment variables to the server's startup
  configuration. An example auto-loaded variable would be $__fish_config_dir

  • env
    exports the handler to lookup between process.env and the stored env.envStore
    object.
  • the stored env.envStore object is populated by the file src/utils/process-env.ts
    which importantly does not store the autoloaded variables on process.env. If we
    were to store the autoloaded variables here, any child process (like the src/utils/exec.ts file)
  • src/utils/process-env.ts uses the names seen in autoloadedFishVariableNames
    and ./fish_files/get-fish-autoloaded-paths.fish.
    Auto-loaded variables retrieved in these files, currently need their
    keys to be in the same order in both files.
  • keys are stored as path arrays (fish path variables follow the format: path1:path2:path3:path\ 4:path_5)
    in the env.envStore object, and are retrieved by the env handler by using the env.get() & env.getAsArray() methods.

• Improve parsing of specific DocumentSymbols (now FishSymbol) by using
  more verbose rules for excluding certain tokens from web-tree-sitter

• Add go to definition and hover support for source files.

• Improve the Hover outputs to add included info from ./src/snippets/*.json files,
  when available.

• Improve the src/utils/node-types.ts readability by separating each kind
  of node we have to handle in a web-tree-sitter tree, for FishSymbol definitions.
  Here we actually renamed FishDocumentSymbol to FishSymbol across the
  entire project. The major differences being how they are parsed

  • functions:

    • function names
    • alias nodes

  • variables:

    • for statement variables
    • set command variables
    • function _ --argument-names a b --inherit-variable c
    • read --array a b c
    • argparse h/help -- $argv

  • commands:

    • source filename filenames
    • complete -c command command names and opts

  • flags/options:

    • --long, -s (short), -type d (unix)
    • --field=2, -f2 would be able to get both flag seen and value,
      -f/--field would be the flag seen, and value would be 2 for both
    • set -gx would be able to get both --global and --export flags
      Of course, this requires that you can define the short flags before
      looking them up against a SyntaxNode in the web-tree-sitter tree.

• cleanup unused files, and refactor some code to improve readability

• added lots more tests, and fixed test-cases that had their expected
  results changed by the new features.

• additions & fixes to code-actions

  • source command file doesn't exist is more precise
  • argparse completions in conf.d/*.fish files are fixed.
  • fish --no-execute is now an experimental feature, and is disabled by
    default.
  • invalid # @fish-lsp-disable NUMBERS now warn the user.

• changed the install scripts to use node instead of fish for the package.json, script sh:build-time

  • this should allow mason.nvim to install the package across their containerized environments, although you would still need fish to run the server
  • other package managers might also appreciate this fix.

Related discussions/issues

• #42
• #76
• #52

[ndonfris]

So, on release/v1.0.9 source commands should apply to workspaces. Still definitely needs more testing though...

There are a couple of minor bugs like the server sending this.startBackgroundAnalysis() more than once (particularly when it's already waiting for a response from the background analysis), or occasionally the client will have a small delay while detecting if the open workspace changed / got closed. I read through the vscode documentation, and it looks like maybe the onConfigurationChange handler might be useful for keeping track of the currently focused document in the client?

A code review on the workspace handler would be greatly appreciated.

Will try to find some time early next week to work on finalizing the changes mentioned here into a PR.

Also, these changes will allow you to jump around to different $__fish_* autoloaded env paths (since the src/server.ts register() and onInitialized() method now handle onDidWorkspaceChange

---

Other notes about recent changes include:

• removal of fish-lsp logger subcommand. Use fish-lsp info --log-file to get the path or
  tail -f (fish-lsp info --log-file) if you want to watch the logs update in real time.

• added LocationLink[] functionality for argparse h/help -- $argv symbol definitions, so that a rename can happen across the completions file and the function file. In the previous example, a client request on either flag defined in argparse,
  _flag_h or _flag_help, will now check both the local scope and the completion file for a matching definition:
  complete -c function_name -s h -l help

• Added the CompletionSymbol type to compare FishSymbol items to CompletionSymbol. I'm interested in hearing what others think the behavior should be for a server's onDocumentSymbol request, particularly for source & complete commands. My initial intuition was that completion flags shouldn't be included in the symbol tree, but should be accessible to references, if applicable. I'm undecided on how sourcing should affect this handler.

[ndonfris]

PR is here #78.

Need's user testing in other language client's before it is likely to get merged.

Any help documenting the server's behavior, would really accelerate the release process. Also, feel free to submit any potential changes there.