Add option to reuse shell

[richard-hajek]

I think it would be beneficial to allow for execution of an entire task in a single shell.

Motivation

It is currently very difficult to use Python's venvs or conda, due to the shell being reset on each command. The recommended way is to use absolute paths inside the venv, such as venv/bin/python instead of source venv/bin/activate;python. This however increasing the friction for newcomers and not all problems of this kind have similar solutions.

My current Taskfile, looks like this:

[...]
tasks:
  build:
    cmds:
      - mamba env create -f environment.yaml || true
      - mamba env update -f environment.yaml --prune
      - bash -c "
        source conda_init;
        mamba activate csidrl;
        poetry install;
        cd lib/SATNet;
        python setup.py install;
        "

Which is ugly and complicated. This approach also complicates cross-platform implementations.

There are other workarounds, such as in #512. There would be other uses of this feature, for example resolving #204.

Proposal

For a task, add subshell boolean. This boolean would be by default true, ensuring compatibility. On false, this option would make go-task invoke all cmds in a single mvdan.cc/sh/v3/interp. When false, this option would also cause all task: cmds to run in the same interp.

The unclear question is a treatment of deps. The current behavior would see them running in different shells, which may produce some confusion. The consistent behavior with the rest of this proposal would however sacrifice concurrency for simplicity and make deps run linearly after each other in the same shell (when subshell: false only). This behavior would also resolve #204. Neither of these options would result in consistent behavior throughout the go-task though.

Advantages

• Enabling or vastly simplifying source-based workflows
• Consistency with (ba/z/)sh scripts
• Resolving the use-case in Add a "setup:" setting to allow running setup scripts before any command? #204

My humble opinion

I believe that each of the advantages by themselves would be a strong-enough for this feature to be accepted, but there are some issues.

Even though this feature may simplify some workflows, the added complexity may not be worth it. The code base as I understand it also wildly doesn't support this, so it would be a major change. I am willing to take a crack at it though.

Neither of these options would result in consistent behavior throughout the go-task though.

In my personal opinion, both options are better than the current behavior.

[pd93]

I've never fully understood the appeal of putting a full shell script inside of a Taskfile. I understand it for simple 1/2 line cases, but as soon as you're doing anything vaguely complex. you're losing out of a bunch of functionality by writing the script in the Taskfile. You can't use tools like Shellcheck, you get no syntax highlighting, and as you said, splitting lines of the script into separate commands means that variables/venvs are not persisted.

Why not just create an sh/bash/zsh file and call it? This feels far cleaner than introducing a complex feature into Task which may introduce confusion in advanced use cases (like you mentioned with deps/subtasks).

[richard-hajek]

For cleaner project directory most immediately. But also if you're gonna have multiple scripts, having a taskfile.yaml at that point, which would just call these scripts, is vastly reducing the utility of the taskfile. I am perfectly capable of calling the scripts myself

But I will probably do it this way in the end ( Taskfile calling the scripts ), for all the reasons you mentioned. And taskfile provides a single entrypoint which is also a very good value utility.

[richard-hajek]

With that being said, the current behavior is still unexpected to newcomers coming from build scripts such as myself. Maybe a note here https://taskfile.dev/usage/#getting-started or https://taskfile.dev/api/#command ? With the expected workflow of calling .(ba/z/)sh scripts?

[pd93]

For cleaner project directory most immediately

This is the only argument I can see. However, (just my opinion) I don't actually see merging different languages syntaxes into a single file as "clean". This is the same reason noone writes their CSS inline in their HTML files (forgive me, I haven't done frontend for many years 😆). It's much cleaner to separate them and have dedicated syntax highlighting and linting for both.

But also if you're gonna have multiple scripts, having a taskfile.yaml at that point, which would just call these scripts, is vastly reducing the utility of the taskfile. I am perfectly capable of calling the scripts myself

The location of the script makes no difference as to how you call the task. You're still just running task mytask. Having the ability to also manually call the script doesn't change this and you're still able to use all the same features of Task that you would if the script is inline.

With that being said, the current behavior is still unexpected to newcomers coming from build scripts such as myself. Maybe a note here https://taskfile.dev/usage/#getting-started or https://taskfile.dev/api/#command ? With the expected workflow of calling .(ba/z/)sh scripts?

This is an extremely frequently asked question. Generally, we would expect users coming from other build/task systems such as make to be used to this functionality, since they work the same way. However, I agree that adding something to our FAQ or style guide is probably a good idea.

[richard-hajek]

One of the reasons of my departure from make was exactly this behavior haha, in addition to the obscure syntax which breaks on whitespaces with undecipherable errors.

However, I agree that adding something to our FAQ or style guide is probably a good idea.

Please do. Also make the tone as strong as possible and offer solutions with reasons we outlined so:

Q: How to use (Python?) virtual environments in a Taskfile.yaml?
A: Please do not. This is a known request but Taskfile is not intended to handle this case. Where shell interaction is necessary, put scripts in a separate file and run them from the Taskfile. See #1115 for more details.

Q: How to runtime persist shell state and variables across commands?
A: Do not. See above.

[richard-hajek]

If the docs will be address, I will consider this issue resolved and this proposal in its original form rejected, which is fine though as more beneficial work flows have emerged.

[pd93]

I will leave as-is for now, so that others have time to add their thoughts to this issue. If there is no change in consensus, we can make the changes to the docs.

[richard-hajek]

However in support of the proposal, consider have the following example

tasks:
   build:
      cmds:
          - conda activate buildenv
          - python setup.py build

This really isn't a complicated use case. Having a separate file script for these two commands feels very wrong, unintuitive, messy and cumbersome.

[postlund]

You can accomplish several shell commands treated as one task like this:

tasks:
   build:
      cmds:
          - |
            conda activate buildenv
            python setup.py build

That's what you want, right?

[pd93]

I think for these simpler cases, it is reasonable to do one of the following instead of using a script

cmds:
 - conda activate buildenv && python setup.py build

Or

cmds:
 - |
   conda activate buildenv
   python setup.py build

Edit: @postlund took the words right out of my mouth 😛

[postlund]

@pd93 Just a tad quicker 😉

[richard-hajek]

You can accomplish several shell commands treated as one task like this

That is true, I forgot about that. Still unintuitive for a newcomer though