Shell scripts within preconditions and vars do not surface using `--dry`

[adamscybot]

Description

Currently, task <task-name> --dry does not display the full scope of what we will execute. Namely, any shell script defined as part of preconditions or vars will not be printed. Only the cmd/cmds content is printed.

Since nothing can stop bad behaviour in those places, e.g. mutating something on disk or whatever else, it prevents --dry from being used accurately for one of its primary use cases, which is to just sense check what something is going to do (especially if project is untrusted).

This is perhaps secondary, but it also prevents using --dry to get a full picture of what what is going on for debugging purposes (post-template-evaluation output that --dry has now is useful). In the case of preconditions you can't use it to check what the precondition is based on. In the case of vars, any associated shell scripts not being printed prevents quick analysis of the totality of all the inputs/evaluations of a task execution.

Version

3.44.1

Operating system

Debian

Experiments Enabled

None

Example Taskfile

version: '3'

vars:
  EXAMPLE_VAR:
    sh: echo 'You can't see me in vars(◣_◢)' && rm -rf ./somethingbad

tasks:
  example:
    preconditions:
      - sh: echo 'You can't see me in preconditions (◣_◢)' && rm -rf ./somethingbad
        msg: 'API_KEY environment variable is required'
    cmds:
      - echo "example"

[vmaerten]

Currently, vars and preconditions are not shown in --dry, but they are still executed. Only cmds and deps are skipped.
I can see why it might make sense for preconditions not to run in dry mode, but how can we get a complete picture of what’s happening if shell variables aren’t interpreted? Considering also that a variable can reference another variable.

@andreynering you categorized this as a bug, what was your idea behind this ?
My proposal would be to skip the execution of preconditions while still displaying them (similar to how cmd is shown). Should we perhaps display them consistently, not only in --dry mode?

For vars, however, I’d suggest keeping the current behavior as is.

cc @pd93 in case you have thoughts on this.

[andreynering]

Hmm... I didn't give this a lot of thought when assigned as a bug.

Yeah, the current behavior is probably by design, but we can still consider improving it. No strong opinions, though.

[adamscybot]

Yeah, sorry folks -- I did flip-flop myself when deciding if this was a bug or a feature req. My thought process at the time for raising a bug was "the intent of the dry run feature is to make it so I can see all of what will execute that is defined within the taskfile(s), probably for safety/security". But actually reading it back, I now feel this interpretation of the intent was complete insanity on my part, and I have no idea how my brain got here -- not enough sleep I guess 😂.

Currently, vars and preconditions are not shown in --dry, but they are still executed

but how can we get a complete picture of what’s happening if shell variables aren’t interpreted?

@vmaerten Yeh. I now suddenly realise it is literally impossible to solve what I wanted with dry run in the traditional sense. The only way to get a picture of what is going to happen is to look at the file.

The only ways taskfile can conceivably help is in very minimal ways (like the --show flag that just has). But the legwork is still with the executor (the human being), and it has to be. It could also provide some kind of richer output showing the vars getting assigned (the value, and the command that was evaluated to reach it). But this is now straying into observability/debuggability rather than a pre-emptive "audit" and so I'm stretching.

Perhaps their could be something in A) making precondition and vars shell execution visible, and then b) having something like --vars (justfile has this) which outputs only the evaluated vars. And maybe a sister --preconditions/--check-condition or similar which would output only if a task can evaluate. Conceivably, a user could then C) combine these with --dry, in order to manually "step through" what will happen more incrementally via repeated calls.

Happy to close this and hang my head in shame unless you think there's meat in the above :).

Actually, I think perhaps the outcome is there could just be a line in the docs under --dry just pre-warning/being explicit that project-defined shell stuff is still potentially gonna execute so don't incorrectly assume that it has guarantees around "doing nothing" like I wrongly did :D. If you don't trust it, containerise it or something or do the legwork of analysing the tasks. I would be happy to raise that if it makes it sense.