Working directory resolved incorrectly for nested includes

[Diaphteiros]

Description

Setup

./
├── Taskfile.yaml
└── common/
    ├── Taskfile.yaml
    └── tasks-development.yaml

The main Taskfile imports the one from the common subdirectory, which imports the tasks-development.yaml one.

Taskfiles

Taskfile.yaml

version: 3

includes:
  shared:
    taskfile: common
    flatten: true

common/Taskfile.yaml

version: 3

includes:
  dev:
    taskfile: tasks-development.yaml

tasks:
  pwd:
    desc: Print the current working directory.
    cmds:
      - pwd

common/tasks-development.yaml

version: 3

tasks:
  pwd:
    desc: Print the current working directory.
    cmds:
    - pwd

Expected Behavior

> task pwd
task: [pwd] pwd
/var/folders/b8/g43jjlg51gs4h13yf49sjmjm0000gn/T/tmp.6yctrFQfZM/tasktest

> task dev:pwd
task: [dev:pwd] pwd
/var/folders/b8/g43jjlg51gs4h13yf49sjmjm0000gn/T/tmp.6yctrFQfZM/tasktest

Actual Behavior

Tasks from the tasks-development.yaml Taskfile use the common subdirectory as working directory.

> task pwd
task: [pwd] pwd
/var/folders/b8/g43jjlg51gs4h13yf49sjmjm0000gn/T/tmp.6yctrFQfZM/tasktest

> task dev:pwd
task: [dev:pwd] pwd
/var/folders/b8/g43jjlg51gs4h13yf49sjmjm0000gn/T/tmp.6yctrFQfZM/tasktest/common

I tried to set dir on various includes, but neither . nor {{.ROOT_DIR}} seem to have any effect on this issue.

Version

v3.41.0

Operating system

macOS (darwin/arm64)

Experiments Enabled

None

[trulede]

Hi, documentation is clear how this works:
https://taskfile.dev/usage/#directory-of-included-taskfile

so you need to set dir on the importing Taskfile, not the imported Taskfile.

version: 3

includes:
  shared:
    taskfile: common
    flatten: true
    dir: ,/common

[Diaphteiros]

@trulede
OK, I understand your thinking, but it was based on a wrong include that was left in the examples from experiments I did. I removed that line from the examples.

I would like the working directory for all tasks in this example - independent of which file they are defined in - to be the root directory. If I do not add any dir statements, this is not the case - the dev:pwd target returns the common subdirectory.
I did not find any way to achieve this. Setting either . or '{{.ROOT_DIR}} in the dir statement of the root Taskfile does not solve the problem.

[hkupty]

I want to emphasize the point of @Diaphteiros as I'm having the same issue. My setup looks something like this:

# ./Taskfile.yaml
includes:
  k8s:
    taskfile: ./taskfiles/k8s.yaml
    dir: '{{.ROOT_DIR}}'
  tools:
    taskfile: ./taskfiles/tools.yaml
    dir: '{{.ROOT_DIR}}'

# taskfiles/k8s.yaml
version: "3"
includes:
  tools:
    taskfile: ./tools.yaml
    dir: '{{.ROOT_DIR}}'
    internal: true

And yet the task in tools yields pwd .../taskfiles. This combined with the issue of absolute paths still being considered relative paths makes it very difficult to properly set the task's directory without causing issue.

[trulede]

Can you provide a complete example? Just because, its messy, it's good to have a task that prints the var at each step of the include chain. I suspect here, the multiple include path complicates things.

I'm not sure that ROOT_DIR is not working incorrectly, so what it returns in the included taskfile (k8s) might be correct. Code is difficult. You might also try USER_WORKING_DIR.

I do have a workaround, pass the directory you want used as a variable to the imported task. If its set, then use that, otherwise, use something else.

[hkupty]

So I reduced the taskfiles to a working example:

# taskfiles/k8s.yaml
---
# yaml-language-server: $schema=https://taskfile.dev/schema.json
version: "3"
includes:
  tools:
    taskfile: ./tools.yaml
    dir: '{{.ROOT_DIR}}'
    internal: true
tasks:
  matrix:
    desc: Builds the matrix of services to be deployed
    deps:
      - task: tools:install-helper
    cmds:
      - bin/helper build-matrix

# taskfiles/tools.yaml
---
# yaml-language-server: $schema=https://taskfile.dev/schema.json
version: "3"
tasks:
  build-go-tool: # Hijacked this task to debug what I'm getting as output
    requires:
      vars:
        - TOOL
    internal: true
    silent: true
    vars:
      BASE: '{{default .TASK_DIR .USER_WORKING_DIR}}'
      TARGET: '{{joinPath .ROOT_DIR "tools" .TOOL}}'
    cmds:
      - echo TASK_DIR         '{{ .TASK_DIR }}'
      - echo USER_WORKING_DIR '{{ .USER_WORKING_DIR }}'
      - echo BASE             '{{ .BASE }}'
      - echo pwd              `pwd`
  install-*:
    vars:
      TOOL: '{{index .MATCH 0}}'
    silent: true
    status:
      - test -f {{ joinPath "bin" .TOOL }}
    cmds:
      - task: build-go-tool
        vars:
          TOOL: '{{.TOOL}}'
      - mv '{{ joinPath "tools" .TOOL .TOOL }}' '{{ joinPath "bin" .TOOL}}'
      - chmod +x '{{ joinPath "bin" .TOOL }}'
# Taskfile.yaml 
---
# yaml-language-server: $schema=https://taskfile.dev/schema.json
version: "3"
includes:
  k8s:
    taskfile: ./taskfiles/k8s.yaml
    dir: '{{.ROOT_DIR}}'
  tools:
    taskfile: ./taskfiles/tools.yaml
    dir: '{{.ROOT_DIR}}'

And here's the difference in output when I run task k8s:matrix and task tools:install-helper. Note that the latter is getting the correct path while the former is not:

❯ go-task k8s:matrix
TASK_DIR /tmp/debug-taskfile/taskfiles
USER_WORKING_DIR /tmp/debug-taskfile
BASE /tmp/debug-taskfile
pwd /tmp/debug-taskfile/taskfiles
mv: cannot stat 'tools/helper/helper': No such file or directory
exit status 1
❯ go-task tools:install-helper
TASK_DIR /tmp/debug-taskfile
USER_WORKING_DIR /tmp/debug-taskfile
BASE /tmp/debug-taskfile
pwd /tmp/debug-taskfile
mv: cannot stat 'tools/helper/helper': No such file or directory
task: Failed to run task "tools:install-*": exit status 1

❯ cd taskfiles
❯ go-task k8s:matrix
TASK_DIR /tmp/debug-taskfile/taskfiles
USER_WORKING_DIR /tmp/debug-taskfile/taskfiles
BASE /tmp/debug-taskfile/taskfiles
pwd /tmp/debug-taskfile/taskfiles
mv: cannot stat 'tools/helper/helper': No such file or directory
exit status 1
❯ go-task tools:install-helper
TASK_DIR /tmp/debug-taskfile
USER_WORKING_DIR /tmp/debug-taskfile/taskfiles
BASE /tmp/debug-taskfile/taskfiles
pwd /tmp/debug-taskfile
mv: cannot stat 'tools/helper/helper': No such file or directory
task: Failed to run task "tools:install-*": exit status 1

The problem with USER_WORKING_DIR is that I'd be forcing the command to be executed from the top-level directory of the repository. It is not a huge issue, just an inconvenience.

[trulede]

For the include graph A->B->C it seems like the variable {{.ROOT_DIR}} might not be available at the time the included include (B->C) is evaluated. None of the special variables seem to be available. What you get in C is then the working dir of B and whatever else resolved.

Taskfile.yml (A)

version: '3'

includes:
  B:
    taskfile: B/Taskfile.yml
    dir: B

tasks:
  default:
    cmds:
      - echo 'aaaaaaaa'
      - echo "PWD $(pwd)"
      - echo 'ROOT_DIR = {{.ROOT_DIR}}'
      - echo 'USER_WORKING_DIR = {{.USER_WORKING_DIR}}'
      - task: B:b

B/Taskfile.yml (B)

version: 3

includes:
  C:
    taskfile: '../C/Taskfile.yml'
    dir: '{{.ROOT_DIR}}/C'

tasks:
  b:
    cmds:
      - echo 'bbbbbbbb'
      - echo "PWD $(pwd)"
      - echo 'ROOT_DIR = {{.ROOT_DIR}}'
      - echo 'USER_WORKING_DIR = {{.USER_WORKING_DIR}}'
      - task: C:c

C/Taskfile.yml (C)

version: 3

tasks:
  c:
    cmds:
      - echo 'cccccccc'
      - echo "PWD $(pwd)"
      - echo 'ROOT_DIR = {{.ROOT_DIR}}'
      - echo 'USER_WORKING_DIR = {{.USER_WORKING_DIR}}'

From the output of C you can see that dir was resolved to /C rather than /workspace/task/C.

task: cannot make directory "/C": mkdir /C: permission denied
task: [B:C:c] echo 'cccccccc'
cccccccc
task: [B:C:c] echo "PWD $(pwd)"
PWD /C
task: [B:C:c] echo 'ROOT_DIR = /workspaces/task'
ROOT_DIR = /workspaces/task
task: [B:C:c] echo 'USER_WORKING_DIR = /workspaces/task/cmd/task'
USER_WORKING_DIR = /workspaces/task/cmd/task

[trulede]

This is where the import path is resolved by the templator:
https://github.com/trulede/task/blob/7f16745077e6c255cf394c60b7836e64f2c0a586/taskfile/reader.go#L199

At that point you have environment variables, and any variables defined in the taskfile. So none of the special variables are in the templating cache, and none of the vars provided by the importing task. That explains the behavior.

I don't think you win that battle. This is what I'm doing, if it helps:

Taskfile.yml

version: '3'

vars:
  PLATFORM_ARCH: linux-amd64
  OUTDIR: 'out'
  SIMDIR: 'sim'

includes:
  dse.fmi-v1.1.15:
    taskfile: https://raw.githubusercontent.com/boschglobal/dse.fmi/refs/tags/v1.1.15/Taskfile.yml
    dir: '{{.OUTDIR}}/{{.SIMDIR}}'
    vars:
      FMI_IMAGE: ghcr.io/boschglobal/dse-fmi
      FMI_TAG: 1.1.15
      SIM: '{{.SIMDIR}}'
      ENTRYWORKDIR: '{{.PWD}}/{{.OUTDIR}}'

and then in the remote taskfile

---
version: '3'

vars:
  # When running from E2E tests (i.e. Docker in Docker), the ENTRYDIR (for
  # Docker commands) must be set to the host relative path.
  ENTRYDIR: '{{if .SIM}}{{.ENTRYWORKDIR}}/{{.SIM}}{{else}}{{.PWD}}{{end}}'


tasks:

  generate-fmimcl:
    desc: Generate an FMI MCL from an existing FMU.
    run: always
    dir: '{{.USER_WORKING_DIR}}'
    label: dse:fmi:generate-fmimcl
    vars:

in my case, I run the imported tasks in USER_WORKING_DIR (see dir:), however you could probably use one of the passed variables such as ENTRYDIR which will be set to ENTRYWORKDIR set from the importing taskfiles vars.

[ChrisPbb]

I have found a workaround for this very fun issue: use dir: '{{env "PWD"}}' on your nested includes

[cenk1cenk2]

I have found a workaround for this very fun issue: use dir: '{{env "PWD"}}' on your nested includes

Thats a very nice and clean solution indeed.

[mvarchdev]

I bumped in this right now... everything was working like expected, until we put include to object format because we wanted to add internal: true :/ spent a lot of time on this one, do you know WHY is this happening? I mean, from the code/logic perspective...

For now, we decided to use global variables