[BUG] gitfs fileserver does not sync to all_saltenvs for remote

[dapak]

Description
When utilizing the all_saltenvs option for a gitfs remote resource, you would expect that the resource would be synced to all environments identified by gitfs specified by file_roots in the master configuration. The resource however is only synced to the base environment.

Setup
The following is the master configuration:

git_pillar_provider: pygit2

fileserver_backend:
  - roots
  - git

gitfs_remotes:
  - https://github.com/saltstack-formulas/vim-formula:
    - all_saltenvs: v0.15.5
    - root: vim
    - mountpoint: salt://vim

file_roots:
  base:
    - /srv/salt/states
  dev:
    - /srv/salt/states

pillar_roots:
  base:
    - /srv/salt/pillar/base
  dev:
    - /srv/salt/pillar/dev
    - /srv/salt/pillar/base

pillar_source_merging_strategy: recurse
pillarenv_from_saltenv: True
top_file_merging_strategy: same

The following is the additional minion configuration:

pillarenv: dev
saltenv: dev

The following is the directory structure of /srv/salt:

[root@lima-salt-master salt]# pwd
/srv/salt
[root@lima-salt-master salt]# tree
.
├── formulas
├── pillar
│   ├── base
│   │   ├── hello
│   │   │   └── world.sls
│   │   ├── nginx
│   │   │   └── init.sls
│   │   └── top.sls -> /srv/salt/pillar/top.sls
│   ├── dev
│   │   └── top.sls -> /srv/salt/pillar/top.sls
│   └── top.sls
└── states
    └── top.sls

7 directories, 6 files

The contents of the states/top.sls:

{{ saltenv }}:
  '*':
    - vim

Please be as specific as possible and give set-up details.

• [X ] VM (lima virtualized instance of Oracle Linux 8 on a MacOS host)

Steps to Reproduce the behavior
The above configuration syncs the vim formula resource to the base environment correctly:

[root@lima-salt-master salt]# salt-run fileserver.file_list
- top.sls
- vim/_mapdata/_mapdata.jinja
- vim/_mapdata/init.sls
- vim/absent.sls
- vim/editor.sls
- vim/files/nerdtree/doc/NERD_tree.txt
- vim/files/nerdtree/nerdtree_plugin/exec_menuitem.vim
- vim/files/nerdtree/nerdtree_plugin/fs_menu.vim
- vim/files/nerdtree/plugin/NERD_tree.vim
- vim/files/nerdtree/syntax/nerdtree.vim
- vim/files/pyflakes/ftplugin/python/pyflakes.vim
- vim/files/pyflakes/ftplugin/python/pyflakes/LICENSE
- vim/files/pyflakes/ftplugin/python/pyflakes/NEWS.txt
- vim/files/pyflakes/ftplugin/python/pyflakes/README.rst
- vim/files/pyflakes/ftplugin/python/pyflakes/bin/pyflakes
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/__init__.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/checker.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/messages.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/scripts/__init__.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/scripts/pyflakes.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/test/__init__.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/test/harness.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/test/test_imports.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/test/test_other.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/test/test_script.py
- vim/files/pyflakes/ftplugin/python/pyflakes/pyflakes/test/test_undefined_names.py
- vim/files/pyflakes/ftplugin/python/pyflakes/setup.py
- vim/files/salt/ftdetect/jinja.vim
- vim/files/salt/ftdetect/sls.vim
- vim/files/salt/ftplugin/rst.vim
- vim/files/salt/ftplugin/sls.vim
- vim/files/salt/syntax/jinja.vim
- vim/files/salt/syntax/sls.vim
- vim/files/vimrc
- vim/init.sls
- vim/map.jinja
- vim/nerdtree.sls
- vim/pyflakes.sls
- vim/salt.sls

It does not sync the resource to the dev environment:

[root@lima-salt-master salt]# salt-run fileserver.update
True
[root@lima-salt-master salt]# salt-run fileserver.file_list saltenv=dev
- top.sls

Additional list of the environments on the fileserver:

[root@lima-salt-master salt]# salt-run fileserver.envs
- base
- dev
- v0.14.2
- v0.15.0
- v0.15.1
- v0.15.2
- v0.15.3
- v0.15.4
- v0.15.5

Expected behavior
I would expect that the vim formula also be present within the dev environment.

Versions Report

salt --versions-report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

Salt Version:
          Salt: 3004.1

Dependency Versions:
          cffi: 1.11.5
      cherrypy: unknown
      dateutil: 2.6.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.10.1
       libgit2: 0.26.8
      M2Crypto: 0.35.2
          Mako: Not Installed
       msgpack: 0.6.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.14
      pycrypto: Not Installed
  pycryptodome: Not Installed
        pygit2: 0.26.4
        Python: 3.6.8 (default, Nov 10 2021, 06:50:23)
  python-gnupg: Not Installed
        PyYAML: 3.12
         PyZMQ: 19.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: oracle 8.5
        locale: UTF-8
       machine: x86_64
       release: 5.4.17-2136.302.7.2.2.el8uek.x86_64
        system: Linux
       version: Oracle Linux Server 8.5

[welcome]

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey.
Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

• Community Wiki
• Salt’s Contributor Guide
• Join our Community Slack
• IRC on LiberaChat
• Salt Project YouTube channel
• Salt Project Twitch channel

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar.
If you have additional questions, email us at saltproject@vmware.com. We’re glad you’ve joined our community and look forward to doing awesome things with you!

[OrangeDog]

This was discussed on Slack, and this is not what the problem was. Indeed, this is the config that you said worked correctly,

What wasn't working was using __env__ in the file_roots.

[dapak]

@OrangeDog

This config does not work correctly per the output of fileserver.file_list for the dev environment above. It should have the vim formula synced to it, assuming all_saltenvs is designed to sync a resource to all salt environments. Like many things, the documentation isn't 100% clear on that so I could be quite wrong.

The __env__ issue I think might be related to this problem. I can file a separate bug for that if you prefer, but I'm still having the problem with specifying even dev in and not even using a configuration with __env__.

[OrangeDog]

Oh, I thought it was solved in the end.
Still no idea why this doesn't work for you then.

[dapak]

@OrangeDog

Unfortunately nope, I'm still scratching my head on this one. I feel like I've tried every combination of configuration at this point to get this to work and I've given up. I've been doing configuration management work with other tooling for over a decade and Salt's breaking my brain on this particular feature.

[cordu-dev]

Hi everybody, I am also struggling with this. It is very strange. I have a simple master config combining roots and git fileserver backends. I use gitfs for a bunch of formulas. But when I run salt to apply states with a different saltenv (other than base) it is going to crash and burn - all my states from formulas are "not available" anymore.

    Specified SLS postgres in saltenv staging is not available on the salt master or through a configured fileserver

Master config snippet:

fileserver_backend:
  - roots
  - git

file_roots:
  staging:
    - /staging/path/to/my/salt/states
  prod:
    - /prod/path/to/my/salt/states

gitfs_provider: pygit2
gitfs_remotes:
  - https://github.com/saltstack-formulas/postgres-formula:
      - all_saltenvs: master

pillar_roots:
  staging:
    -  /staging/path/to/my/salt/pillar
  prod:
    -  /prod/path/to/my/salt/pillar

[leifb]

I'm still having the same problem with salt-master 3006.1. The all_saltenvs option is not working at all.

After a lot of trail and error I was able to figure out two things:

1. Some configuration changes seem to not have any effect unless you manually delete the gitfs cache.
   On my install, the cache is located here: /var/cache/salt/master/gitfs.
   I did not test which of the options were not applied, I just remember that salt-run fileserver.file_list saltenv=development suddenly had an output after deleting the cache manually.
2. I was able to find a workaround by defining the ref for every saltenv I use.

Here is the full config I use that works for me:

fileserver_backend:
  - roots
  - gitfs

gitfs_provider: pygit2  
gitfs_disable_saltenv_mapping: True
top_file_merging_strategy: same

pillar_roots:
  __env__:
    - /srv/salt/pillar

file_roots:
  __env__:
    - /srv/salt/states

gitfs_remotes:
  - https://github.com/saltstack-formulas/letsencrypt-formula.git:
    - saltenv:
      - development:
        - ref: v3.2.0
      - production:
        - ref: v3.2.0
      - test:
        - ref: v3.2.0