XDG Base Directory

[noraj]

The idea would be to follows XDG Base Directory Specification standard.

Instead of only looking at ~/.config/nuclei/config.yaml, nuclei would resolve $XDG_CONFIG_HOME and try global config too. So with default value, for example, it could look for (in order):

1. $XDG_CONFIG_HOME/nuclei/config.yaml (in most cases would resolve to ~/.config/nuclei/config.yaml) to look for user-defined config
2. /etc/nuclei/config.yml (system, global, for all users, default config)
3. Hardcoded ~/.config/nuclei/config.yaml fallback

Same for templates, instead of dumping them at ~/nuclei-templates, it would look at:

1. $XDG_DATA_HOME/nuclei/templates/ (in most cases would resolve to $HOME/.local/share/nuclei/templates/)
2. $XDG_DATA_DIRS/nuclei/templates/ 1st value (in most cases would resolve to /usr/local/share/nuclei/templates/)
3. $XDG_DATA_DIRS/nuclei/templates/ 2nd value (in most cases would resolve to /usr/share/nuclei/templates/)
4. Hardcoded ~/nuclei-templates fallback

That would help to comply with standard paths, ensure to respect users custom structures, ease distribution packaging, unmess the root of user home directory.

Related issues on BlackArch Linux for packaging nuclei

• nuclei-templates not used by nuclei package BlackArch/blackarch#3382
• nuclei: add nuclei-template support BlackArch/blackarch#4357
• https://github.com/BlackArch/blackarch/blob/master/packages/nuclei/config.yaml
• nuclei: remove forced -config flag. BlackArch/blackarch#4723
• nuclei: revert s7x change BlackArch/blackarch#4726
• https://wiki.archlinux.org/title/XDG_Base_Directory

[dwisiswant0]

Config loading¹ in Nuclei is intentionally based on std os.UserConfigDir (handled through the github.com/projectdiscovery/goflags package), which resolves to OS-specific, well-defined paths:

On Unix systems, it returns $XDG_CONFIG_HOME as specified by https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html if non-empty, else $HOME/.config. On Darwin, it returns $HOME/Library/Application Support. On Windows, it returns %AppData%. On Plan 9, it returns $home/lib.

So Nuclei is already compliant with platform conventions.

For distro setups, you can run it once with -update-template-dir to put templates wherever you want.

$ nuclei -update-template-dir /tmp/nuclei-templates

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.6.1

		projectdiscovery.io

[INF] nuclei-templates are not installed, installing...
[INF] Successfully installed nuclei-templates at /tmp/nuclei-templates
[INF] Current nuclei version: v3.6.1 (latest)
[INF] Current nuclei-templates version: v10.3.6 (latest)
[INF] New templates added in latest release: 176
[INF] Templates loaded for current scan: 9080
[INF] Executing 9078 signed templates from projectdiscovery/nuclei-templates
[WRN] Loading 2 unsigned templates for scan. Use with caution.
[INF] Scan completed in 89.400172ms. No results found.
$ jq -r . ~/.config/nuclei/.templates-config.json
{
  "nuclei-templates-directory": "/tmp/nuclei-templates",
  "custom-s3-templates-directory": "/tmp/nuclei-templates/s3",
  "custom-github-templates-directory": "/tmp/nuclei-templates/github",
  "custom-gitlab-templates-directory": "/tmp/nuclei-templates/gitlab",
  "custom-azure-templates-directory": "/tmp/nuclei-templates/azure",
  "nuclei-templates-version": "v10.3.6",
  "nuclei-ignore-hash": "1a666944243307efff85d5bbd943b190",
  "nuclei-latest-version": "v3.6.1",
  "nuclei-templates-latest-version": "v10.3.6",
  "nuclei-latest-ignore-hash": "1a666944243307efff85d5bbd943b190"
}

And that path gets saved in the config, and Nuclei will keep using it from then on.

If there are edge cases or distro-specific requirements where that approach falls short, feedback would be very welcome so we can evaluate potential improvements.

Footnotes

1. I've also drafted the nuclei-action v3 spec that explain how the config system works. See https://github.com/projectdiscovery/nuclei-action/discussions/104. ↩

[noraj]

Nice to know for the template directory that there is a nuclei-templates-directory node in the config. However for the config location that was not the point. The point was, right now, nuclei is only looking for one location (which is returned by os.UserConfigDir), there is no system for system/global config + user config, only a user config. Which is not handy for packaging and system distribution. It means each user will have it's own config setup at the first lunch, and now way for the distribution maintainer to provide a custom configuration. To workaround that, we have made a dirty command wrapper that forces config /usr/share/nuclei/config.yaml but that has the drawback to prevent users to have their own configuration. Advanced tools often have a fallback system, where they look for user config, if it does not exist fallback on system config and if does not exist either fallback to a default config or offer the user to copy a default config as their user config.

For example zsh will looks for ~/.zshrc and if does not exist then /etc/zsh/zshrc. The same system for nuclei would mean mean that distribution maintainers could provide a custom configuration, for example in /usr/share/nuclei/config.yaml, when the package is installed, deploy it at /etc/nuclei/config.yaml. People by default would use the system wide config at /etc/nuclei/config.yaml, but they would have the liberty to overwrite it with their own $XDG_CONFIG_HOME/nuclei/config.yaml.

[dwisiswant0]

Right, thanks for the clarification! I see the distinction you're making now.

While Nuclei already resolves user config paths correctly via os.UserConfigDir, it is still missing a proper config hierarchy (i.e. a fallback mechanism). Atm, user config is treated as the only source of truth instead of the final layer in a config stack.

I'll go ahead draft new RFC for config loader model.

[dwisiswant0]

I've drafted an proposal for a new config loader, would love your thoughts on #6755. Any feedback helps us prioritize this.