Support for nested stacks directory

[mkoo21]

⚠️⚠️⚠️ Since we do not accept all types of pull requests and do not want to waste your time. Please be sure that you have read pull request rules:
https://github.com/louislam/dockge/blob/master/CONTRIBUTING.md

Tick the checkbox if you understand [x]:

• I have read and understand the pull request rules.

Description

This PR accompanies the discussion in #214.

Currently, dockge assumes all stacks are exactly one level deep in the stacks directory, and the "name" of a project equals the name of the folder containing it. This assumption is not quite right, and leads to weird behavior where dockge can be "tricked" into thinking a random folder is a docker compose project, or will simply not distinguish between two separate projects.

Some examples of where this assumption breaks:

• you have multiple projects living in the same folder, perhaps with more than one config file (e.g. $stacksDir/project/compose-dev.yml and $stacksDir/project/compose-prod.yml). If there is also a compose.yml dockge will assign both projects the name "project" and not distinguish between them.
• you have multiple instances of one project (e.g. $stacksDir/application1/authentik and $stacksDir/application2/authentik). Dockge will assign both projects the name authentik and will not distinguish them.
• All of these also apply in case of a naming conflict with a project that is run from outside the configured stacks directory (e.g. $notStacksDir/application3/authentik will produce yet another naming conflict).
• If the config file name (like compose-dev.yml is not in the acceptedComposeFileNames) dockge will simply not work with this project. This behavior is undocumented.

In these cases the user will tell docker compose to tag the project with different names. So docker compose will be aware of the difference but dockge will behave strangely. Plus, people with many (say, 20+) projects running on one machine will most certainly want to use some nesting to organize their stacks directory. For these reasons, I think letting docker compose should be the main source of truth.

A real example I had while testing this is I had a server directory I want to use as my stacksDir that contains a folder called authentik. This folder is actually a volume mount for my authentik project, and does not contain a config file. The actual authentik stack is running from a different location on my machine. Even though dockge tries to skip folders that don't contain an acceptedComposeFilename, the name of the folder matches the name of a project that appears in the output of docker compose ls, so dockge thinks it is a project managed by dockge and then says it cannot find the configuration file:

This PR treats docker compose ls as the main source of truth, and puts the folder scan afterwards as a "nice to have" for discovering unknown projects. And it fixes the issue described above!

There are two commits in this PR that cover the two pieces of functionality changed. They can be reviewed separately, and the first half can be merged independently if you don't like the second. This changes some existing functionality, but everything that used to work should work the same as before, if not slightly better.

The first part focuses on the stacks.ts logic that assumes a stack "managed by dockge" means it lives in the folder $stacksDir + $projectName. It asks docker compose for a list of projects and treats all the projects where the config file starts with stacksDir as being managed by dockge (current master does a folder search then asks docker compose for services not managed by dockge, but they all go into the string map which can cause conflicts).

The second part restores dockge's ability to "discover" projects that had never been run through the CLI, by adding the folder search after the call to docker compose. But it makes that search recursive using node FS. In case of a naming conflict between a "known" project and a "discovered" project, it keeps the "known" project but adds some extra logging and makes dockge's behavior extra transparent. Naming conflicts between projects "known" to docker compose will still happen if the user makes an error using docker compose (e.g.. not tagging projects with names), but at least dockge's behavior will match docker compose's.

Fixes #(issue)
#214

Type of change

Please delete any options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Checklist

• My code follows the style guidelines of this project
• I ran ESLint and other linters for modified files
• I have performed a self-review of my own code and tested it
• I have commented my code, particularly in hard-to-understand areas
  (including JSDoc for methods)
• My changes generate no new warnings
• My code needed automated testing. I have added them (this is optional task)

Screenshots (if any)

Please do not use any external image service. Instead, just paste in or drag and drop the image here, and it will be uploaded automatically.

[tobi1449]

I've played around a bit with this PR and it seems to work great.
One issue I noted however is that it prevents you from creating a new stack via dockge itself.
See cmcooper1980@cbfe3f3 for a potential fix.

[smileBeda]

How could I help test this? I have a particular Docker stack that is probably profiting from this feature:

1. ~/Docker/servicename/docker-compose.yml > OK, Dockge gets it
2. ~/Docker/websites/singlewebsite/docker-compose.yml ? NOK, Dockge sees the service but does not manage it
3. ~/Docker/websites/anothersinglewebsite/docker-compose.yml ? NOK, Dockge sees the service but does not manage it

Reason I have this stack:

• ~/Docker/ has several very distinct services (service name, eg: jellyfin, manager, gitea) and thus all good to keep them in first level
• ~/Docker/websites/ has only actual websites running (eg singlewebsite, anothersinglewebsite), so I bundled them into one folder to keep things a bit more organized. These also have other types of file permissions etc.

I believe the feature here is exactly what I need and would like to help testing it if possible :)

[mkoo21]

@smileBeda It's a normal node/JS application without anything unusual. You need to have an appropriate version of NodeJS and npm || pnpm || yarn installed, then you should be able to npm install && npm run build:frontend && npm run start from this repo, or do something like node ./extra/env2arg.js docker buildx build -t dockge-test -f ./docker/Dockerfile . (this will create an image named dockge-test) if you want to use docker. You can look in the "scripts" section of the package.json and run those with npm run $NAME, and in the docker subdirectory for more information

[pepsipu]

would love if this was merged 🙏

[axelfony]

Merge would be brilliant for those of us that manage their docker-compose.yml in the respective application parent directory.

[diaodeng]

感谢各位大佬的共享，我在合并这些拉去请求的代码到我的仓库分支，在我的环境中有多个dockge，代理客户机中与本机的容器有同名的，而且代理客户机与本机的dockge目录映射不一致，会导致同名的容器无法管理，因为获取到的是另一个（我这里是本机中的无法管理，获取到的是代理客户机的stack目录）dockge的stack目录，导致到不到compose脚本执行失败，我尝试改了下，但没成功，我没有能力解决这个问题，能帮忙看一下

[Dracrius]

Great idea but needs work. By stripping all the async from the search this PR makes dockge extremely sluggish to respond to even refreshing the page as it will take 40 seconds to show a stack list.

[mkoo21]

@diaodeng it's the way docker compose works that the project name is inferred from the folder name which leaves the possibility of conflicts. In these cases it is up to you to specify project names to avoid conflicts

@Dracrius that's a bit surprising, it does not take me 40 seconds to recursively list all files from the root directory of my server. But as I put in the description I split the PR up to account for this, you can just take commit fc4ad7f to get the list from docker compose without the search feature. If you are using it on a dev machine or something and there are obvious names that should be excluded that's probably worth noting. But most likely dockge (which is in maintenance mode per the developer) is not a better fit for your use case than portainer or komodo.

[Dracrius]

@diaodeng it's the way docker compose works that the project name is inferred from the folder name which leaves the possibility of conflicts. In these cases it is up to you to specify project names to avoid conflicts

@Dracrius that's a bit surprising, it does not take me 40 seconds to recursively list all files from the root directory of my server. But as I put in the description I split the PR up to account for this, you can just take commit fc4ad7f to get the list from docker compose without the search feature. If you are using it on a dev machine or something and there are obvious names that should be excluded that's probably worth noting. But most likely dockge (which is in maintenance mode per the developer) is not a better fit for your use case than portainer or komodo.

I'll take a look and do some test when I get home from work on the separate parts. It seems I was a bit tired cause you are still using async. It would have been a pretty obvious cause and effect haha but it must be something else. I'd be curious to see what differences we have in setup but it could be as simple as you using a much more recent processor on your server then mine.

Default dockge is so snappy everything loads up the second I refresh the page so it was really obvious cause without exaggeration I timed it to 40 seconds cause at first I thought it out right broke dockge but while I was making some food I'd noticed it would load eventually.

[diaodeng]

@diaodeng it's the way docker compose works that the project name is inferred from the folder name which leaves the possibility of conflicts. In these cases it is up to you to specify project names to avoid conflicts

@Dracrius that's a bit surprising, it does not take me 40 seconds to recursively list all files from the root directory of my server. But as I put in the description I split the PR up to account for this, you can just take commit fc4ad7f to get the list from docker compose without the search feature. If you are using it on a dev machine or something and there are obvious names that should be excluded that's probably worth noting. But most likely dockge (which is in maintenance mode per the developer) is not a better fit for your use case than portainer or komodo.

我的问题，不是名称冲突，是我之前配置错stack目录启动的容器，导致目录跟现在不一致

[diaodeng]

关于查找文件很慢（文件太多，递归深度太深），我增加了一个环境变量来控制递归深度，默认是2；
同时合并了这个项目大部分的pull请求；
修复了部分问题，修改了一些样式来适应小屏，不过可能不是大家都喜欢我调整的样式；
终端、compose编辑器增加了全屏按钮。
我是一只菜鸟，可能会导致其他问题；
https://github.com/diaodeng/dockge/tree/panda62
同时我也推了一个docker镜像：https://hub.docker.com/repository/docker/panda62/dockge/general