Skip to content

Docker: improve docker compose structure #722

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

LukeVader-IV
Copy link

update so that the docker compose file specifies the env vars for each container.
the benefits:

  • less env vars to be set by the end user
  • containers are only given vars relevant to them
  • when running docker compose up, only containers affected by changed vars are restarted

additionally, rename .env.example to example.env so that it is not a hidden file.

Previously, each container would be handed all the env vars int the .env file.
This is bad practice for a variety of reasons.
This commit changes it so the docker compose file specified the env file for each container.
the benefits:
- less env vars to be set by the end user
- containers are only given vars relevant to them
- when running docker compose up, only containers affected by changed vars are restarted
files starting with a `.` chatacter are "hidden files" in UNIX systems.
some less knowledgable users may get confused by not seeing a file they expect.
changing the example env file to not be a hidden file, making it easier to discover for a user
Copy link

vercel bot commented Jul 9, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
adventurelog ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 14, 2025 8:48pm

note: unsure about the "required" field, as any required vars are now defined in the docker compose file
@LukeVader-IV
Copy link
Author

I would like to note that this comes with the benefit of reducing the amount of people asking for help, as potential mistakes are reduced.
some changes that are now no longer set through the .env file can still be set through the docker compose file, for users still willing to change these things.

The only issue is what to do with the "required" field. Should it be left, changed, or removed completely?
note that technically the application will work on localhost without the .env file even existing.

@LukeVader-IV LukeVader-IV marked this pull request as draft July 11, 2025 07:44
Note: this currently fails at compile time
@UndyingSoul
Copy link
Contributor

Thoughts on #693 ?
Does my PR cover the proposed changes in this PR? Any comments are appreciated.

@LukeVader-IV
Copy link
Author

@UndyingSoul I am not 100% sure as I have never seen such a docker compose file.
But as far as i can tell, no.
My aims with this MR, and how they compare to the current state of #693 :

  • reduce amount of variables to be set, e.g. remove PUBLIC_SERVER_URL from the env file (only in docker-compose), as people might try to change this and only cause problems. or how ORIGIN == FRONTEND_URL
    • The commit does not reduce the amount of vars expected to be set by the end user.
    • It does not hide vars that should generally not be set by the user.
  • Give each container only the env vars they require.
    • The commit does seem to do this, but I have never seen this x-env method. I am confident that this will cause confusion, and therefore easily avoidable issues.
  • Secrets for vars that should be handled with more care. I was still working on this. ( [REQUEST] Support _FILE env variables #717 )
    • It does not change how sensitive data is passed to containers

I think that (In terms of docker) the commit introduces additional complexity, reducing only one issue, and probably introducing a bunch of confusion. It looks like it is deprecating the .env file, and I don't understand why.

@UndyingSoul
Copy link
Contributor

UndyingSoul commented Jul 24, 2025

You bring up some excellent points. Thanks for the feedback.

My PR does not address your first point in any way. My personal preference is to expose everything that can be configured to the user. While I agree that having the frontend/backend URLs exposed to the user can easily lead to problems, I do think there is value in having as much documentation in the docker-compose.yml, which includes everything that can be configured.

The second point you raised is a great one, yml anchor and alias notation can be kind of difficult to understand if you're not used to seeing it. The reason I choose anchors and aliases over just putting the environment variables directly under each service is so I could reuse the common environment variables between the backend and DB. I also think it's great idea to have all the main configuration in one spot, similar to an .env file.

Which brings me to your last point, I took a closer look at this PR and it looks like we're doing very similar things, but our implementation is different. I chose to implement the secret file handling in the entrypoint.sh, which, in my opinion, it was the minimally invasive option. Which is better? I'm not sure.

Then, I wanted to address the depreciation of the .env file. The way docker handles .env files is very confusing, and it's tough to find the source of truth. With both of our implementations, any .env file in the same directory as the docker-compose.yml, will override any overlapping environment variables defined in the docker-compose.yml file. In a similar way, if you ENV_VAR=value docker compose up, the local environment also overrides what's defined in the docker-compose.yml. you can also load an env file with the 'env_file:` directive in the compose file, and can be a list of files loaded in the top-down order. When I created my PR, I wanted to minimize breaking changes for anyone with an existing .env file, I've tested this with docker and podman and have had the same results.

Thanks again for bringing up those important topics, if you think I should document this more, whether it in the AdventureLog docs or on either of our PRs, let me know.

Edit: Fixed some grammatical/spelling errors.

@LukeVader-IV
Copy link
Author

Note: as we are discussing only docker-compose and environment vars from here on out, reference changes up to and including 537790c only. I will do a revert of the last commit when I have a moment on my private system.

Right, I see what you're saying, but I don't agree.

My personal preference is to expose everything that can be configured to the user.

Let's take the .env file as a given. It allows the user to define a subset of vars that are necessary to be defined by the user. Vars that generally should not be touched ( PUBLIC_SERVER_URL ) can still be modified by the user by editing the docker-compose.yml
This indicates to the user that this is probably not a variable to modify unless they know what they're doing.
When I say "reduce amount of variables to be set", It does not mean that less options are exposed to the user.

I am a big fan of the GNOME HIG Design Principles. While this is mainly focused on application design, It can be applied in other places too. Here I would like to point out two points under the "Reduce User Effort" section:

  • If something can be done automatically, do it automatically.
    ORIGIN and FRONTEND_URL should be equal? do that.
    CSRF_TRUSTED_ORIGINS is just the frontend + backend url? do that.
    There should be no requirement for the user to enter duplicate information, it can be automated.
  • Try to minimize the number of steps required to perform a task.
    Reducing the amount of variables a user needs to understand to be able to deploy the app (see previous)

I could similarly quote KDE's motto, encoded into the KDE HIG. "Simple by default, powerful when needed". With this commit, I am removing complexity by default, while still allowing more complex changes to be made.

Again, I would like to point out that nothing is stopping a user to make a complete mess by changing the behavior in the docker compose.

yml anchor and alias notation can be kind of difficult to understand if you're not used to seeing it. The reason I choose anchors and aliases over just putting the environment variables directly under each service is so I could reuse the common environment variables between the backend and DB.

If you look at my docker-compose.yml + .env, I handle exactly this. One variable is set in the .env which gets passed to the relevant containers.

I also think it's great idea to have all the main configuration in one spot, similar to an .env file.

I don't think I need to add anything to this, it speaks for itself.

I took a closer look at this PR and it looks like we're doing very similar things, but our implementation is different. I chose to implement the secret file handling in the entrypoint.sh, which, in my opinion, it was the minimally invasive option. Which is better? I'm not sure.

All right, i have no issue with this. When I made my first comment, I had no time originally to look if you handled file secrets. If your implementation is working, that's enough. 👍🏻

The way docker handles .env files is very confusing, and it's tough to find the source of truth.

What? I do not agree at all. Neither on confusing or tough to find the source of truth.

My original issue with the project's docker-compose, is that it simply passed everything inside the .env file as arguments to all containers.

I am no stranger to docker compose, and every single project uses either a .env file in the same way as I did in this commit, or regular env vars in the docker compose . It is unlikely that this project will be anyone's first interaction with docker compose and environment vars.
Doing it in a different way, in my opinion, is a really bad mistake. This would be equivalent to a small Linux distro deciding to swap the left mouse button and the right. It's wrong, not because it inherently is, but because it is doesn't respect how people actually use a system for no tangible benefit.

When I created my PR, I wanted to minimize breaking changes for anyone with an existing .env file, I've tested this with docker and podman and have had the same results.

This remains true with the changes I proposed. An existing deployment would remain entirely unaffected.

@UndyingSoul
Copy link
Contributor

I realized I had some incorrect assumptions about how Docker Compose handles the .env file and variable evaluation in the docker-compose.yml file. I ran a quick experiment and found an interesting behavior that affects both of our approaches.

Test Setup

docker-compose.yml:

services:
  test:
    image: docker.io/ubuntu:latest
    environment:
      SOME_USER: ${SOME_USER:-dockercomposeuser}
      SOME_PASSWORD: dockercomposepassword
      SOME_EMAIL: ${SOME_EMAIL}
      SOME_USER_COMBO: ${SOME_USER}:${SOME_PASSWORD}
      SOME_SOLUTION: ${SOME_SOLUTION:-dockercomposesolution}
    entrypoint: ["bash", "-c", "printenv"]

.env:

SOME_USER=dockerenvuser
SOME_PASSWORD=dockerenvpassword
SOME_EMAIL=envemail

Expected Output

One might expect printenv to output:

SOME_USER=dockerenvuser
SOME_PASSWORD=dockerenvpassword
SOME_EMAIL=envemail
SOME_USER_COMBO=dockerenvuser:dockerenvpassword
SOME_SOLUTION=dockercomposesolution

Actual Output

Using Podman Compose:

$ podman compose up
SOME_USER=dockerenvuser
SOME_PASSWORD=dockercomposepassword
SOME_EMAIL=envemail
SOME_USER_COMBO=dockerenvuser:dockerenvpassword
SOME_SOLUTION=${SOME_SOLUTION:-dockercomposesolution}

Using Docker Compose:

$ docker compose up
SOME_USER=dockerenvuser
SOME_PASSWORD=dockercomposepassword
SOME_EMAIL=envemail
SOME_USER_COMBO=dockerenvuser:dockerenvpassword
SOME_SOLUTION=dockercomposesolution

Discussion

  • SOME_USER: Works as expected—.env overrides the default value.
  • SOME_PASSWORD: Contrary to my earlier comments, this doesn't use the .env value, since it’s hardcoded in the YAML file.
  • SOME_EMAIL: Correctly pulls from .env, as expected, though there's no fallback.
  • SOME_USER_COMBO: Evaluates as expected, and behaves similarly to a CSRF-style variable interpolation.
  • SOME_SOLUTION: Here’s the odd part—Podman Compose doesn’t evaluate the default value properly and instead uses the literal string ${SOME_SOLUTION:-dockercomposesolution}. Docker Compose, however, evaluates it as expected.

Additional Notes

After some digging, I found this issue is specific to older versions of Podman Compose. I was using v1.3.0, which has a known bug related to variable substitution containers/podman-compose#1105. After updating to v1.5.0, it now behaves as your PR intends.


Overall, I support the approach you've taken in your PR—default values in docker-compose.yml strike a good balance between flexibility and maintainability. I also appreciate the use of YAML anchors and aliases for reuse (e.g., with Postgres credentials), which can help reduce duplication.

Curious to hear what others think too—it’s just the two of us so far.

@LukeVader-IV
Copy link
Author

LukeVader-IV commented Jul 26, 2025

Firstly, I want to apologise if I sounded defensive or otherwise unpleasant during this interaction, that is in no way my intention. I appreciate that you are working on this project, and I admire regular FOSS contributors. I find that when communicating over text, everything sounds more assertive or aggressive.

However, I truly believe the particular issue of our discussion is an open and shut case.

I want to go over one more example to hopefully clarify my stance.
I ask you to take the perspective of an inexperienced user, barely knows what docker is.
This is what I believe the experience would be from their perspective:

  • option A, the x-env method:
    • Edit the docker-compose
    • Large document, may confusing if not familiar with yaml notation
    • One badly placed character can cause confusing errors (or worse: not)
  • option B, the .env file method
    • Edit the .env file
    • Small document
    • Clearly described options
    • Fill in the blank
    • Missing vars will be set to default, with a small warning that it is not set.
    • More likely a user has encountered it before.

I think that this perspective is fair, as there is an install script in case you can't run docker compose up -d.
Not to mention the idea that "accessibility benefits everyone". This holds true whether it's level boarding on a tram line, an elevator in a building, or clarity and simplicity in software.
Design for accessibility, and you design for everyone.

@LukeVader-IV
Copy link
Author

About the podman issue. These are two similar yet very distinct projects. I definetly support people making sure tools can be used by either docker or podman.
However, the goal of the docker to podman translation that you used in your demonstration, is to let docker things work on podman. Any deviation between docker's behaviour and the translation layer is an issue with podman's translation layer. differences between the two should never be used as justification for any decision.
The same goes with distro-packaged versions of docker. I have no issue with this, but any issue only encountered in a distro-packaged version of docker is not an issue of docker, and shouldn't be taken as expected behaviour.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants