Skip to content

Conversation

benscobie
Copy link
Collaborator

@benscobie benscobie commented Feb 9, 2025

This PR adds artifact creation to the release workflow, providing downloadable archives alongside the GitHub release, which can be extracted, configured and ran without having to mess around with building it yourself. This also provides an avenue for non-Docker installation methods such as community-scripts/ProxmoxVE#97 . I've included an install script, basic run scripts and config files, which serve as a base for other implementations that want to use the release archives.

A basic install guide would look like:

  1. Download the appropriate archive from the GitHub release page
  2. Extract the archive to the final installation directory
  3. User creates a directory to hold Maintainerr data
  4. Run install.ps1 / install.sh
  5. Update ./server/.env and ./ui/.env
  6. Use start_ui.ps1 / start_ui.sh and start_server.ps1 / start_server.sh to start the UI and server.

An upgrade would currently involve:

  1. Make a copy of server/.env and ui/.env
  2. Delete the installation directory (the data dir should be moved out, or ideally not be in here at all)
  3. Follow the install guide from step 2. Use your backed up .env files instead of updating the existing ones.

Workflow

I have added a new step to the release workflow that builds the project in the same way as we do in Docker. This will output an archive for each platform (currently Linux amd64, Linux arm64 and Windows x64), which will then be attached to the release created in GitHub by semantic-release.

I have updated how semantic-release is called as well. First we do a dry-run to get the version number, which also validates that the actual release will go okay (most of the time). The Docker & release archives steps will then fire off in parallel, and once those are complete, semantic-release is is called again to perform the GitHub release.

Finally, I managed to reduce the Docker image size from 650MB to around 350MB by removing node dev dependencies after the build. This saving also applies to the new release artifacts.

Configuration changes

In order to make configuration easier for users, I have updated the UI & server to make use of .env files. There are a few different variations in-use here:

  • .env - This can be seen as our 'default' configuration options. The %GIT_SHA% token is replaced at build time. The .env files below will override configuration defined in here.
  • .env.distribution - This file is included in the release archive and is the file that the user modifies to change configuration.
  • .env.docker - This file contains the hard-coded values that were in the Dockerfile. I have removed them from there because of this issue with Portainer. There is no change for Docker users here. They can continue to specify configuration via environment variables, which have a higher precedence than .env files.
  • .env.local - This file is not included in any release artifacts. It is used to specify configuration for the local development environment.

As part of the build process, .env.distribution / .env.docker, is copied to .env. The .env file is copied to .env.production.

The final output is:
/ui/.env - this was .env.distribution or .env.docker
/ui/.env.production - this was .env
/server/.env - this was .env.distribution or .env.docker
/server/.env.production - this was .env

Code changes

  • I have updated the about page to get the new DATA_DIR configuration value
  • Removed any fallback values when referencing undefined process.env props. These defaults are now centrally specified in the .env files.

Testing

  • Windows x64 - Tested on Windows 11
  • Linux x64 - Tested on Debian 11
  • Linux arm64 - Tested on an emulated arm64 VM

Future

I would like to improve the install, upgrade and run/stop processes in a future release. A few ideas I've had:

  • Provide docs/scripts for integration with systemd, Windows Services, etc.
  • Create an install script that can be curl'd which grabs the latest release and installs it. Inspired by Sonarr here.
  • Create a Windows installer which will handle installing & upgrading, as well creating a Windows Service to run the apps.
  • I would love to have just a single process to run. I originally wanted to ship with pm2 but I couldn't get it to work. We might be able to create a custom server that inits both the UI & server.
  • Provide docs for platforms like Synology

@benscobie benscobie force-pushed the release-workflow branch 4 times, most recently from 20134b8 to 90efc7a Compare February 13, 2025 16:22
@benscobie benscobie changed the title Create release artifacts chore: Create release artifacts Feb 13, 2025
@benscobie benscobie force-pushed the release-workflow branch 5 times, most recently from 896393c to 3770854 Compare April 13, 2025 00:32
@benscobie benscobie marked this pull request as ready for review April 13, 2025 00:34
@jorenn92
Copy link
Owner

@benscobie, I (finally) had some time to take a look at this. It's a big change, but it looks solid!
I'd like to take the time to try building locally as well though, before approving this.

@benscobie
Copy link
Collaborator Author

benscobie commented Apr 15, 2025

I'd like to take the time to try building locally as well though, before approving this.

Of course!

The packaging is all located within these scripts to make it easier to test locally, you just need to pass a Git SHA as an arg (can be anything really though):
./distribution/prepare_artifact_linux.sh gitSHA
./distribution/prepare_artifact_win.ps1 gitSHA

What I was doing locally was:

  • yarn build
  • yarn workspaces focus --all --production
  • ./distribution/prepare_artifact_win.ps1 c74416fa702d6268fde3a77ed9a5df1306d513a1
  • Take the output from the _output folder and put it somewhere like C:\Maintainerr\
  • Configure the .env files
  • Run install.ps1/sh
  • Run start_server.ps1/sh & start_ui.ps1/sh

You can also execute the release workflow to generate the artifacts, just target this branch (though it is in my fork so likely won't be listed..). Just enable Dry run and uncheck Build and push Docker images. They'll be available alongside the action, e.g. https://github.com/benscobie/Maintainerr/actions/runs/14433809972

@jorenn92
Copy link
Owner

I'd like to take the time to try building locally as well though, before approving this.

Of course!

The packaging is all located within these scripts to make it easier to test locally, you just need to pass a Git SHA as an arg (can be anything really though): ./distribution/prepare_artifact_linux.sh gitSHA ./distribution/prepare_artifact_win.ps1 gitSHA

What I was doing locally was:

* `yarn build`

* `yarn workspaces focus --all --production`

* `./distribution/prepare_artifact_win.ps1 c74416fa702d6268fde3a77ed9a5df1306d513a1`

* Take the output from the `_output` folder and put it somewhere like `C:\Maintainerr\`

* Configure the `.env` files

* Run install.ps1/sh

* Run start_server.ps1/sh & start_ui.ps1/sh

You can also execute the release workflow to generate the artifacts, just target this branch (though it is in my fork so likely won't be listed..). Just enable Dry run and uncheck Build and push Docker images. They'll be available alongside the action, e.g. https://github.com/benscobie/Maintainerr/actions/runs/14433809972

Very nice, thank you for the clarification!

@p1carats
Copy link
Contributor

p1carats commented May 28, 2025

I've tried to compile and use Maintainerr outside of Docker based on this PR. It works perfectly using base configuration, but I came through several issues when setting a basePath :

  • basePath support is broken outside of Docker as the build script on GitHub Actions misses both the NEXT_PUBLIC_BASE_PATH env var (required on compilation) and basePath:'' replacement before compile
  • even when I manually fix the previous issue, there is https://<domain.tld>/api/events/stream 404 (Not Found) spamming the console. I've looked a bit through commits and it seems to be an error related to absolute paths when targeting the api on some calls?

I've done quick dirty fixes for the 1st part (build) on my local repo to test, and I successfully managed to build a static build. I'll try to make a clean commit/PR in the coming weeks to fix the first point if you want, but I don't currently have time to look into the second error. Just wanted to report this 😄

@MickLesk
Copy link

We are eagerly waiting... 😄

@benscobie
Copy link
Collaborator Author

* basePath support is broken outside of Docker as the build script on GitHub Actions misses both the NEXT_PUBLIC_BASE_PATH env var (required on compilation) and basePath:'' replacement before compile

I must've broken this at some point along the way as I'm sure I tested this 🤔 But yes I believe we just want to copy the same commands from the Dockerfile out (with the .env.distribution file instead I think). I must admit, coming back to this I found it a bit difficult to figure out what was all going on so congrats for getting that far!

* even when I manually fix the previous issue, there is `https://<domain.tld>/api/events/stream 404 (Not Found)` spamming the console. I've looked a bit through commits and it seems to be an error related to absolute paths when targeting the api on some calls?

We've had a few people with this issue on the Docker install (do you, if you have it installed?) so I think it's unrelated. We needed some more info from them to help track it down but never heard anything back.

I've done quick dirty fixes for the 1st part (build) on my local repo to test, and I successfully managed to build a static build. I'll try to make a clean commit/PR in the coming weeks to fix the first point if you want, but I don't currently have time to look into the second error. Just wanted to report this 😄

Sure, sounds good! This PR is slow moving anyway, jorenn is quite busy with real life stuff at the moment.

@p1carats
Copy link
Contributor

Quick follow-up on my report from yesterday — I’ve opened a PR to address all basePath-related issues I encountered across the codebase: #1806.

This includes fixes for the /api/events/stream 404 errors we discussed, along with other missing or broken paths due to improper handling of the basePath.

Once this PR is merged, I’ll let @benscobie handle merging upstream into this branch. From there, I’ll move on to the original issue I was aiming to solve: adding proper basePath support for standalone releases 😃

@benscobie
Copy link
Collaborator Author

benscobie commented May 29, 2025

Quick follow-up on my report from yesterday — I’ve opened a PR to address all basePath-related issues I encountered across the codebase: #1806.

This includes fixes for the /api/events/stream 404 errors we discussed, along with other missing or broken paths due to improper handling of the basePath.

Once this PR is merged, I’ll let @benscobie handle merging upstream into this branch. From there, I’ll move on to the original issue I was aiming to solve: adding proper basePath support for standalone releases 😃

Thank you for fixing those issues ❤️ I've just merged the latest changes in. I haven't tested the workflow out again yet, but the merge conflicts were minimal so everything should still be okay. (EDIT: That was not the case, looking now...) FIXED

@p1carats
Copy link
Contributor

p1carats commented Jun 2, 2025

I’ve been thinking about this PR, and I believe it would be best to split it into two separate ones:

  • First PR: Focus solely on the .env behavior changes and the associated code updates (e.g. data dir config, removal of fallback process.env, etc.).
  • Second PR: Concentrate on alternative build methods, artifact creation, and distribution logic.

This split would make review and merge management much easier, as this PR is getting quite large. It would also help isolate any side effects introduced by the config/code changes, since the distribution method remains unchanged—making troubleshooting simpler if issues arise.

Additionally, most merge conflicts during new releases tend to come from core code changes, so merging those sooner would reduce friction down the line.

The second PR could then be more polished and focused on delivering a robust alternative install experience when everyone got time to dedicate to the project, with room to explore nice extras like systemd/Windows Service integration as you mentioned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants