Best way for collaboration network and deployment

[aiomayo]

Hi everyone,

I'm currently working on building a large, scalable Minecraft server network, and I'm looking for the best practices when it comes to deployment, scaling, and collaboration.

My Plan So Far

Infrastructure: Using Rancher and Kubernetes for orchestration and scaling.

Proxy: Using Gate as the Minecraft proxy.

Configuration Management: Using GitHub to store plugin configurations, world data, etc.

Automation: The idea is that when changes are pushed to GitHub, Rancher/Kubernetes should automatically deploy or update the relevant server containers.

Key Goals

Developer friendly workflow: Team members should be able to work on configurations locally (like plugins or world data), push their changes to GitHub, and have it deployed automatically.

Git-based deployments: Similar to a GitOps approach, changes in Git trigger deployments.

Support both dynamic and static servers:

• Dynamic servers are ephemeral (e.g., game servers that spin up/down based on demand like eg bedwars).

• Static servers are persistent (e.g., survival servers where world state matters).

Separation of concerns: Use templates to separate the Minecraft server jars and base environment (for example content that gets generated by the paper jar by itself and doesn't need changes) from custom content like plugins or world folders.

Inspiration: SimpleCloud.app or CloudNet

I've looked at tools like SimpleCloud, which have a neat template based system. You can:

Create dynamic templates with only the plugins and config you want to modify.

Let the base server jar and other essentials be handled automatically.

Dynamically spin up or shut down instances easily.

Persist data only when needed (e.g., static servers).

This is the kind of flexibility I want to bring into a containerized, Git-based deployment system.

What I'm Looking For

I'd love advice or recommendations on:

Tools or setups that match this kind of workflow.

Ways to structure GitHub repos (one per server? templates? environments?).

Best practices for managing dynamic vs static servers in Kubernetes.

How to inject updated plugin configs into a running Minecraft container.

Examples of using initContainers, sidecars, or other methods to pull plugin configs.

Any pitfalls to avoid when scaling Minecraft networks in Kubernetes.

Chatgpt Summary of the text:

To sum it up:

I'm using Rancher + Kubernetes for orchestration.

I'd like a GitHub-based workflow for plugin and server config management.

I want to scale Minecraft servers dynamically while keeping some persistent.

Ideally, deployments should be automated from Git.

Looking to build something similar to SimpleCloud, but containerized and Git-driven.

Thanks in advance for any insights, tools, or examples you can share, even if it's not any of my plans. I just need a better way to scale and contribute more professionally. Thx for taking the time to read all that and thx for any support

[jeremylvln]

Hi @aiomayo. Thanks a lot for opening this discussion!

There are a lot of questions, I will try to give you some advices:

• There are tools that exists to sync the state of a Kubernetes cluster from files contained in Git repositories. I personally love ArgoCD for this. And it's actually how I manage my own Shulker cluster in my infrastructure. Whenever I push updated configuration files to my GitHub repository, they are automatically synced into my Kubernetes cluster. This will allow you to use GitOps only to fill and maintain up to date your Kubernetes cluster

• About updating your Minecraft configuration files in a running container, it's a bit more complex. Currently, Shulker is aimed to work with completely stateless workload: Shulker is responsible for starting your servers with the configuration you've written in Shulker's manifests, but when they are running, you are on your own. I can think of a way to facilitate the mounting of ConfigMap volumes into the containers. By updating the ConfigMaps, their mounted files would be automatically updated. But this also means that you'll still have to listen to these files in your plugins, which can be a bit tedious. TL;DR for this, you'll have to create your own system for now. But we can think of an in-between solution where Shulker could help you simplify this

• You can deploy persistent servers using Shulker's MinecraftServer manifests. The main difference with MinecraftServerFleet is that a fleet is stateless and can be scaled whereas the first one is meant to be static. Note that Shulker's volume management is not that pretty and would need some work to be fully ready

If you can explain me some specific use-cases Shulker can not cover properly, I could open feature requests to fill these blanks.

Thanks!