Scaling websocket link servers in production deployment

[lahwaacz]

Hi,

I've managed to deploy a Trame application on a Kubernetes cluster by simply converting the single Dockerfile to Kubernetes resources. However, this is not scalable - I've noticed that Trame runs a separate Python process per websocket link, which is quite heavy on memory resources (about 500 MiB on fresh start for our application). Regardless of the memory limits I set, clients can easily bring the deployment down by simply refreshing the page fast enough (websocket link servers seem to have quite long life even when unused). When the pod gets out of memory, Kubernetes restarts it and this affects all clients.

Do you have some guidelines for production deployments on Kubernetes (or another system)? Or in general, is it possible to aggregate websocket links to a handful of worker processes so the memory consumption cannot grow above all limits?

[jourdain]

So technically if you deploy trame in kubernete, it should be a setup similar to JupyterHub where you get a dedicated pod per user.

Using the default docker image lead indeed to 1 process per user within that same pod. And if your trame app is based on ParaView, that ~300MB of memory is about right. Depending on your app design, you could collocate several apps into the same process, but at that point you are slicing 1 CPU across many users for all the network, data processing and rendering...

So no easy answer unfortunately, but without knowing more about your app and your infrastructure it won't be easy to help. Also it might get into details that can go beyond general community support. If you need professional support we can help.

[lahwaacz]

So technically if you deploy trame in kubernete, it should be a setup similar to JupyterHub where you get a dedicated pod per user.

The behavior I'm observing is (currently) very different - on JupyterHub user = session as usual, but on Trame this would be something like user = connection. If a JupyterHub user reloads their browser tab, it refreshes the page for the same user server, whereas reloading our Trame app starts a new session and new websocket server.

Note that I'm a sysadmin and not a developer in this context, so I hope that I've correctly attributed the behavior to Trame in general. If it's application-specific defect, I'd apologize! The application in question is mri-viewer developed by @karelvrabeckv (you might remember supporting us before).

[jourdain]

I see, what I meant with JupyterHub for Kubernete and trame, is that the kubernete setup should be similar where you have an app (JupyterHub) that aim to dynamically allocate a job (JupyterLab) and let your redirect to it. The trame app in question is the same thing as JupyterLab. Here you are missing the Hub part that start and manage jobs. This is not the responsibility of trame. It could be of a TrameHub (does not exist) or something like JupyterHub repurposed for trame.

The current docker image do the job management within a single image by starting a process per user and dealing with network routing internally. If you adopt a job approach with kubernete, then the docker should just be trame without any extra (no launcher, no apache).

Application wise, something can be done to prevent starting a new process from the same client and just reconnect to the process if already available/started.

HTH