A personal platform for R programming.
"Sea Of Thieves" by @lacerenza_design
This is a platform for R programmers, powered by traefik and docker, and designed to work on a Raspberry Pi cluster. With this platform, you can deploy shiny apps, plumber APIs, monitoring tools and more.
Primary motivation is to load balance shiny applications with traefik to support concurrent usage. Some other things have been added simply because it's fun to self host (e.g. cran mirror) or could be useful for shiny applications (e.g. plumber api).
Instructions are provided in each folder, start with traefik and then deploy any of the other services you're interested in.
Note that this platform uses bind mounts to communicate files between the host and container. This means that set up typically involves:
- Creating a diretory on your host e.g.
mkdir shiny-dir - Creating an environment variable prior to deploying e.g.
export SHINY_DIR=shiny-dir - Deploying the stack
docker stack deploy -c shiny.yml shiny
- Support concurrent shiny sessions with traefik load balancer
- Develop at
rstudio.example.comand deploy atapps.example.com/appname - Collect server load metrics at
prometheus.example.com - Visualize server load metrics at
grafana.example.com - Alert excessive server load with
alertmanager.example.com - Deploy a postgres database at
db.example.com - Deploy a homepage at
example.com - Deploy a blog at
blog.example.com - Deploy a plumber api at
api.example.com - Host a CRAN mirror at
cran.example.com - Mirror your git repositories at
git.example.com - Host a S3 like storage system at
minio.example.com
As a demonstration, here are a couple of these features in action, deployed from my Raspberry Pi cluster:
- https://api.tylerlittlefield.com/__docs__/
- https://cran.tylerlittlefield.com
- https://tylerlittlefield.com
- https://blog.tylerlittlefield.com
- https://apps.tylerlittlefield.com/hello
- https://git.tylerlittlefield.com
Below are some details on the hardware/software this platform is currently running on:
- Raspberry Pi 4 Model B 8GB (3 total, 1 main, 2 workers)
- Ubuntu 20.10 'Groovy Gorilla'
- USB boot (as opposed to booting from a microSD card)
- Everything powered over ethernet with the official PoE hats
- 1 network switch with PoE
- A 3TB WD Passport (external hard drive) which is plugged into one of the workers
You can find some rough notes on my set up here: https://github.com/tyluRp/rpi-cluster
The network stuff, like making my cluster available to the internet, is out of scope for this repo. I followed the following guide to get started: https://engineerworkshop.com/blog/connecting-your-raspberry-pi-web-server-to-the-internet/
If you are interested in the platform and want to host it on something like AWS, Digital Ocean, or Azure, this is totally possible (and most likely easier).
tiangolo/dockerswarm.rocksmakes up most of this project. Many of the.ymlfiles are based on thetraefik.ymlfile found in this repo.stefanprodan/swarmprommakes up most of the monitoring stack with minor changes to work on ARM CPUs.- @lacerenza_design for the README artwork.