Simpler - concrete - getting started instructions.

[tomandersen]

I wanted to test an instance out, but what to do?

It wasn't too bad,

But I think you will get more 'sales' if you have a follow these steps kind of thing.

1. DigitalOcean droplet with docker
2. git fetch repo, cd into repo, (Is it easier to use docker pull?)
3. nano .env
4. add AUTH_TOKENS=1234xxxxx // PICK something
5. run
6. set up sub domain
7. add load balancer to handle https certificate or maybe there is an easier/cheaper way to do this with the dockerfile?

(the above instructions are wrong...)

Also I have questions that are not answered but are likely obvious to you:

1. If I start on a 1GB machine, will redis expand as needed to use memory, and how many jobs can I have in flight if my jobs average 500 bytes in size? (I see the config options for holding onto completed jobs). I can figure out the redis memory footprint (kinda) but what is the overhead of the app, etc? Some rough guidelines might be nice.

2. Is the redis set to crash or toss jobs if memory runs out?

etc.

I know this isn't the smallest ask. We are running about 10 instances of the old Bull with a custom express app, and it looks like BullMQ-Proxy is all we need.

Thanks,

--T

[manast]

I think that as more users start using the proxy we can provide higher level tutorials. As there are so many different cloud providers out there, it is difficult to pick one for an example. In general I think devops should not have big problems in getting the proxy working, some knowledge about docker, https certificates, Redis hosting and so on is expected on the person managing the proxy.

Regarding your questions, Redis does not autoscale memory in general, only Upstash can do this but it has other tradeoffs like not being as fast as native Redis for example. If Redis runs out of memory you will not be able to add more jobs to the queue, and most likely not even process any jobs. Normally your Redis hosting solution will send you an alert when passing over 80%. In general you set up options like "removeOnComplete" https://api.docs.bullmq.io/interfaces/v5.BaseJobOptions.html#removeOnComplete to make sure the memory stays within some sane limits.