Add systemd startup order dependencies #531
Description
Problem
During the https://github.com/StackStorm/ova/pull/68 work, we've found it's a common case when services are started during machine boot time in not the best order, eg. st2api is up and accepting connections, when RabbitMQ didn't start yet, resulting in errors and lost requests.
NB! This will affect only service startup order (which are enabled) during the machine boot stage. Same for shutdown.
Solution
In systemd, there is a rich mechanism to control service startup order via Wants, After, Requires.
https://wiki.archlinux.org/index.php/DeveloperWiki:Systemd says:
The most typical case is that A requires the service B to be running before A is started. In that case add Requires=B and After=B to A.
If the dependency is optional then add Wants=B and After=B instead
https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Wants= says:
A weaker version of Requires=. Units listed in this option will be started if the configuring unit is. However, if the listed units fail to start or cannot be added to the transaction, this has no impact on the validity of the transaction as a whole. This is the recommended way to hook start-up of one unit to the start-up of another unit.
"Relaxed" unit dependency sounds exactly what's needed, because services like RabbitMQ/MongoDB could be deployed on external systems, HA-friendly, while nginx could be not present locally at all.
This should be well tested in both single-machine, as well as HA-emulated deployment to avoid any kind of regressions and corner cases. If regression is unavoidable, - skip this effort.
As a result it will improve st2 service reliability for a single-machine deployments.