-
-
Notifications
You must be signed in to change notification settings - Fork 109
Description
To improve observability and provide better ways to service autoscaling, Granian should collect some runtime metrics and expose them through well known APIs (prometheus/openmetrics at least).
Ideally the following metrics should be available:
- no of requests processed (per worker)
- python call latency (bucket per worker)
- no of active threads in the blocking pool (per worker)
- HTTP response codes count (per worker)?
- blocking threads idle/wait time bucket (per worker)?
- backlog/backpressure/active tasks in Rust runtime (per worker)?
Best option in terms of impl seems to be using the metrics
crate.
For this to work, some form of IPC is needed between the main process – that will expose those metrics in std formats like prometheus – and the workers – that will actually store the metrics. Thus, the IPC impl should allow the master process to collect and store metrics from all the workers. Given IPC is generally easier on Unix, we might want to restrict this feature to non-windows envs.
This was originally requested in #275, and also emerged in discussions like #406