Add Promenade::PeriodicStats for Periodical Stats Instrumentation

[robertomiranda]

Promenade::PeriodicStats

This PR introduces the use of Promenade::PeriodicStats to configure and start an infinite loop that periodically instruments statistical data. The configuration sets the frequency at which the stats are collected and logged, specifically for Promenade::Pitchfork::Stats and Promenade::Raindrops::Stats.

Depends on #62

The Promenade::PeriodicStats is configured with a specified frequency (in seconds) and a logger (Rails.logger).

Promenade::PeriodicStats.configure(frequency: 1, logger: Rails.logger) { # periodic instrumentation goes here }

This PR introduces instrumentation for preforking Rack servers like Unicorn and Pitchfork, and also introduces metrics for Pitchfork leveraging Pitchfork::Info.

Promenade::Raindrops::Stats

The Promenade::Raindrops::Stats class collects and reports metrics on active workers and queued requests in servers like Rainbows, Unicorn, and Pitchfork using Prometheus gauges. It initializes by obtaining TCP listener stats from Raindrops::Linux, setting the listener address, and fetching active worker and queued request counts. The instrument method updates the Prometheus gauges with these values, and the class method self.instrument allows easy instrumentation with an optional listener address.

Promenade::PeriodicStats.configure(frequency: 1, logger: Rails.logger)  do
  tcp_listener_names.each do |name|
    Promenade::Raindrops::Stats.instrument(listener_address: name)
  end
end

• rack_active_workers: Number of active workers in the Application Server
• rack_queued_requests: Number of requests waiting to be processed by the Application Server

Promenade::Pitchfork::Stats

The Promenade::Pitchfork::Stats class collects and reports metrics on Pitchfork worker processes using Prometheus gauges. It tracks the total, live, idle, and busy worker counts. The class retrieves these metrics using Pitchfork::Info and Raindrops::Stats, and updates the Prometheus gauges accordingly.

Promenade::PeriodicStats.configure(frequency: 1, logger: Rails.logger)  do
  Promenade::Pitchfork::Stats.instrument
end

• pitchfork_workers_count: Total number of configured workers.
• pitchfork_live_workers_count: Number of live (booted) workers.
• pitchfork_capacity: Number of idle workers.
• pitchfork_busy_percent: Percentage of busy workers.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (328463b) to head (997f682).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##            master       #63   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           34        37    +3     
  Lines          761       855   +94     
=========================================
+ Hits           761       855   +94     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[errm]

This seems fine to me.

It's a bit of a weird design, but that is basically forced on us by the decision we took to have the exporter endpoint running in a separate sidecar process...

The downside of this is:

e.g. prometheus is scraping every 30 seconds, we have to decide how often to update the metrics in the PeriodicStats, if we choose 30s too, then the stats could be 29 seconds out of date ... and sometimes we could just scrape the same data twice if things line up wrongly, clearly every second is a bit wasteful (especially if the stats thing we are calling is expensive)

If the stats method we are calling is lightweight enough, we could just add it to rack.after_reply like I did with the yjit instrumentation... https://github.com/errm/promenade/blob/master/lib/promenade/yjit/middleware.rb

• If pitchfork supports that 🤷

[robertomiranda]

closed by #66