Investigate and possibly implement real-time tracing: qb-blackbox { --pid <PID> | --command <cmd> }

[jnpkrn]

Currently, there seems to be an opportunity to get on-the-fly stream of messages
temporarily hitting blackbox, enabler being on-disk materialized files.

This would also give an opportunity for a deepest look into the run possible,
beside mere debugging/stracing/probing, which is rather cumbersome for
not so isolated observations.

[kgaillot]

A similar capability is already available using signals: SIGUSR1 starts the collection of data, and SIGTRAP causes a write-out of the blackbox.

[jnpkrn]

Thas misses several points:

• you talk just about pacemaker-specific provision
• blackbox has a limited capacity whereas stream is possibly
  "endless'
• may not solve infloop/deadlock problems when signal handling
  will never be served

Compared to big-gun solutions like attaching debugger, this would
save one from active attendance on debugging session, stopping
the process (making the timeouts kick in), expertise required from
the user. This indeed assumes the trace messages are spread
reasonably in the code.