Rework the logging module.

[trivialfis]

This is a summary of various issues related to the existing logging module.

• It's a thread local variable, which runs into issues with logging in child threads and distributed systems. Logging callback should not be thread local. #9147
• It creates symbol conflicts. LOG_DEBUG/LOG_INFO/etc... preprocessor macros create collisions with project that use those already #6678 CHECK_* macro in logging.h collides with CHECK_* macro from absl #10400
• It doesn't translate into binding-specific logging facility very well. For example, how to use Python logging with file output?
• It should have a separate channel/logger for different types of logs. For instance, the memory allocation log should not be mixed into other debug logs.

[CodersAcademy006]

A possible solution to rework the logging module:

1. Thread-Safe Central Logger: Replace thread-local logging with a singleton logger that is safe for multi-threaded and distributed environments.
2. Namespace-Safe API: Wrap logging macros under a namespace, e.g., xgb::log::Debug("message"), to prevent symbol collisions.
3. Python Integration: Redirect logs from C++ to Python’s logging module and provide a Python API to set log levels, output files, and handlers.
4. Log Categories: Introduce channels like Training, Memory, IO to filter logs effectively.
5. Configurable Logging Levels: Allow runtime setting of levels (DEBUG, INFO, WARNING, ERROR) via C++ or Python APIs.
6. Backward Compatibility: Maintain old macros temporarily with deprecation warnings.

I'll try to implement it once and then submit a pull-request for this

Thank You.

[hcho3]

Thread-Safe Central Logger: Replace thread-local logging with a singleton logger that is safe for multi-threaded and distributed environments.

NO, the global singleton is even worse than thread-local singleton. Please do not proceed further with this design.

[hcho3]

@CodersAcademy006 Please share with us a design doc before doing any more work on this. The design doc should explain where and how the callback method will be configured and saved. The new design should at minimum avoid thread-local singleton and global singleton.

[CodersAcademy006]

Hi @hcho3, thanks a lot for the feedback on my initial suggestion. You were right, a global singleton isn't the way to go, and I appreciate you pointing me in a better direction. After thinking about your requirements, my new proposal is to use an explicit Context object. The idea is pretty simple: it's an object that holds settings like the log level and a callback, which users would pass directly into functions like Train(). This would make the logging setup explicit, thread-safe by design, and provide a much cleaner hook for language bindings. If this general direction makes sense, I’d be happy to write up a full design doc that covers all the practical details, including a solid backward-compatibility plan and how this could support more granular controls in the future.

Will that work?