V2.x - Attribute support

[felixjulianheitmann]

This is quite similar to the MDC support by #2907 except for one major difference: MDC is stored in global thread local storage vs. attributes are stored per logger.

Each logger has an attributes_ map which is a std::string/std::string map. During logging, these attributes are passed with the log message to the formatter. An attributes_formatter (%*) will format these the same way that the mdc_formatter does.

I could use inheritance or a common function to reduce the code duplication.

What's the use case?

We are using spdlog on a webserver and have multiple threads handling requests. They share a common logger instance. Since there are multiple threads writing to the same logger, we cannot use MDC to associate additional context to the log messages. The thread writing to the sink will always be a different one than the one executing the web-handler.

This approach will provide the ability to have context associated with a logger instance rather than with a thread. I can have multiple threads writing to the same logger and providing context.

Considerations

1. It's thread safe but is it useful?

After the implementation, I noticed a flaw in my train of thoughts. If I have multiple threads that can safely put context on a logger and write them together with the log messages, I still get a form of race condition.

Logger A wants to do some file operation and calls logger.attrs().put("file-path", "my/path.txt") to have the file path be logged with any subsequent logs.

Logger B may fail parsing an incoming JSON and calls logger.attrs().put("error", exception.what()).

If these two then log "at the same time", the logging message will contain both context attributes.

Maybe I shouldn't reuse the same logger for entirely different operations. But even if they are similar, they are not the same and logs from one task don't matter for the other. I am now quite confident, this is what @tt4g tried to explain to me in #3083 (comment) that I am just grasping now.

2. using a different map

Currently every attribute has to be formatted as a string before inserting it into the API. Maybe this can be worked around by providing a formattable type or similar in the attributes API rather than a string. I haven't dug into fmtlib/std::format enough to see whether that's possible somehow without going down the template rabbit hole.

[felixjulianheitmann]

1. It's thread safe but is it useful?

Maybe the goal is slightly different. Maybe, I do want thread local storage but on a per-logger basis and I would want my contextual info to be forwarded to the thread pool actually writing to the sinks.

I cannot use single-threaded loggers but I also don't really see the benefit of having multiple threads mix their current individual context

[felixjulianheitmann]

Okay, some builds seem to fail. I will have a look at that next week.

[felixjulianheitmann]

Regardless, of whether the additional builds pass, this PR is in a philosophical dilemma.
As far as I can see, there is two ways of implementing contextual attributes for log messages:

1. an attribute map valid across all logger instances on a per thread basis
   -> current MDC implementation does that
2. an attribute map valid across all threads on a per logger instane basis
   -> this PR implements that.

Shortcomings of 1.:

• no async loggers possible
• unrelated loggers logging on the same thread share the context state

Shortcomings of 2.:

• contextual info is shared across possibly unrelated tasks. Our use-case of a thread pool for web-handlers sharing a single logger instance would mix the contextual state of possibly different tasks.

A final less convenient solution: per-message context info
We could overload the info/warn/error/... calls to allow the inclusion of per message contextual info. That kind of defeats the purpose though. If I want to include additional info in this specific message ... just write it into the message.

My bottom line is: this PR makes sense to me, personally. For the project I want to use it for, our structure needs to be updated. If we used a separate logger instance per web-handler thread pool, this PR would perfectly solve our issues.

I will go ahead and see, if I can fix the build issues and discuss internally, how to proceed.

[tt4g]

As far as I know, logging libraries are torn between implementing either of these two.
Personally, I prefer this PR implementation.

[felixjulianheitmann]

Future improvements could include templated versions that accept any form of attribute type that's formattable by the fmt library. This implementation is kept simple for a first implementations sake.

Any future work on this should be doable without breaking the interface.

[bobhansen]

In https://github.com/bobhansen/structured_spdlog, it was important that the data context be thread local (so inner library functions could include context they were oblivious to), but also be able to pass it between threads for context-sharing (e.g. a webserver passing the session ID to worker threads).

We included an explicit method to snapshot and assume contexts (which we implemented as linked lists of std::shared_ptrs).

[felixjulianheitmann]

Explicit methods for passing the thread memory boundary or for putting something in thread local memory seems reasonable.

This has been open for quite some time and we've been thinking about it. Usually, the context you're working on stays on the same thread. You push context, perform the work and kill the context.

If you have specific worker threads - a web server or some offloaded calculations -, these threads are started explicitly and are loaded with work deliberately. That's where it seems reasonable to also push the context onto the thread.

It doesn't fit our current code architecture but seems to make the most sense. I like your approach