Introduce trace level logging

[tomschr]

Situation

The standard logging library supports different log levels: DEBUG, INFO, WARNING, ERROR, and CRITICAL/FATAL.

However, there is no way to have a TRACE level for more extensive debugging.

Use Case

The TRACE logging level is the most granular and verbose setting in a logging strategy. Having everything as DEBUG will soon makes the log unbearable long. Splitting the level into some DEBUG and some TRACE makes the log shorter. We can switch TRACE level on if needed.

For example, a TRACE level could be helpful in these situations:

• Debugging complex loops and algorithms
  If you have a for or while loop that runs hundreds of times and fails on iteration #47, DEBUG might be too spammy if it only prints "Looping...", but TRACE is perfect for printing the loop index and variables at every single turn.
• Data parsing and string manipulation
  When parsing JSON, XML, or binary data, a single missing character can break the code. TRACE is excellent for logging the raw payload before and after transformation.
• Command execution
  Dealing with many daps calls and other commands, TRACE can be helpful to see the exact command invocation and the results.
• Concurrency and race conditions
  In multi-threaded applications, you need to know exactly when a thread enters or leaves a method to spot race conditions. TRACE can log the precise entry and exit points with thread IDs.

Possible Implementation

import logging
import sys

# Configure logging
# Note: Python doesn't have a built-in TRACE level, 
# so we often map it to value 5 (below DEBUG which is 10)
TRACE_LEVEL_NUM = 5 
logging.addLevelName(TRACE_LEVEL_NUM, "TRACE")

def trace(self, message, *args, **kws):
    if self.isEnabledFor(TRACE_LEVEL_NUM):
        self._log(TRACE_LEVEL_NUM, message, args, **kws) 

logging.Logger.trace = trace
logger = logging.getLogger(__name__)

# Set level to TRACE to see everything
logger.setLevel(TRACE_LEVEL_NUM) 
handler = logging.StreamHandler(sys.stdout)
formatter = logging.Formatter('%(levelname)s: %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)

def calculate_factorial_sum(numbers):
    total_sum = 0
    
    logger.debug("Starting calculation loop...")
    
    for num in numbers:
        # DEBUG shows us the flow
        logger.debug(f"Processing number: {num}")
        
        factorial = 1
        for i in range(1, num + 1):
            factorial *= i
            # TRACE shows us the granular internal state
            logger.trace(f"   [Inner Loop] num={num}, i={i}, current_factorial={factorial}")
            
        total_sum += factorial
        logger.trace(f"   [End Outer Loop] total_sum is now {total_sum}")

    logger.info(f"Final Result: {total_sum}")

# Run the function
data = [3, 4]
calculate_factorial_sum(data)

NOTE: Because TRACE generates a massive amount of string operations and I/O (writing to disk/console), it significantly slows down applications. A QueueHandler is a must to handle the massive amount of logs in an efficient way.