Option `--no-log-color` seems to has no effect

[lazypingu]

Hi,
I noticed that a log file specified via --log-file together with --no-log-color still contains the ANSI color escape sequences.

How to reproduce
Perform e.g.:

ugrd --no-log-color --log-file log ./

and check the log file for colors.

Used versions

• zenlib 3.1.5
• ugrd 2.0.2

Not urgent or important at all! Just wanted to mention since I stumbled across this.

[desultory]

Thanks for opening an issue about this. I'm aware of the problem and it has a bit of history.

The --no-log-color bit more or less makes it so it doesn't use the "color log formatter" which mostly exists to color the log levels. The colorize stuff (what is used for the log line contents in most cases) is another thing entirely, and I've considered making this option do something to effectively disable the colorize function, but I'm hesitant to do that because it won't work with manual ansi codes, and if you're trying to use colorize for something other than logging, it breaks that.

I don't know if a simple way to strip ansi stuff, but am open to ideas and think it would be cool to implement this in the logger itself so it can strip contents instead of simply adjusting the formatter.

This logic is handled here: https://github.com/desultory/zenlib/blob/main/src/zenlib/util/main_funcs.py#L46-L93

[lazypingu]

Hm, not sure how this could be done in the logger itself.
What comes to my mind would be a "custom" formatter instead of the formatter from the logging library used here:

zenlib/src/zenlib/util/main_funcs.py

Line 79 in 8752066

formatter = ColorLognameFormatter(format_str) if not args.no_log_color else Formatter(format_str)

This formatter could parse the record and strip the ansi sequences via an regex.
Not sure if this would be a clean or elegant solution.

[desultory]

Hm, not sure how this could be done in the logger itself. What comes to my mind would be a "custom" formatter instead of the formatter from the logging library used here:

zenlib/src/zenlib/util/main_funcs.py

Line 79 in 8752066
formatter = ColorLognameFormatter(format_str) if not args.no_log_color else Formatter(format_str)

This formatter could parse the record and strip the ansi sequences via an regex. Not sure if this would be a clean or elegant solution.

yea that's about the best way I can think of but I'm not sure how great it would be. ANSI sequences can be simple or rather complex. It can be a bit hard to safely remove the format codes while preserving the text. I'm also not sure how slow it would be if applied to EVERY logged line, you could maybe optimize it to only do that whenever it detects the start code, but it still feels a bit messy to me. If python had some built in way to strip this stuff, I'd already be using it. I hesitate to use an external library for just this function as well.

It gets messier because if the color logname formatter is not used, but colorize is used within logging functions, i don't think this filter would work. You'd have to make it so another special formatter is used when the colored one is not used. This could possibly work for setups where loggify us used, but if the user manually provides a logger to a class, this won't work (I guess that's ok). I think some sort of global toggle that colorize can use could maybe be more thorough but if for some reason colorize is used to create formatted files, or used outside of a logger, then it would break.

If a "decolorize" function was added, it could possibly be used with an alternate formatter which is used instead of the default one, the main concern is how slow that could be because it doesn't stop it from formatting it in the first place.

[lazypingu]

Maybe it's also worth to ask:
Is this really worth the effort, additional code complexity and potential speed concerns?

A "decolorize" could probably also be done with external tools if needed.
But that's probably more a genreal decision.

[desultory]

concerning speed, I think the slow part will be the decolorize bit that would be used anyways (if the goal is to totally suppress color codes from log output).

If you can make a decolorize function, I could help add some test case for it. I think testing it against colorize first would be nice, but colorize doesn't cover all ansi codes, and at some points, ugrd may log the output of a program so arbitrary color codes should be considered. As long as running it doesn't make things noticeably slow it's fine, but I'd be interested to see if any performance impact could be measured. I'm just guessing it could have an impact since it will run against all logged output, and with log level 5 (-dd) that could start to make a real impact.

[lazypingu]

A very simple (and naive) variant could use a simple regex pattern to remove the ANSI color sequences like this:

from re import compile

pattern = compile(r'\x1B\[(\d*;)*\dm*')

...

def decolorize(record):
    return pattern.sub("", record)

The pattern would be compiled once and used multiple times, which might be helpful regarding efficiency and speed.
I've shortly tested this with a sample programm and the output of a test ugrd run, which seems to work fine.

Note that the regex used here is very simple and potentially not comprehensive. There are better variants out there for matching ANSI sequences. Also sites like https://regex101.com/ can be helpful for testing these patterns.

I currently do not have much free time for this, so this might be just a tiny humbe idea.

[desultory]

A very simple (and naive) variant could use a simple regex pattern to remove the ANSI color sequences like this:

from re import compile

pattern = compile(r'\x1B[(\d*;)\dm')

...

def decolorize(record):
return pattern.sub("", record)

The pattern would be compiled once and used multiple times, which might be helpful regarding efficiency and speed. I've shortly tested this with a sample programm and the output of a test ugrd run, which seems to work fine.

Note that the regex used here is very simple and potentially not comprehensive. There are better variants out there for matching ANSI sequences. Also sites like https://regex101.com/ can be helpful for testing these patterns.

I currently do not have much free time for this, so this might be just a tiny humbe idea.

thanks for that, and yeah I don't have a lot of time either (especially during the week) right now.

I'll try to make some test cases for that

[lazypingu]

Thanks! Not urgent or important at all! Just stumpled accross it so I wanted to mention.
Thanks for your great contributions!

[desultory]

Thanks! Not urgent or important at all! Just stumpled accross it so I wanted to mention. Thanks for your great contributions!

yeah I was thinking it could kinda be an alternate mode to this:

zenlib/src/zenlib/logging/colorlognameformatter.py

Lines 24 to 34 in 8752066

	def format(self, record):
	# When calling format, replace the levelname with a colored version
	# Note: the string size is greatly increased because of the color codes
	from zenlib.util import colorize
	old_levelname = record.levelname
	color_info = self.level_colors.get(record.levelname, {"color": "white"})
	record.levelname = colorize(record.levelname.ljust(self.level_str_width), **color_info)
	format_str = super().format(record)
	record.levelname = old_levelname
	return format_str

There could just be a "DecolorLognameFormatter" or something, I can't think of a better name at the moment but you'll notice that's a trend with names like "loggify, colorize, and ugrd" :P