Sorting out logging output

[michaelwood]

Switching to using logging

logging should be a slightly better way to deal with increasing need for output and possibly different locations for the output

Considerations

• Which errors needs to go to stderr , are they a fault with the application or are they handling something expected but unusual in the data and therefore need to notify the user
• What level to use for what errors?
• Does the data run log that is on the dashboard capture both stderr and stdout?
• We often use call_command and check the output is expected. In call_command the stderr and stdout are local to that instance rather than sys.stderr/stdout
• What is a useful string format for the logging (do we want it always in a CSV format for example)

django call_command

A wrapper for call_command?

def call_command_with_logging(...):
   stream = TextIO()
   handler = StreamHandler(stream)
   logging.getLogger("").addHandler(handler)
   call_command(...)
   logging.getLogger("").removeHandler(handler)
   return textio

or in each management command we could setup the handler

#...
from django.core.management.base import BaseCommand, CommandError

from logging import StreamHandler

logger = logging.getLogger(__name__)
handler = StreamHandler()
logger.addHandler(handler)


# class Command(BaseCommand) [ this is the context ] :
    def handle(self, *args, **options):

        # By default 'logging' will log to sys.stderr which means that call_command does not capture any output with it's `stderr=<StringIO>` 
        handler.setStream(self.stderr)

#...

[michaelwood]

related to #190