ReduceLROnPlateau not reducing in certain scenarios

[rmchurch]

In distributed settings with the validation epochs less than the default patience (10), it appears the ReduceLROnPlateau scheduler is not reducing the learning rate, even though the globally averaged total_loss would appear to dictate it should. This could be because the global total_loss is only created in the logging, and the local total_loss does not meet the criteria for ReduceLROnPlateau.