Do you know you use SyncBN when I do not running in a distributed environment and do not use --use_syncbn? It cause an bug which is hard to figure out. And your code even not give any error message. It wasted a great deal of my time and energy to debug it. Maybe my ability is rubbish, but I want to say this code really sucks.