Unsupervised pretraining for nnunet

[chris-rapson-formus]

I think unsupervised/self supervised pretraining is highly promising and will give fantastic results in the years to come
Isensee 2018

Given how expensive manual labelling is, and the success of less supervised pretraining in other deep learning tasks, I tend to agree. When the UnetFormer paper came out, a significant part of their claimed improvement seemed to be related to pretraining. I'm surprised that I haven't seen anything similar developed as part of the nnunet framework in the... 7 years since Fabian made that comment. Or have I missed something? Is pretraining a low priority for MIC-DKFZ because they target their work towards certain competitions, which have rules that limit how effective pre-training can be?