problems about hyperparameters configuration

[jxqhhh]

Hi! I think your LDMNet-pytorch project is really excellent, but I found out that if I ran using 'python main.py with mnist' then the train loss would greatly increase at last. I think this may be due to that you're confused by these two different concepts——"epoch" and "iteration". And in the paper the author use the term 'epoch' instead of 'iteration'.
When I change the hyparameters by using 'python main.py with mnist epochs_update=200 max_epochs=50000' (which means I use 100 iterations for every epoch), I find that the train loss will steadily decrease rather than greatly increase.
Looking forward to your reply, thanks a lot!

[thalitadru]

Hi, Thanks for your interest in this implementation. Unfortunately I'm not maintaining this code regularly and it is lacking more documentation (sorry for that). About the two parameters you mentioned:

• The parameter epochs_update refers to the update frequency of the manifold coordinates (i.e. every epochs_update epochs, an alpha update is computed).
• The parameter max_epochs refers to the number of epochs used in training.
  I do not handle iterations directly at any stage, as it is handled inside the skorch API. You may interfere with them by changing the batch_size argument.
  I hope this clarifies your doubts, I'm open to more questions.

[Christian-lyc]

I found that when you increase the train size, the train loss will decrease steadily.

[jxqhhh]

I found that when you increase the train size, the train loss will decrease steadily.

Does the so-called train size means the batch size?

[Christian-lyc]

I found that when you increase the train size, the train loss will decrease steadily.

Does the so-called train size means the batch size?

Sorry, I re-tried and found when you set epochs_update=1. The train loss will decrease under some hyparameters (\mu and \lamda_bar). Or maybe we need to adjust both train_size and the other things we can tune.