About model evaluation

[halcyon370]

Hello, is this model file "mobnet4f_cmu_adadelta_t1_model.pb" the same one just mentioned in your paper? I tried to evaluate on the CMU(MPII/NZSL) dataset, and I got AUC 0.875, which I expected to be 0.924 according to the paper. Could you give me some information about the evaluation details? Thanks!

[www516717402]

The same problem.

[philipposg]

Hello.
In order to reproduce the results you have to crop around the center of the hand.
As we mention in the paper:

For CMU we crop the images to a square box centered on the hand with side size of 1.2H, where H is the size of the person’s head.

Also, keep in mind that the model uploaded is trained only in the CMU dataset (in the paper it is denoted as CMU (224x224) ). Consequently, the expected performance on CMU should be 0.919 and not 0.924.

Best Regards.