Recommendations for Replicability

[dehdari]

The notebook final_results_gender_test.ipynb can benefit from some slight modifications that will allow others to replicate exactly the results:

• After the label encoder is fitted, print what it looks like (i.e. lb.classes_), so that we know the order of the labels if we want to just decode without training our own model. This is a simple suggestion that makes a big difference. (I think it's ['female_angry', 'female_calm', 'female_fearful', 'female_happy', 'female_sad', 'male_angry', 'male_calm', 'male_fearful', 'male_happy', 'male_sad'], but I'm not 100% sure).

• Use a fixed seed for the shuffle in shuffle(newdf), something like shuffle(newdf, random_state=1) .

• Include a script or function that assembles the two different data sources into the RawData directory, from original Zip files. The main issue here is ensuring that copying the different subdirectories in the Savee dataset get the same filename changes (eg. " (1)", " (10)") as the checked-in notebook get. Different operating systems like MacOS and Linux behave differently than Windows in how they copy files when there is already an existing file by the same name. I wouldn't mind writing this, but I actually can't replicate the data assemblage. (cf. Training from scratch doesn't reach the same loss #22)

[orkenstein]

@dehdari did you reproduce the result using provided model?
I was able to get 'male_angry' with provided test .wav, but all the others audio-files are predicted incorrectly, even for the trainings data.

[dehdari]

Same here. Training doesn't reproduce exactly, and I get the same results as the OP in #22. Inference on the pretrained checked-in model works on the angry male example in Live Demo.

[amith-dsilva]

@dehdari how did you get the male_angry with the provided test. I am getting an error can you please help me out in this.