reproducing language modeling results

[tonytan48]

Hi Mr.Gong, congradulate on your work on new language modeling technique. I was trying to reproduce your experiment but some error of the dictionary occured:

If I substitute the data.py from original mos repo, like "import data_mos as data" the following error occurs:

May I ask for the solution for this? I am not sure is the problem of tokenization part "data.py" or the embedding_regularization part.

Best,
Qingyu Tan

[tonytan48]

Thank you for the prompt reply.

After pulling the latest branch with the updated "data.py". I still have an error of :
Traceback (most recent call last):
File "main.py", line 285, in
train()
File "main.py", line 228, in train
log_prob, hidden[s_id], rnn_hs, dropped_rnn_hs = parallel_model(cur_data, hidden[s_id], return_h=True, targets=cur_targets, is_switch=is_switch)
File "/home/qingyu.tan/miniconda3/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/qingyu.tan/projects/advsoft/mos-awd-lstm-lm/model.py", line 83, in forward
emb, sigma = embedded_dropout(self.encoder, torch.ones_like(self.encoder.weight), input, dropout=self.dropoute if (self.training and self.use_dropout) else 0, is_training=self.training)
File "/home/qingyu.tan/projects/advsoft/mos-awd-lstm-lm/embed_regularize.py", line 34, in embedded_dropout
X = embed._backend.Embedding.apply(words, masked_embed_weight,
File "/home/qingyu.tan/miniconda3/envs/py36/lib/python3.6/site-packages/torch/nn/backends/backend.py", line 10, in getattr
raise NotImplementedError

May I know can you reproduce the error?
My torch version is 0.4.1.post2