Gradient accumulation was implemented on AllenNLP at the GradientDescentTrainer class after v. 0.9.0 ( see this pull request and the current trainer code ).
As this repository uses AllenNLP v. 0.9.0, the code should be upgraded to be compatible with the latest version of AllenNLP, and num_gradient_accumulation_steps should be allowed as a parameter in src/parameters.py .