Fix: dataloader state dict indexerror

[esivonxay-cognitiv]

Before submitting

• Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

What does this PR do?

This fixes the issue reported in #196.

Note: I had to change part of another unittest in this fix. I think the existing test was written expecting _num_samples_yielded_combined to be the combined total # of samples yielded by the CombinedStreamingDataset.

However, I believe the StreamingDataLoader._num_samples_yielded_combined field is a dictionary which maps the worker_id to a list containing the number of samples yielded from each dataset within a CombinedStreamingDataset.

Is my understanding of "combined" in _num_samples_yielded_combined correct?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (main@3be136b). Learn more about missing BASE report.

Additional details and impacted files

@@          Coverage Diff          @@
##             main   #198   +/-   ##
=====================================
  Coverage        ?    78%           
=====================================
  Files           ?     33           
  Lines           ?   4488           
  Branches        ?      0           
=====================================
  Hits            ?   3492           
  Misses          ?    996           
  Partials        ?      0