Is it possible to share the retrieved news articles? #9
Description
Hi there,
Thanks for open-sourcing the code. I was trying to re-create the retrieved news articles following retrieve_cc_news_bm25+ce.py. However, it is rather slow to run and uses a lot of memory (e.g., ~500 GB). I am not sure if my retrieved results are correct and consistent with what you were using. Can you help to clarify more details or share the retrieved results for reproduction purpose?
Specifically, which version of elasticsearch was used in your experiments? The BM25 model relies on elasticsearch and it seems that version 8.8 and 7.17 are both available as of now.
Moreover, because of connection instability, I could not directly download the CC news corpus, so I used the datasets provided in the previous github issue.
Currently, I am using elasticsearch 7.17 on Ubuntu and the shared CC news corpus to do the retrieval. It takes quite a lot of time to run it once and it'd be great if you can share more details about this step.
Many thanks,
Qi