Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hotpot.json data is repeated #60

Open
fantastic-413 opened this issue Nov 9, 2024 · 1 comment
Open

hotpot.json data is repeated #60

fantastic-413 opened this issue Nov 9, 2024 · 1 comment

Comments

@fantastic-413
Copy link

There are two identical data (the 6th and 506th data) in the hotpot.json file.
image
So when you run the results of ircot_hipporag.py on the hotpotqa data, if you interrupt halfway (idx>=6) and rerun the code, due to this mechanism, you will skip the 506th data and only get the results of 999 data.
image
image
But the results of using 1000 data but with the same data may not necessarily be better than the results of 999 data.

@yhshu
Copy link
Contributor

yhshu commented Nov 9, 2024

Thank you for your feedback. We will fix this dataset problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants