Does it remove duplicates?

[ApioxFR]

When selecting a country, it will scrape locations of different cities some of them are very close to each other, and often results will overlap, this can lead to alot of duplicate scraping.

1. does the program remove duplicates while scraping or at the very end when "All tasks" are done?
2. Does the program skip duplicates when it encounters them during scraping ?

[Chetan11-dev]

Yes it does, remove duplicates from all task, and a specific task as well.