-
-
Notifications
You must be signed in to change notification settings - Fork 31
Description
Scraping / Inserting data via Selenium was originally done to make the usage of this synchronisation script easier. Especially since some sites didn't offer an API, it was not free to use, or it was lacking data (e.g. for private profiles).
Since some sites have introduced some quite effective anti-bot measures (e.g. captchas on IMDb), this reduces the usefulness of this script/application.
With this issue, re-implement the most-important APIs to replace the Selenium-based scraping&inserting. The sites where a simple CSV upload/download is performed are out-of scope for the first phase of this.
The user might then have to generate private API credentials (e.g. client ID + secret) - this would need to be documented as a step-by-step how-to guide.
For each of the sites, this should be a feature toggle to let the user chose which way to use.
Metadata
Metadata
Assignees
Labels
Projects
Status