Open
Description
This is possible for a subset of scrapers, which should be identified on a case by case basis. The subset is those that scrape a "Most recent page", meaning they will always return at least 1 result. If we get 0 results, we should send a logger.error()
that will go to Sentry
Examples of scrapers of the subset
This should be as easy as
- setting a default class attribute on
AbstractSite.should_have_results = False
- overriding the proper scrapers
should_have_results = True
- put a check on
parse
and send a logger if the conditions are metlen(self.cases) == 0 and should_have_results
juriscraper/juriscraper/AbstractSite.py
Lines 153 to 176 in 426dbfa
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
PR'd Issues 🤞