fix(sd): update extract_from_text #1293
Conversation
Solves #1292 Now parsing: disposition, docket_number and judges
| "aff in pt, vacate, & rem in pt": "Affirm in part, vacate and remand in part", | ||
| "aff in pt & vacate": "Affirm and vacate", # https://www.courtlistener.com/opinion/9502826/state-v-scott/pdf/ |
There was a problem hiding this comment.
We may need to fine tune these dispositions.
There was a problem hiding this comment.
I can see this one
aff in pt & vacate": "Affirm and vacate"
should be
aff in pt & vacate": "Affirm in part and vacate"
Any other?
There was a problem hiding this comment.
I would suggested the past tense. Dismiss should match Affirmed. Dismissed. Reversed and Remanded, ... etc.
Also - Aff in pt & vacate should be Affirmed in Part and Vacated
flooie
left a comment
flooie
left a comment
There was a problem hiding this comment.
The current regex pattern for citations is too restrictive. It seems to require a citation for processing, causing cases without one to be skipped. This should be loosened to ensure that all relevant data is captured, even when a citation is missing.
The back scraper and extract_from_text method only work with PDFs from 2005 onward, as that’s when the court transitioned to a new format. Before 2005, the extraction fails due to a change in text patterns.
Should add HTML cleanup code as well.
The regular backscraper is likely to fail starting around 2009-2010 because of overly strict regex constraints. Adjusting the pattern to accommodate format variations would improve reliability.
overall I liked what you did.
|
@grossir this is still with you right? |
|
@flooie yes, I still have to get back to this |
grossir
force-pushed
the
fix_sd_extract_from_text
branch
from
5712f7b to
d680a1a
Compare
grossir
force-pushed
the
fix_sd_extract_from_text
branch
from
d680a1a to
e94da40
Compare
|
I didn't implement logic for the HTML pages before 20056 since we have that data and it would complicate the scraper logic for little gain |
Solves #1292
Now parsing: disposition, docket_number and judges