Skip to content

Issues: commoncrawl/news-crawl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

15 Open 41 Closed
15 Open 41 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

WARCBolt support for HTTP/2
#66 opened Oct 23, 2024 by sebastian-nagel
Port topology and resources to StormCrawler 3.1 or upwards
#60 opened Nov 16, 2023 by jnioche
3
Avoid following advertisements in news feeds and sitemaps
#58 opened Nov 14, 2023 by sebastian-nagel
produce WET files?
#55 opened Jul 19, 2023 by chris-ha458
6
Consider archiving of news feeds and sitemaps
#54 opened Dec 4, 2022 by sebastian-nagel
Explore schema.org annotations for seed completions
#53 opened Dec 4, 2022 by sebastian-nagel
Use wikidata to complete seeds
#50 opened Oct 18, 2022 by sebastian-nagel
1
Do not use "http/2" protocol version in HTTP headers in WARC files
#42 opened Oct 4, 2020 by sebastian-nagel
2
Allow to follow news sites not providing RSS/Atom feed or news sitemap enhancement
#41 opened Jul 24, 2020 by sebastian-nagel
2
Automatic removal of ephemeral sitemaps enhancement
#40 opened Jul 24, 2020 by sebastian-nagel
NewsSiteMapParserBolt: do not detect feeds as sitemaps
#35 opened Jan 9, 2020 by sebastian-nagel
Add HTTP protocol version to HTTP request message
#34 opened Dec 12, 2019 by sebastian-nagel
1
Check cross-submits for sitemaps
#32 opened Jul 5, 2019 by sebastian-nagel
1
Extract publishing date
#18 opened May 29, 2017 by fhamborg
2
Improve feed parser robustness
#13 opened Nov 23, 2016 by sebastian-nagel
@jnioche
4
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.