Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial PR to add support for stackoverflow teams #78

Open
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

allen-munsch
Copy link

@allen-munsch allen-munsch commented Mar 30, 2023

This PR does the following:

  • Downloads posts which consist of questions and answers
  • It adds the version to the requirements.txt
  • It adds aiosqlite to, and allows validate_config to await
  • It adds a rate_limit utility function, which Locks and appeared to work ( I was kicked off stackoverflow ) several times because they just turn off IP addresses
  • Fixes multiple Rendering issues with data-source-panel.tsx

I was super confused by stackoverflows api filtering. Like filter=!nOedRLbqzB means withbody so the post has the body markdown. Need to validate this is correct.

I tried to get the articles too, but ran into 406 errors, and it wasn't helpful. I'd need to

requests.exceptions.HTTPError: 406 Client Error: Not Acceptable for url: https://api.stackoverflowteams.com/2.3/articles?&team=asdfasdf&filter=!nOedRLbqzB&page=1&fromdate=1325397600

In any case, I also added an icon, and reduced the size of it.

2023-03-29_20-18_1
2023-03-29_20-18
2023-03-29_20-05

# i ran a test
python -m data_source.sources.stackoverflow
# but messed up the ack queue, cleared it, and it looked like stuff might be working
# delete from ack_queue_task where _id in (select _id from ack_queue_task);

So i also include a clear_ack_queue.sh, and a clear_data_source.sh

🎉 oh hey look at that

2023-03-29_21-29
2023-03-29_21-27

app/db_engine.py Outdated Show resolved Hide resolved
@allen-munsch
Copy link
Author

allen-munsch commented Mar 31, 2023

i'm not working on this any longer. feel free to reopen/fork. thanks

@Roey7 Roey7 reopened this Apr 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants