Expand private-dependencies CI check

[timmc-edx]

We have a CI check in edx-internal called check-private-dependencies that checks if all of the regular PyPI private_requirements.txt are installable.

Acceptance criteria:

• Expand this to cover git+https/git+ssh dependencies as well
  • Will likely require rewriting URLs to all be git+https
  • Will likely require using GITHUB_TOKEN, as these are generally in private repos

[jristau1984]

Make sure there is a follow up ticket to get this check into the containerized world once ready.

[robrap]

@jristau1984 @timmc-edx: I'd like to move this to the backlog for several reasons:

1. I'd like for the dust to settle on how we handle private-dependencies before diving into this.
2. I think it is important to hit our deadlines for containerization, and I think this can wait.
3. It may be slightly more efficient to fix once in the containerized world.
4. Although this is annoying, I imagine it causes 15-30 minutes of interruption. If it takes 1-2 or more days to implement, I'd rather keep us (and mostly Tim) focused on containerization.

Most importantly, if I am wrong, and this keeps happening, we can easily bring it back on the board. Objections?

[timmc-edx]

I'm fine moving it to the backlog, but I'd characterize the impact a little differently. 15-30 minutes interruption is about right, but it also blocks the pipeline for an additional hour, delaying other work.

Taking another look, I'm actually a little confused about something -- the existing support for EDXAPP_EXTRA_REQUIREMENTS notes that it has to skip git-URL dependencies, but... we don't have any of those for EDXAPP_EXTRA_REQUIREMENTS. We do have them for EDXAPP_PRIVATE_REQUIREMENTS. The other thing is that all of the git-URL dependencies currently request an editable install (via extra_args: -e) but I'm not sure that that is even a meaningful thing. We could probably get rid of it but I don't want to do that experiment. We can try removing it for containerization, although that's going to make sync annoying too. In the meantime, that extra_args is going to interfere a bit with transforming the YAML into something we can pipe into pip in a PR check. That means we probably can't even expand this to the non-git-URL parts of EDXAPP_PRIVATE_REQUIREMENTS as quickly as I'd thought.

Summary: The "easy" portion of work would be more than I expected, and the "hard" part will probably become a little easier after containerization. So we should just do the whole thing after containerization, yes.

[robrap]

I moved to the backlog and set a review date for May. We can discuss after then based on where we are, and whether this is blocked or not.

[dianakhuang]

https://github.com/orgs/community/discussions/46566 This seems to be a discussion about the difficulty of getting the token to allow the reading of the other private repos.

[robrap]

Ideas:

• I have no idea if this would resolve the issue, but someone in that discussion is proposing use of https://github.com/qoomon/actions--access-token.
• Is it possible to have a github action kick off this install check outside of github and report back?

[dianakhuang]

yeah, i was going to try to understand exactly what the limitations of current github tokens are and it might be okay as long as we use the personal access token of a bot account. just need to understand what's going on here.