Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

regularly clean up closed SESSIONS entries #54

Open
martinpitt opened this issue Sep 28, 2022 · 1 comment
Open

regularly clean up closed SESSIONS entries #54

martinpitt opened this issue Sep 28, 2022 · 1 comment

Comments

@martinpitt
Copy link
Member

The running pods and sessions map in redis can go out of sync for any number of reasons: session pods crashing and the "websocket EOF" message not making it through, or API pods crashing at the wrong time, or new sessions in state wait_target which were never connected to. In the app service

  • track the time of session creation in SESSIONS
  • do a regular sweep over:
    • all running session pods, and delete the ones which are not in SESSIONS
    • all SESSIONS entries, and delete the wait_target ones which are older than 1 hour, and the closed ones, and the running ones which are older than, say, one day?
@martinpitt martinpitt changed the title regularly clean up abandoned pods and SESSIONS entries regularly clean up closd SESSIONS entries Sep 30, 2022
@martinpitt martinpitt changed the title regularly clean up closd SESSIONS entries regularly clean up closed SESSIONS entries Sep 30, 2022
@martinpitt
Copy link
Member Author

Retitling. Our deployer service account in k8s cannot remove pods, and there is no other standard account that can. In PR #61 I let the session containers die by themselves. So this reduces to cleaning up the SESSIONS map.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant