Skip to content
This repository has been archived by the owner on Jul 18, 2023. It is now read-only.

Pytest stalling on the tier jobs [need investigation] #1349

Closed
san7ket opened this issue Oct 30, 2018 · 3 comments
Closed

Pytest stalling on the tier jobs [need investigation] #1349

san7ket opened this issue Oct 30, 2018 · 3 comments
Assignees
Labels

Comments

@san7ket
Copy link
Contributor

san7ket commented Oct 30, 2018

No description provided.

@san7ket san7ket added the bug label Oct 30, 2018
@san7ket san7ket changed the title Pytests staalling on the tier jobs [need investigation] Pytests stallling on the tier jobs [need investigation] Oct 30, 2018
@san7ket san7ket changed the title Pytests stallling on the tier jobs [need investigation] Pytest stalling on the tier jobs [need investigation] Oct 30, 2018
@san7ket
Copy link
Contributor Author

san7ket commented Jan 7, 2019

Issue Walk-through:

  1. Pytest submits the Job to Saucelabs via it's tunnel.
  2. Saucelabs executes the test, send back the results. At the same time, it should close the current opened session for test, but it doesn't.
  3. This open session as a pytest worker associated with it gw0, gw1 etc., worker cannot stop as the session is still open.
  4. The session on the saucelabs will stay open indefinitely, until a interrupt from within saucelab is triggered. This interrupt would kill -9 the open session without notifying the pytest worker.
  5. In out automation, worker wait's indefinitely to receive data from sauce session.

Mostly occurs in long running tests, randomly in tier2 and tier3. Now that we know the issue, I will look into if we could add a timeout to workers executing tests and later identify all those test and re-check their logic

@san7ket
Copy link
Contributor Author

san7ket commented Jan 7, 2019

The issue, mentioned above creates a environment suitable for us hitting [1].

  1. test item scheduling does not schedule N items to N workers pytest-dev/pytest-xdist#40

@san7ket
Copy link
Contributor Author

san7ket commented Feb 27, 2019

Added few other fixes, closing this issue for now

@san7ket san7ket closed this as completed Feb 27, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

1 participant