Process all relevant jobs from openQA

[mimi1vx]

As discussed in slack ->
https://suse.slack.com/archives/C02CANHLANP/p1709109430256279

this change forces bot process and sync all results, not only latest.

for example:
with latest:

for r in ret: print(f"{r['id']}: .... {r['result']}")
    ...
    13580193: .... softfailed
    13580199: .... softfailed
    13618826: .... passed

without latest:

 >>> for r in ret: print(f"{r['id']}: .... {r['result']}")
   ...
   13392313: .... failed
   13392314: .... failed
   13392315: .... failed
   13534621: .... passed
   13534659: .... failed
   13535630: .... failed
   13535631: .... failed
   13556841: .... user_cancelled
   13574056: .... failed
   13574057: .... failed
   13574058: .... failed
   13580193: .... softfailed
   13580199: .... softfailed
   13580851: .... softfailed
   13601665: .... failed
   13602658: .... failed
   13604911: .... failed
   13606694: .... failed
   13618826: .... passed
   >>>

[okurz]

LGTM

But @kraih can you judge about the impact of retrieving and storing more information from openQA into qem-dashboard?

[okurz]

@Mergifyio rebase

[mergify]

rebase

✅ Branch has been successfully rebased

[kraih]

But @kraih can you judge about the impact of retrieving and storing more information from openQA into qem-dashboard?

Hard to say, short term probably no impact. We already have 2.039.238 jobs in the dashboard database. This will accelerate our need to implement functionality for cleaning up old openQA jobs though. That does not exist yet and we are currently keeping job data indefinitely.

[Martchus]

With that, can we remove the "no ready" label?

[kraih]

Hard to say, short term probably no impact.

Small update on this. We do depend on an updated timestamp in some places to pick the latest job. If there are any ordering issues with how updates are submitted to the dashboard, i could see this become an issue and the wrong job getting selected.

Specifically the code here updates the timestamp every time. If an older job ends up getting the newest timestamp, we will run into ordering issues.

[okurz]

Hard to say, short term probably no impact.

Small update on this. We do depend on an updated timestamp in some places to pick the latest job. [...]

So what's your suggestion. Should this prevent us from accepting this PR?

[kraih]

So what's your suggestion. Should this prevent us from accepting this PR?

This should probably be double checked before the PR is accepted, yes.

[okurz]

@mimi1vx I assume in general the idea is still applicable. Would you like to rebase and state about manual test results or close?

[mimi1vx]

Its problem in discussed in slack valid ? And still it was stopped by @kraih :D

[okurz]

Its problem in discussed in slack valid ? And still it was stopped by @kraih :D

Sorry, I couldn't parse your sentences. Please rebase this and rephrase the commit or close the pull request.