ref(project-cache): Yield back to runtime in expensive branches

[Dav1dde]

Be a bit more cooperative for the runtime and yield back after expensive messages.

Eviction is quite costly for large projects, and with scheduled fetches being biased towards it's possible that we do this a lot before being force yielded by the runtime.

#skip-changelog

[jjbayer]

Is the purpose of this to prevent the project cache service from hogging the runtime, and increasing the probability that other services make progress?

Wouldn't the runtime itself already be pretty good at this?

As of 0.2.14, each Tokio task has an operation budget.

https://tokio.rs/blog/2020-04-preemption

[Dav1dde]

Wouldn't the runtime itself already be pretty good at this?

Yes it is, but we can still give it a few hints, especially for the eviction branch, the eviction worst case is taking 50ms of CPU, but that does not decrease the cooperative budget, because it is a purely synchronous call and nothing is using the budget. So even after doing that long task, the runtime spends time on the task.

See #4241 to eliminate this long running blocking task, which is probably the better approach.

[Dav1dde]

No longer necessary with #4241.