Idea: Resource utilization based specific read query limiting

[rishabhkumar92]

What is the problem you are trying to solve?

Currently Mimir supports resource utilization based ingester read path limiting but it blocks all read when utilization is above certain threshold.

Which solution do you envision (roughly)?

Block selective query from dashboards i.e dont block alerting and recording rule to have low impact in case utilization is spiky on Ingesters.

Have you considered any alternatives?

No response

Any additional context to share?

No response

How long do you think this would take to be developed?

Not sure

What are the documentation dependencies?

No response

Proposer?

No response

[dimitarvdimitrov]

this is related to what we were speaking with @jhalterman, @pracucci and @pr00se about quality of service when reactive limits turn on in ingesters. I think rules can have a higher QoS than dashboard queries.

[rishabhkumar92]

This will be very helpful for not blocking recording rules path as that's permanent data loss which we can't even replay.