Similar to how we are introducing strictRouting in #804, I am wondering if this config should be a per-rule basis instead of global. We can think of each routing rule as having a varying level of "strictness", from least to most:

• Strict: Route only to the target group
• Default: Fall back to default group if target group is all unhealthy
• Best Effort: Fall back to unhealthy clusters if all of target group + default group are unhealthy

cc @Peiyingy thoughts?

Different levels of strictness makes sense. However, if we move that to routing rule definition, is won't cover the default routing case at all. Maybe we can keep this global config, and allow per-rule config to override it? WDYT?

I wonder if we should move it so that the default rule is specified in the rule configs along the other rules, instead of being treated separately? Kind of like resource group definitions in Trino -- where the last selector is typically a "catch all" that you fall back to if no other rules match (example). Not sure if that totally makes sense for routing rules, though -- I forget all the places that the default is used.

Global config with override makes sense to me as well, if we make it a strictnessLevel config (same as the per-rule config)

Maybe we can keep this global config, and allow per-rule config to override it?

Not sure I understand correctly, the best effort will not make query choose a unhealthy target within the customized group, it only apply to adhoc/default group. What do we need the override for?

If we set best effort at global level, and the rule targeted group has no healthy cluster:

• without strictRouting, query goes to adhoc, may fail if adhocs are also unhealthy.
• with strictRouting, query fail at target group unhealthy clusters.

if we override the best effort per this rule, I think the result is gonna be the same?

But I agree that thinking in terms of routing “strictness levels” (STRICT / DEFAULT / BEST_EFFORT) is the right long-term model, and that it maps well to how we want routing behavior to evolve.
As for moving the rule to same fallback as resource group, I don't think MVEL has any blocker on this maybe it will add one more default rule to traverse every time we inspect a rule. Maybe we can align more on this and make a separate PR if needed since that remodels the rule behavior a bit, then come back and generalize the strictness.

if we override the best effort per this rule, I think the result is gonna be the same?

Thanks for clarifying that. Yes that would be the same here. I guess in rules, we can still decide that when there's fail back to default, do we want to make it bestEffort or not. Say globally that's disabled, but for this rule, if it falls back to defaults, and defaults are unhealthy, we still want it to fall back to unhealthy clusters. I'm not sure if this granularity is necessary, though.

I wonder if we should move it so that the default rule is specified in the rule configs along the other rules, instead of being treated separately?

Having a implicit default routing rule makes sense to me. I don't know if it's necessary to specify that.

I think under the current design, we've already handled all possible fallbacks:

1. If there are no available active healthy cluster for the selected routing group, fallback to ActiveDefaultBackends (ie. active healthy cluster in defaultRoutingGroup).
   -> Handled by StrictRouting

2. If there's no If there are no available active healthy cluster for the default routing group, fallback to unhealthy ones
   -> Handled by BestEffortRouting

I agree we should unify the naming, but the current code design makes sense to me.

I'm a bit concerned with the readability of the ternary branch. Shall we change it to something like this ⬇️ We can also benefit from different exception messages

List<ProxyBackendConfiguration> activeDefaults = gatewayBackendManager.getActiveDefaultBackends();
List<ProxyBackendConfiguration> healthyDefaults = activeDefaults.stream()
        .filter(backEnd -> isBackendHealthy(backEnd.getName()))
        .toList();

if (bestEffortRouting && healthyDefaults.isEmpty()) {
    return selectBackend(activeDefaults, user)
            .orElseThrow(() -> new IllegalStateException(
                    "No active default backend found under best-effort routing"));
}

return selectBackend(healthyDefaults, user)
        .orElseThrow(() -> new IllegalStateException(
                "No healthy default backend found"));