Feature Request: A/B Testing flow modification to accommodate incompatible versions

[SuphalAthlur]

Describe the feature

Currently, this is the flow in A/B testing strategy after the canary analysis phase succeeds:
Route all traffic to primary -> Copy new template spec to primary pods -> Scale down canary deployment

Issue: This would cause primary pods to serve two versions during the brief period when the update is being rolled out- some applications may not be tolerant to this. There should be a way to avoid this.

During canary analysis, this can be mitigated by using header based matching (spec.analysis.match) for routing traffic to primary/canary. By obtaining the user's identity/session cookies from the header fields, we may ensure that a particular session (or a user or a set of users) is always served a particular version. But we do not have any such mechanism once the canary succeeds and the rolling update is underway on the primary deployment - which is serving all the live traffic.

Proposed solution

Proposed flow:
Route all traffic to canary (successfully tested right before) -> Copy template spec to primary pods -> Route all traffic to primary -> Scale down canary deployment
Similar to blue-green strategy.

How it solves the issue: This would make sure that, at no point, is a deployment serving live traffic - hosting pods serving different application versions. This alternative flow could be provided as a configurable option, thereby not denying anything existing to Flagger users.

Any alternatives you've considered?

If Istio is being used as the service mesh provider, then maybe using it's sticky sessions support could help: https://istio.io/latest/docs/reference/config/networking/destination-rule/#LoadBalancerSettings-ConsistentHashLB
But this would bring in many caveats:

• If a host that a session is coupled with is down, istio would pick another host at random without any regard for the application version being served by the old host
• This is because Istio is unaware of the application version information - it routes purely based on the destination host identity
• Such strict constraints may not be required - the usecase described here would be satisfied as long as the requests are routed to one of the hosts serving that version - not necessarily the same host each time
• This could make load balancing trickier and impose unnecessary constraints, Flagger has the potential to provide a much more elegant solution