Configure the level of IPAM with UDNs

[phoracek]

The current proposal for UDN's IPAM configuration is limiting as it would not allow a (simple) expansion covering other anticipated use-cases. This patch aims to make it more future-proof.

There are two limitations that this patch should solve:

1. The current proposal does not leave space for default Subnets value
2. The current proposal does not leave space for granular control of IPAM

This patch should solve both by introducing a new attribute called IPAMLevel. This attribute dictates how much of the network IP configuration will be done by OVN and how much is left to the user.

The initial implementation of 4.18 will need only two values: Disabled and FullyManaged. Later releases should introduce a new level, where OVN will manage configuration of logical routers (routing, NAT, default GW), but configuration of individual Pod's / VM's IPs will be left to the user.

Apart from introducing this new field, this patch also changes the behavior of nil Subnets. While with the current enhancement not setting subnets means disabling IPAM, with this patch, not setting subnets is not allowed unless Disabled IPAMLevel is explicitly set. This will allow us to introduce default subnet value in the future, if we decide to go that path.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[maiqueb]

I have some wording suggestions to make it clearer.

I think it is a nice suggestion, that opens the door for a more granular IPAM configuration, according to the use-cases sought by the user.

[maiqueb]

Tks.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: maiqueb
Once this PR has been reviewed and has the lgtm label, please assign abhat for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• enhancements/network/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tssurya]

/assign @trozet / @npinaeva / @ormergi / @jcaamano for reviews

[tssurya]

I assume for layer3 there can never be a disabled mode right? for both role primary and role secondary?

[phoracek]

I think so too - L3 has to do routing between nodes, and it could not do routing without subnet awareness.

[phoracek]

Tell me if you would prefer me to reword this line. I think that the condition is already present in the list, but it is not obvious (subnets and ipamlevel points need to be merged).

[openshift-ci]

@phoracek: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[npinaeva]

I think if we already have some future options like partially-managed in mind, it would be nice to include those options into this enhancement, to make sure our current API will be properly extended when needed. we don't need to implement all of the options at the same time though.

Trying to figure out important parts of the subnet management process

1. disabled (empty subnet now) - doesn't support most of the ovn-k features, including egress traffic outside the cluster (would be nice to have a full list of unsupported features, or, if easier, what is supported). Is not allowed for primary networks.
2. subnet-no-ipam - subnet is known to the ovn-k, but the ip allocation from that subnet is not done by ovn-k. That should allow egress traffic (again, full list of supported/unsupported features would be nice here). Based on the feature set, we may need to figure out if this should be allowed for primary networks or not.
3. fully-managed - IPAM is done by ovn-k, all UDN features are supported, can be used with any network type.
   IIC, Options 2 and 3 could support default subnet allocation, but as we know UDN inter-connect may happen at some point, we should discuss the algorithm for that default subnet selection from the beginning.

@phoracek does that ^ sound right?

[phoracek]

@npinaeva thanks for the review. My goal with this PR is to keep the API open for the anticipated enhancements. I wanted to avoid designing the specific IPAM feature here - I don't trust myself to do it right and I'd rather leave that to you subject matter experts.

But if you would find it useful to have an example of such an extension in this PR, I'm happy to do that (just take it with a grain of salt).

Your summary of the 3 options matches my understanding / expectations +1.

@npinaeva what would you like me to add to this PR? I can add the third option for IPAMLevel and describe it with a few words. I don't feel comfortable going in detail about each of the options though - this PR is not introducing those functions, and I'm not an SME/authority to say what their "mission" is.