Support separate network for HTTP service

[lentzi90]

It is quite common that a separate out of band (OOB) network is used for BMCs. This can make it tricky to use virtualmedia with Ironic when deployed by IrSO. I want to support the following configuration:

• Provisioning network: 192.168.0.0/24 - no VLAN. Provisioning IP belongs here, dnsmasq provides DHCP here.
• OOB network: 192.168.1.0/24 - VLAN 2. No DHCP. BMCs live here. Ironic needs access to this network to communicate with them. IRONIC_HTTP_URL belongs here since the BMCs need to access virtualmedia through it.

Before IrSO, I have been using IRONIC_HTTP_URL to achieve this. I am not sure if it is the "correct" way, but it works. I simply make sure that the node where Ironic runs has an IP assigned that I can use and set the variable to it. Our keepalived image only supports a single IP currently so I have been doing it in other ways. However, I am hoping that we could add this also as a feature there.

I do not have a complete design ready in my head, but here is something as a discussion starter:

apiVersion: ironic.metal3.io/v1alpha1
kind: Ironic
metadata:
  name: ironic
  namespace: baremetal-operator-system
spec:
  networking:
    dhcp:
      rangeBegin: "192.168.0.200"
      rangeEnd: "192.168.0.220"
      networkCIDR: "192.168.0.0/24"
    interface: "eth0"
    ipAddress: "192.168.0.100"
    ipAddressManager: "keepalived"
    imageServerIpAddress: "192.168.1.50" # new field
    # imageServerPort: 80 # already implemented
    # imageServerTLSPort: 443 # already implemented
    keepalived: # new struct, requires support in keepalived scripts
      - ipAddress: 192.168.0.100
        interface: "eth0"
      - ipAddress: 192.168.1.50
        interface: "eth1"
    # keepalivedOverrideConfig: | # alternative escape hatch
      # vrrp_instance V_1 {
      #   state MASTER
      # ...

[metal3-io-bot]

This issue is currently awaiting triage.
If Metal3.io contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.
The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[dtantsur]

This can make it tricky to use virtualmedia with Ironic when deployed by IrSO

Isn't what ExternalIP is for?

Btw, I think we need to expose IRONIC_HTTP_URL in some way, but I'd rather prefer this way to be just new thing like AccessIP, so that users don't have to think about the exact shape of the URL (ports etc).

[lentzi90]

Isn't what ExternalIP is for?

I need to check if this would work. My gut feeling is that it would cause IPA to try to send back inspection data through the ExternalIP, which would not work. IPA does not have access to the OOB network. Or is there a way to make IPA use the normal IP even when the external IP is specified?

[dtantsur]

Or is there a way to make IPA use the normal IP even when the external IP is specified?

No, not really. Then bootServerIP could be the reasonable thing to do indeed. I use bootServer instead of imageService in your example, because "image server" can also refer to the cache from which Ironic can serve images to IPA, and you're not going to change that. WDYT?

If we add keepalived to networking, maybe it's a good reason to deprecate ipAddressManager... Just food for thought.

[lentzi90]

Right, I see. I used imageServerIPAddress since we already have imageServerPort and imageServerTLSPort. But now I am not completely sure what they refer to. 😅 If they are for the same thing, then I think we should keep the same naming convention. Otherwise no objections for bootServerIP.

If we add keepalived to networking, maybe it's a good reason to deprecate ipAddressManager... Just food for thought.

Sure! We just need to make sure that it is clear how to enable keepalived. If we deprecate ipAddressManager, it would be mandatory to set some keepalived config to enable it I guess. It may feel clumsy to have the IP duplicated in the "default" use case:

spec:
  networking:
    interface: "eth0" # Still needed, or do we default it from keepalived?
    ipAddress: "192.168.0.100" # Still needed, or do we default it from keepalived?
    keepalived:
      - ipAddress: 192.168.0.100
        interface: "eth0"