Omni fails to start on Flatcar Linux [fdae:41e4:649b:9303::1]:8091: bind: cannot assign requested address

[navaneeth-dev]

I am deploying Omni on Flatcar Container Linux with Docker Compose but It fails to start:

Error:

omni  | {"level":"info","ts":1759491196.790328,"caller":"etcdserver/server.go:2319","msg":"server has stopped; stopping cluster version's monitor","component":"server","component":"embedded_etcd"}
omni  | {"level":"info","ts":1759491196.7903378,"caller":"etcdserver/server.go:2342","msg":"server has stopped; stopping storage version's monitor","component":"server","component":"embedded_etcd"}
omni  | {"level":"info","ts":1759491196.7932367,"caller":"embed/etcd.go:621","msg":"stopping serving peer traffic","component":"server","component":"embedded_etcd","address":"127.0.0.1:37367"}
omni  | {"level":"error","ts":1759491196.7935839,"caller":"embed/etcd.go:912","msg":"setting up serving from embedded etcd failed.","component":"server","component":"embedded_etcd","error":"accept tcp 127.0.0.1:37367: use of closed network connection"}
omni  | {"level":"info","ts":1759491196.793966,"caller":"embed/etcd.go:626","msg":"stopped serving peer traffic","component":"server","component":"embedded_etcd","address":"127.0.0.1:37367"}
omni  | {"level":"info","ts":1759491196.7941427,"caller":"embed/etcd.go:428","msg":"closed etcd server","component":"server","component":"embedded_etcd","name":"default","data-dir":"_out/etcd/","advertise-peer-urls":["http://localhost:2380"],"advertise-client-urls":["http://localhost:2379"]}
omni  | Error: failed to run server: failed to run discovery server over Siderolink: failed to start discovery service: 2 error(s) occurred:
omni  |         failed to listen: listen tcp [fdae:41e4:649b:9303::1]:8093: bind: cannot assign requested address
omni  |         timeout
omni  | 2025/10/03 11:33:16 failed to run Omni: failed to run server: failed to run discovery server over Siderolink: failed to start discovery service: 2 error(s) occurred:
omni  |         failed to listen: listen tcp [fdae:41e4:649b:9303::1]:8093: bind: cannot assign requested address
omni  |         timeout
omni exited with code 1

Config - variables get replaced for me:

  OMNI_IMG_TAG=latest
  OMNI_ACCOUNT_UUID=36267e53-f04c-431a-acc8-05e0362a10ae
  NAME=omni
  EVENT_SINK_PORT=8091
  
  ## Keys and Certs
  TLS_CERT=/docker/appdata/lego/certificates/_.fossindia.ovh.crt
  TLS_KEY=/docker/appdata/lego/certificates/_.fossindia.ovh.key
  ETCD_VOLUME_PATH=/docker/appdata/omni/etcd
  ETCD_ENCRYPTION_KEY=/docker/appdata/omni/omni.asc
  
  ## Binding
  BIND_ADDR=0.0.0.0:443
  MACHINE_API_BIND_ADDR=0.0.0.0:8090
  K8S_PROXY_BIND_ADDR=0.0.0.0:8100
  
  ## Domains and Advertisements
  OMNI_DOMAIN_NAME="${OMNI_DOMAIN_NAME}"
  ADVERTISED_API_URL="https://${OMNI_DOMAIN_NAME}"
  SIDEROLINK_ADVERTISED_API_URL="https://api.${OMNI_DOMAIN_NAME}:8090/"
  ADVERTISED_K8S_PROXY_URL="https://kube.${OMNI_DOMAIN_NAME}:8100/"
  SIDEROLINK_WIREGUARD_ADVERTISED_ADDR="${OMNI_WG_IP}:50180"
  
  ## Users
  INITIAL_USER_EMAILS="${OMNI_ADMIN_EMAIL}"
  
  ## Authentication
  #Auth0
  AUTH='--auth-auth0-enabled=true --auth-auth0-domain=${AUTH0_DOMAIN} --auth-auth0-client-id=${AUTH0_CLIENT_ID}'

I am using default compose.yaml & above omni.env. If I run this command in the VM then restart the container, it works.

sudo ip -6 addr add fdae:41e4:649b:9303::1/64 dev eth0

My question do I need to create a subnet for fdae:41e4:649b:9303::/64 on my router?

How to fix?

[smira]

Most probably the real error is above in the log, look for Wireguard/SideroLink errors - my guess it fails to start it.

Please never assign such addresses on the host.

[navaneeth-dev]

Exact config, I just removed certificates to test on Debian, and it works but not in Flatcar. There are no other errors in logs.

I see this on Debian.

6: siderolink: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 qdisc fq_codel state UNKNOWN group default qlen 500
    link/none 
    inet6 fdae:41e4:649b:9303::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::839e:a177:19a9:3a30/64 scope link stable-privacy 
       valid_lft forever preferred_lft forever

[smira]

I don't have much to add here, SideroLink doesn't run, why it's the case I can't guess. you need to find the error in the logs.

[navaneeth-dev]

FIXED: For some reason flatcar creates this systemd-networkd config which blocks any config on the wireguard interface.

core@omni ~ $ sudo cat /usr/lib/systemd/network/zz-default.network
[Network]
DHCP=yes
KeepConfiguration=dhcp-on-stop
IPv6AcceptRA=true

[Match]
Name=*
Type=!loopback bridge tunnel vxlan wireguard
Driver=!veth dummy

[DHCP]
UseMTU=true
UseDomains=true

[DHCPv4]
RoutesToDNS=false

Fix:

files:
    - path: /etc/systemd/network/10-siderolink.network
      mode: 0644
      contents:
        inline: |
          [Match]
          Name=siderolink
          
          [Link]
          Unmanaged=yes

Thanks for your time. Is the docs open source? I would like to update it with many fixes.

[smira]

Yes, docs are here: https://github.com/siderolabs/docs/tree/main/public/omni