ipv6 vip can not use out of k8s

[xiejun946]

k8s :1.21.10
docker :20.10.12
kernel: 5.4.182-1.el7.elrepo.x86_64
image :kube-vip-cloud-provider:v0.0.3

this is master machine

[root@hybxvuca01 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.152.194.149 harbor.uat.crpcg.com
10.153.184.109 api.k8s.local
10.153.184.98 devops-db.bx.crpcg.com
10.153.184.98 devops-db.bx.crpharm.com
10.153.184.106 hybxvuca01.crpcg.com
2406:440:5400:80:0:70:0:1005 hybxvuca01.crpcg.com
2406:440:5400:80:0:70:0:1004 hybxvuca02.crpcg.com
10.153.184.107 hybxvuca02.crpcg.com
10.153.184.102 apollo.uat.crpcg.com
10.153.184.98 devops-db.bx.crpcg.com
10.153.184.98 devops-db.bx.crpharm.com
2406:440:5400:80:0:70:0:1062 demo.k8s.com
2406:440:5400:80:0:70:0:1062 nginx-demo.uat.crpcg.com
[root@hybxvuca01 ~]# ip a s ens224
3: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:50:56:81:42:ec brd ff:ff:ff:ff:ff:ff
inet6 2406:440:5400:80:0:70:0:1005/116 scope global noprefixroute
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe81:42ec/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@hybxvuca01 ~]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.101.164.119 10.153.184.104 80:31517/TCP,443:31726/TCP 21d
ingress-nginx-controller-admission ClusterIP 10.106.121.95 443/TCP 21d
ingress-nginx-controller-ipv6 LoadBalancer fd00::ecf5 2406:440:5400:80:0:70:0:1062 80:30290/TCP,443:30168/TCP 3d20h
[root@hybxvuca01 ~]# curl -6 demo.k8s.com

<title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style>

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and working. Further configuration is required.

For online documentation and support please refer to nginx.org.
Commercial support is available at nginx.com.

Thank you for using nginx.

this is same network macheine

10.153.184.102 apollo.uat.crpcg.com
10.153.184.102 apollo-config.uat.crpcg.com
2406:440:5400:80:0:70:0:1062 demo.k8s.com
[root@hybxvuca01 ~]# ip a s ens224
48: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:81:b3:32 brd ff:ff:ff:ff:ff:ff
inet6 2406:440:5400:80:0:70:0:1050/116 scope global noprefixroute
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe81:b332/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@hybxvuca01 ~]# ping6 2406:440:5400:80:0:70:0:1062

can not ping this ip

[xiejun946]

i try to use vip on nginx-ingress，but another machine can not ping the ipv6 vip address,so ithink there are problem on kube-vip-provider。i seem that the vip bond in kube-ipvs0 interface
root@hybxvuca01 kube-vip-cloud-provider-0.0.7]# ip a s kube-ipvs0
7: kube-ipvs0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default
link/ether 02:3f:6a:ff:81:b3 brd ff:ff:ff:ff:ff:ff
inet 10.96.0.1/32 scope global kube-ipvs0
valid_lft forever preferred_lft forever
inet 10.96.0.10/32 scope global kube-ipvs0
valid_lft forever preferred_lft forever
inet 10.101.164.119/32 scope global kube-ipvs0
valid_lft forever preferred_lft forever
inet 10.153.184.104/32 scope global kube-ipvs0
valid_lft forever preferred_lft forever
inet 10.106.121.95/32 scope global kube-ipvs0
valid_lft forever preferred_lft forever
inet 10.105.80.77/32 scope global kube-ipvs0
valid_lft forever preferred_lft forever
inet6 fd00::74a2/128 scope global
valid_lft forever preferred_lft forever
inet6 2406:440:5400:80:0:70:0:1062/128 scope global
valid_lft forever preferred_lft forever
inet6 fd00::acc5/128 scope global
valid_lft forever preferred_lft forever

[xiejun946]

[root@hybxvuca01 ~]# kubectl get cm -n kube-system kubevip -o yaml
apiVersion: v1
data:
cidr-ipv6: 2406:440:5400:80:0:70:0:1064/116
range-global: 10.153.184.104
kind: ConfigMap
metadata:
creationTimestamp: "2023-11-20T08:46:10Z"
name: kubevip
namespace: kube-system
resourceVersion: "8322860"
uid: 47192de2-a244-4b8f-800b-0dca68be492b

this is my configmap of the kubevip

[root@hybxvuca01 ~]# ip a s ens192
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:81:d9:18 brd ff:ff:ff:ff:ff:ff
inet 10.153.184.106/24 brd 10.153.184.255 scope global noprefixroute ens192
valid_lft forever preferred_lft forever
inet 10.153.184.109/32 scope global ens192
valid_lft forever preferred_lft forever
inet 10.153.184.104/32 scope global ens192
valid_lft forever preferred_lft forever
inet6 2406:440:5400:80:0:70:0:1064/128 scope global
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe81:d918/64 scope link
valid_lft forever preferred_lft forever

ithink the problem occur on the ipv4 gateway cannot not connection to d ipv6 gateway
the vip bond to ens192,this is eth ipv4 interface,but another machine connect to eth ipv6 address through the ipv6 gateway

[xiejun946]

i seem that Support single stack IPv6 or IPv4, can not work on double stack ，beacase 的ipv6 vip bond on eth error interface ,if it bond on the ipv6 interface ,it can work

[xiejun946]

i try to use 2 service between ipv4 and ipv6,but the two service share the same configmap!