[BUG] sometimes' pod always being status of ContainterCreating when set inactivityTimeout

[changluyi]

Kube-OVN Version

1.12.25

Kubernetes Version

1.30

Operation-system/Kernel Version

3.10.0-1160.el7.x86_64

Description

kube-ovn-controller get a log like this， the pod gets the allocated IP, and then hang, the handleAddOrUpdatePod function will never exist

Steps To Reproduce

I don't know how to reproduce

Current Behavior

pod is always being in status of ContainerCreating

Expected Behavior

pod is created successfully

[dosubot]

I found related issues that might be helpful:

• kube-ovn-pinger is stuck on ContainerCreating state #2859
  The suggested solution for this issue was to reinstall everything from scratch. The user reported that after reinstalling, the problem was resolved, and the kube-ovn-pinger was no longer stuck in the ContainerCreating state.

To continue talking to Dosu, mention @dosu.

[bobz965]

是不是什么锁竞争导致的呀？

[changluyi]

目前追溯到是inactivityProbe开了后，这个锁hang住了

[changluyi]

目前追溯到rpclient的问题，在这两个位置都能hang住，导致rpcMutex没有释放，尝试更新rpc库没有解决，目前提issue给libovsdb社区。

ovn-org/libovsdb#391

看到这个pr 把echo 写成异步的，似乎能解决，但没弄明白当时为啥没合并
cenkalti/rpc2#38
ovn-org/libovsdb#368