High I/O wait on linux

[p5n]

Steps to reproduce: just start shadow-tls config -c /etc/shadow-tls/server.json and htop shows that 3 cpu cores are busy with I/O wait.

Config:

{
    "disable_nodelay": false,
    "fastopen": true,
    "v3": true,
    "strict": true,
    "server": {
        "listen": "0.0.0.0:5222",
        "server_addr": "127.0.0.1:5223",
        "tls_addr": {
            "wildcard_sni": "off",
            "dispatch": {
                "cloudflare.com": "1.1.1.1:443",
                "captive.apple.com": "captive.apple.com:443"
            },
            "fallback": "cloud.tencent.com:443"
        },
        "password": "*********",
        "wildcard_sni": "authed"
    }
}

[jorshua2784]

{
"disable_nodelay": false,
"fastopen": true,
"v3": true,
"strict": true,
"server": {
"listen": "0.0.0.0:5222",
"server_addr": "127.0.0.1:5223",
"tls_addr": {
"wildcard_sni": "off",
"dispatch": {
"cloudflare.com": "1.1.1.1:443",
"captive.apple.com": "captive.apple.com:443"
},
"fallback": "cloud.tencent.com:443"
},
"password": "*********",
"wildcard_sni": "authed"
}
}

[jorshua2784]

{ "disable_nodelay": false, "fastopen": true, "v3": true, "strict": true, "server": { "listen": "0.0.0.0:5222", "server_addr": "127.0.0.1:5223", "tls_addr": { "wildcard_sni": "off", "dispatch": { "cloudflare.com": "1.1.1.1:443", "captive.apple.com": "captive.apple.com:443" }, "fallback": "cloud.tencent.com:443" }, "password": "*********", "wildcard_sni": "authed" } }

[li12138]

我今天也遇到一样的情况,换域名可以解决

[ihciah]

Usually the network IO does not cause IO wait.
Please try to check if the reason is too much error log. For example, you can read the log, or change the log level to off and see if it would work fine.

[p5n]

There is no error messages

strace shows just waiting

# strace -p 1304169
strace: Process 1304169 attached
futex(0x7c37328ff990, FUTEX_WAIT_BITSET|FUTEX_CLOCK_REALTIME, 1304176, NULL, FUTEX_BITSET_MATCH_ANY

it is strange but running /usr/bin/shadow-tls config -c /etc/shadow-tls/config.json causes 100% i/o wait on 1 cpu

[love4taylor]

Higher version kernels combined with io_uring usage lead to high iowait, see https://bbs.archlinux.org/viewtopic.php?id=287343

You need to switch back to epoll by using MONOIO_FORCE_LEGACY_DRIVER=1, this may not be the root cause, but it is indeed effective.