command execution timed out in go-client when Docker container is restarted #405
Description
Hello,
I have deployed aerospike:ce-6.3.0.1 under docker-compose with the aerospike.conf
service {
}
logging {
file /var/log/aerospike/aerospike.log {
context any info
}
console {
context any info
}
}
network {
service {
address any
port 3000
}
heartbeat {
mode mesh
address local
port 3002
interval 150
timeout 10
}
fabric {
address local
port 3001
}
}
namespace cache {
default-ttl 30d # use 0 to never expire/evict.
memory-size 4G
nsup-period 120
replication-factor 1
storage-engine memory
}
I'm using go-client v6.12.0 in my go aplication also deployed as docker container in the same docker network.
When my go application container restarts due to redeployment I start getting the following error on go-client side on just creating a new client connection
clientPolicy := aero.NewClientPolicy()
client, err := aero.NewClientWithPolicy(clientPolicy, cfg.Host, cfg.Port)
if err != nil {
return ErrAeroConn(err, ctx)
}
client logs look like this
2023-06-11T17:04:55.930+0000 Alias 172.18.0.3:3000 failed: ResultCode: NETWORK_ERROR, Iteration: 0, InDoubt: false, Node: <nil>: network error. Checked the wrapped error for detail
ResultCode: TIMEOUT, Iteration: 0, InDoubt: false, Node: <nil>: Timeout
read tcp 172.18.0.30:55270->172.18.0.3:3000: i/o timeout
2023-06-11T17:04:55.930+0000 Seed 172.18.0.3:3000 failed: ResultCode: NETWORK_ERROR, Iteration: 0, InDoubt: false, Node: <nil>: network error. Checked the wrapped error for detail
ResultCode: TIMEOUT, Iteration: 0, InDoubt: false, Node: <nil>: Timeout
read tcp 172.18.0.30:55270->172.18.0.3:3000: i/o timeout
2023-06-11T17:04:55.930+0000 ResultCode: INVALID_NODE_ERROR, Iteration: 0, InDoubt: false, Node: <nil>: Failed to connect to hosts: [172.18.0.3:3000]
ResultCode: NETWORK_ERROR, Iteration: 0, InDoubt: false, Node: <nil>: network error. Checked the wrapped error for detail
ResultCode: TIMEOUT, Iteration: 0, InDoubt: false, Node: <nil>: Timeout
read tcp 172.18.0.30:55270->172.18.0.3:3000: i/o timeout
2023-06-11T17:04:55.931+0000 No nodes available; seeding...
2023-06-11T17:04:55.932+0000 Seeding the cluster. Seeds count: 1
2023-06-11T17:04:55.932+0000 Node Validator has 1 nodes and they are: [172.18.0.3:3000]
2023-06-11T17:05:10.922+0000 Failed to connect to host(s): [aerospike:3000]; error: ResultCode: TIMEOUT, Iteration: 0, InDoubt: false, Node: <nil>: command execution timed out on client: See
ResultCode: TIMEOUT, Iteration: 0, InDoubt: false, Node: <nil>: command execution timed out on client: See Policy.Timeout
Error is reproduced until aerospike container is restarted.