Can't start kvrocks in Docker container on macos.

[ruslangm]

Search before asking

• I had searched in the issues and found no similar issues.

Version

MacOS version: 12.6, MacBook Pro Apple M1 Max
Docker desktop version: 4.12.0 (85629)
kvrocks version: latest from Docker hub
https://hub.docker.com/layers/apache/kvrocks/latest/images/sha256-9ef819b07e071f03125846c08e01ee4651710f2ee70518987fbaf6944a6e5b62?context=explore

Minimal reproduce step

❯ docker run -it -p 6666:6666 apache/kvrocks
<jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
<jemalloc>: (This is the expected behaviour if you are running under QEMU)

❯ docker ps -a
CONTAINER ID   IMAGE            COMMAND                  CREATED              STATUS                            PORTS                    NAMES
2c5f0f887f81   apache/kvrocks   "./bin/kvrocks -c ./…"   About a minute ago   Exited (137) About a minute ago                            

What did you expect to see?

kvrocks start log.

What did you see instead?

<jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
<jemalloc>: (This is the expected behaviour if you are running under QEMU)

Anything Else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[git-hulk]

I tried it on Mac Pro(intel) and M1 Pro, but they work well. I'm wondering if the CPU arch is different between M1 Pro and M1 Max. @ruslangm could you paste the output of the command uname -a? And it will print the information looks like below:

Darwin hulks-MBP 21.4.0 Darwin Kernel Version 21.4.0: Mon Feb 21 20:35:58 PST 2022; root:xnu-8020.101.4~2/RELEASE_ARM64_T6000 arm64

[ruslangm]

@git-hulk btw, I just built and ran kvrocks from sources according to the build section from doc. and it works good.

❯ uname -a
Darwin nd9w29p9vd 21.6.0 Darwin Kernel Version 21.6.0: Mon Aug 22 20:19:52 PDT 2022; root:xnu-8020.140.49~2/RELEASE_ARM64_T6000 arm64

[ruslangm]

nightly build provides one more line of error:

❯ docker run -it -p 6666:6666 --name apache/kvrocks:nightly
<jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
<jemalloc>: (This is the expected behaviour if you are running under QEMU)
qemu: uncaught target signal 4 (Illegal instruction) - core dumped

[git-hulk]

@ruslangm Thanks for your feedback. It looks like the docker client wrongly recognizes the CPU platform. Would you like to have a try at pulling the image with explicitly specify the platform arch: docker pull apache/kvrocks:2.2.0 --platform arm64

I can reproduce it on my M1 Pro if wrongly specified the platform:

❯ docker pull apache/kvrocks:2.2.0 --platform amd64
2.2.0: Pulling from apache/kvrocks
Digest: sha256:24dc33d8d2b69b1bfad073e63f13edc53bfa38235eb2f7320377ea7aa2691f49
Status: Downloaded newer image for apache/kvrocks:2.2.0
docker.io/apache/kvrocks:2.2.0
❯ docker run -it -p 6666:6666 apache/kvrocks:2.2.0
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
<jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
<jemalloc>: (This is the expected behaviour if you are running under QEMU)
``

[ruslangm]

@git-hulk sure.

❯ docker pull apache/kvrocks:2.2.0 --platform arm64
2.2.0: Pulling from apache/kvrocks
4e7e0215f4ad: Already exists
2b847dde0550: Pull complete
c81f793545a7: Pull complete
e371fbf3a5a8: Pull complete
3ce5b4e75401: Pull complete
9e36f583daf1: Pull complete
4f41328fbfd0: Pull complete
19c45c51368f: Pull complete
Digest: sha256:24dc33d8d2b69b1bfad073e63f13edc53bfa38235eb2f7320377ea7aa2691f49
Status: Downloaded newer image for apache/kvrocks:2.2.0
WARNING: image with reference apache/kvrocks was found but does not match the specified platform: wanted linux/arm64, actual: linux/amd64
docker.io/apache/kvrocks:2.2.0

❯ docker run -it -p 6666:6666 apache/kvrocks@sha256:24dc33d8d2b69b1bfad073e63f13edc53bfa38235eb2f7320377ea7aa2691f49
<jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
<jemalloc>: (This is the expected behaviour if you are running under QEMU)

[PragmaTwice]

@git-hulk sure.

❯ docker pull apache/kvrocks:2.2.0 --platform arm64
2.2.0: Pulling from apache/kvrocks
4e7e0215f4ad: Already exists
2b847dde0550: Pull complete
c81f793545a7: Pull complete
e371fbf3a5a8: Pull complete
3ce5b4e75401: Pull complete
9e36f583daf1: Pull complete
4f41328fbfd0: Pull complete
19c45c51368f: Pull complete
Digest: sha256:24dc33d8d2b69b1bfad073e63f13edc53bfa38235eb2f7320377ea7aa2691f49
Status: Downloaded newer image for apache/kvrocks:2.2.0
WARNING: image with reference apache/kvrocks was found but does not match the specified platform: wanted linux/arm64, actual: linux/amd64
docker.io/apache/kvrocks:2.2.0

❯ docker run -it -p 6666:6666 apache/kvrocks@sha256:24dc33d8d2b69b1bfad073e63f13edc53bfa38235eb2f7320377ea7aa2691f49
<jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
<jemalloc>: (This is the expected behaviour if you are running under QEMU)

I notice there is one line in your log that seems to be the reason of the error you encountered:

WARNING: image with reference apache/kvrocks was found but does not match the specified platform: wanted linux/arm64, actual: linux/amd64

[git-hulk]

That's so weird. Your docker's qemu is running in linux/amd64 but from my side is aarch64. So you got the error since the docker client thought its arch is arm64 and pulled the image with the arm64 platform.

❯ ps aux|grep qemu
hulk             13890   1.9  7.4 418260112 1247520   ??  S     8:28PM   3:30.06 /Applications/Docker.app/Contents/MacOS/qemu-system-aarch64 -accel hvf -cpu host -machine virt,highmem=off -m 8092 -smp 5 -kernel /Applications/Docker.app/Contents/Resources/linuxkit/kernel -append page_poison=1 vsyscall=emulate panic=1 nospec_store_bypass_disable noibrs noibpb no_stf_barrier mitigations=off linuxkit.unified_cgroup_hierarchy=1    vpnkit.connect=tcp+bootstrap+client://192.168.65.2:62364/bcb9f8f0f5a636d920d70e033a3333c9f4fe0807b19bb748bbd1561af926d878 vpnkit.disable=osxfs-data console=ttyAMA0 -initrd /Applications/Docker.app/Contents/Resources/linuxkit/initrd.img -serial pipe:/var/folders/ls/jsfyf9g52h94_tmgys0895qw0000gn/T/qemu-console1146358452/fifo -drive if=none,file=/Users/hulk/Library/Containers/com.docker.docker/Data/vms/0/data/Docker.raw,format=raw,id=hd0 -device virtio-blk-pci,drive=hd0,serial=dummyserial -netdev socket,id=net1,fd=3 -device virtio-net-device,netdev=net1,mac=02:50:00:00:00:01 -vga none -nographic -monitor none

[ruslangm]

@git-hulk can it be related to difference between Docker Desktop for Intel Chip and Docker Desktop for Apple Chip?

[git-hulk]

I also tested it on my intel chip, it also works well. You can take a look at qemu with ps aux|grep qemu. If I guess right, it should be running in x86_64 mode.

[ruslangm]

❯ ps aux | grep qemu
user             20177  13.8  1.2 418377584 387504   ??  R     7:38PM  13:39.97 /Applications/Docker.app/Contents/MacOS/qemu-system-aarch64 -accel hvf -cpu host ...

[git-hulk]

❯ ps aux | grep qemu
user             20177  13.8  1.2 418377584 387504   ??  R     7:38PM  13:39.97 /Applications/Docker.app/Contents/MacOS/qemu-system-aarch64 -accel hvf -cpu host ...

ummm... I have no idea why it happened, your QEMU is running in aarch64 but it thought itself needs the amd64 image:

was found but does not match the specified platform: wanted linux/arm64, actual: linux/amd64

[tisonkun]

Moved questions to the Q&A forum.