Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Formatting keep stuck at pulling image #282

Closed
mrdrivingduck opened this issue Aug 5, 2023 · 16 comments
Closed

Formatting keep stuck at pulling image #282

mrdrivingduck opened this issue Aug 5, 2023 · 16 comments
Labels
wait-for-reply

Comments

@mrdrivingduck
Copy link
Member

I'm trying to format my disks, but the progress seems not to move on. I'm using curveadm v0.3.0.

#curveadm format --status
Get Format Status: [OK]  

Host          Device    MountPoint          Formatted  Status       
----          ------    ----------          ---------  ------       
server-host1  /dev/osa  /data/chunkserver0   1/90      Pulling image
server-host1  /dev/osb  /data/chunkserver1   1/90      Pulling image
server-host1  /dev/osc  /data/chunkserver2   1/90      Pulling image
server-host2  /dev/osa  /data/chunkserver0   1/90      Pulling image
server-host2  /dev/osb  /data/chunkserver1   1/90      Pulling image
server-host2  /dev/osc  /data/chunkserver2   1/90      Pulling image
server-host3  /dev/osa  /data/chunkserver0   1/90      Pulling image
server-host3  /dev/osb  /data/chunkserver1   1/90      Pulling image
server-host3  /dev/osc  /data/chunkserver2   1/90      Pulling image

If there any way to rollback this status and format again? I'm sure there is no image on my machine now:

#docker image ls
REPOSITORY    TAG       IMAGE ID       CREATED        SIZE
@Cyber-SiKu
Copy link
Collaborator

Please follow the [Instructions for Asking Questions] (#271)
ask questions.

You can modify the logging level like this:

# ~/.curveadm/curveadm.cfg
[defaults]
log_level = info <---
 …

Then post your log (~/.curveadm/logs/xxx.log).
And please specify your docker version

@mrdrivingduck
Copy link
Member Author

mrdrivingduck commented Aug 10, 2023 •
edited
Loading

Log:

2023-08-10 02:06:11     [INFO]: (log.go:101):   Init logger success
    LogPath: /root/.curveadm/logs/curveadm-2023-08-10_10-06-11.log
    LogLevel: info

2023-08-10 02:06:11     [INFO]: (log.go:101):   Get current cluster success
    ClusterId: -1
    ClusterName: 

2023-08-10 02:06:11     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-5382e6eb967461d786f8744b3104e9b7 --all
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-9e939ab24ceaaf69708ec2fb956ba69d --all
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-f41da9000e5270a4930662b848d85aaa --all
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-9e939ab24ceaaf69708ec2fb956ba69d --all
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-9e939ab24ceaaf69708ec2fb956ba69d --all
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-f41da9000e5270a4930662b848d85aaa --all
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-5382e6eb967461d786f8744b3104e9b7 --all
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-f41da9000e5270a4930662b848d85aaa --all
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-5382e6eb967461d786f8744b3104e9b7 --all
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osa
    output: 0253e273-ab0b-4c56-a46a-bb6b05f34a46
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osb
    output: 79dc11e8-cd9c-4286-8e6e-1d556a6408fc
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osa
    output: fa00f065-5df4-4928-9c46-f4918ea71508
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osc
    output: 65a8f22d-e4e4-493b-b19d-74736bd675b3
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osa
    output: f6eae9b2-d23f-48c0-9ab1-582431403245
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osc
    output: 6de0ba97-16c7-4ec5-9995-5b90f7e91d6d
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osb
    output: 190dcd8c-ce49-4c63-8400-9f1e5fbcfc85
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osb
    output: b224104e-6cb8-4b4c-a1b3-132492e82427
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osc
    output: 54a04b35-6802-4170-b536-ccaaf048d281
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osa
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osa
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osa
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osc
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osb
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osc
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osb
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osc
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osb
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver0
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver0
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver0
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver2
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver1
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver2
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver1
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver2
    output: 
    error: <nil>

2023-08-10 02:06:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver1
    output: 
    error: <nil>

Docker:

$ docker -v
Docker version 24.0.5, build ced0996

@mrdrivingduck
Copy link
Member Author

OS Kernel: 4.19.91

@Cyber-SiKu
Copy link
Collaborator

@mrdrivingduck Please be careful to hide your sensitive information (ip) and the like.

@caoxianfei1
Copy link
Contributor

@mrdrivingduck agjust the debug mode and retry.

@mrdrivingduck
Copy link
Member Author

@Cyber-SiKu Thanks for your advice, I have wiped out the information.

@caoxianfei1 Here is the result. Seems there are fewer lines? 🤔

2023-08-10 05:32:53     [INFO]: (log.go:101):   Init logger success
    LogPath: /root/.curveadm/logs/curveadm-2023-08-10_13-32-53.log
    LogLevel: debug

2023-08-10 05:32:53     [INFO]: (log.go:101):   Get current cluster success
    ClusterId: -1
    ClusterName: 

2023-08-10 05:32:53     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo df --output=pcent /dev/osa
    output: Use%
  1%
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo df --output=pcent /dev/osc
    output: Use%
  1%
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo df --output=pcent /dev/osa
    output: Use%
  1%
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo df --output=pcent /dev/osb
    output: Use%
  1%
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo df --output=pcent /dev/osc
    output: Use%
  1%
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo df --output=pcent /dev/osb
    output: Use%
  1%
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo df --output=pcent /dev/osa
    output: Use%
  1%
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo df --output=pcent /dev/osb
    output: Use%
  1%
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo df --output=pcent /dev/osc
    output: Use%
  1%
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Status}}' --filter name=curvebs-format-5382e6eb967461d786f8744b3104e9b7 --all
    output: 
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Status}}' --filter name=curvebs-format-9e939ab24ceaaf69708ec2fb956ba69d --all
    output: 
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Status}}' --filter name=curvebs-format-f41da9000e5270a4930662b848d85aaa --all
    output: 
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Status}}' --filter name=curvebs-format-9e939ab24ceaaf69708ec2fb956ba69d --all
    output: 
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Status}}' --filter name=curvebs-format-9e939ab24ceaaf69708ec2fb956ba69d --all
    output: 
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Status}}' --filter name=curvebs-format-5382e6eb967461d786f8744b3104e9b7 --all
    output: 
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Status}}' --filter name=curvebs-format-f41da9000e5270a4930662b848d85aaa --all
    output: 
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Status}}' --filter name=curvebs-format-5382e6eb967461d786f8744b3104e9b7 --all
    output: 
    error: <nil>

2023-08-10 05:32:53     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Status}}' --filter name=curvebs-format-f41da9000e5270a4930662b848d85aaa --all
    output: 
    error: <nil>

@mrdrivingduck
Copy link
Member Author

I downgrade Docker and manually pull the image, and the format command proceeds and returns. Now it seems to be stuck at mounting:

2023-08-11 07:42:01     [INFO]: (log.go:101):   Init logger success
    LogPath: /root/.curveadm/logs/curveadm-2023-08-11_15-42-01.log
    LogLevel: debug

2023-08-11 07:42:01     [INFO]: (log.go:101):   Get current cluster success
    ClusterId: -1
    ClusterName: 

2023-08-11 07:42:01     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Connect remote SSH
    user: root
    host: *
    port: 22
    forwardAgent: false
    privateKeyPath: /root/.ssh/id_rsa
    timeoutSec: 10
    maxRetries: 3
    tries: 1
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-5382e6eb967461d786f8744b3104e9b7 --all
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-9e939ab24ceaaf69708ec2fb956ba69d --all
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-9e939ab24ceaaf69708ec2fb956ba69d --all
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-f41da9000e5270a4930662b848d85aaa --all
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-f41da9000e5270a4930662b848d85aaa --all
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-5382e6eb967461d786f8744b3104e9b7 --all
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-5382e6eb967461d786f8744b3104e9b7 --all
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-9e939ab24ceaaf69708ec2fb956ba69d --all
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker ps --format '{{.Names}}' --filter name=curvebs-format-f41da9000e5270a4930662b848d85aaa --all
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osa
    output: d4e2d0f1-6ae2-4907-885e-07f609674a41
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osa
    output: 7ae73c9c-a31a-43f6-8c3b-d4a35eaf9b9e
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osb
    output: d29ba228-4545-4e17-aa1b-70b37c4714e3
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osb
    output: e6ebdcc8-912c-46b7-8e14-23c7eab453dd
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osc
    output: 543403c4-1a8c-4543-9336-9c88ddbeebb8
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osc
    output: e2de31f3-70a3-48fc-93bd-9c9e003874de
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osc
    output: e862665f-17e9-494a-b8f9-54f3b9dd9f27
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osa
    output: 0557ac90-a627-48fb-91c7-cd6506ea93ad
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osb
    output: ad9a61ec-738b-49b6-a984-3a3995d0edbc
    error: <nil>

2023-08-11 07:42:01     [ERR]:  (log.go:109):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osa
    output: umount: /dev/osa: not mounted
    error: Process exited with status 32

2023-08-11 07:42:01     [ERR]:  (log.go:109):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osb
    output: umount: /dev/osb: not mounted
    error: Process exited with status 32

2023-08-11 07:42:01     [ERR]:  (log.go:109):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osa
    output: umount: /dev/osa: not mounted
    error: Process exited with status 32

2023-08-11 07:42:01     [ERR]:  (log.go:109):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osb
    output: umount: /dev/osb: not mounted
    error: Process exited with status 32

2023-08-11 07:42:01     [ERR]:  (log.go:109):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osc
    output: umount: /dev/osc: not mounted
    error: Process exited with status 32

2023-08-11 07:42:01     [ERR]:  (log.go:109):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osa
    output: umount: /dev/osa: not mounted
    error: Process exited with status 32

2023-08-11 07:42:01     [ERR]:  (log.go:109):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osc
    output: umount: /dev/osc: not mounted
    error: Process exited with status 32

2023-08-11 07:42:01     [ERR]:  (log.go:109):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osc
    output: umount: /dev/osc: not mounted
    error: Process exited with status 32

2023-08-11 07:42:01     [ERR]:  (log.go:109):   Execute command
    remoteAddr: root@*:22
    command: sudo umount  /dev/osb
    output: umount: /dev/osb: not mounted
    error: Process exited with status 32

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver0
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver1
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver0
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver1
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver2
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver2
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver1
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver0
    output: 
    error: <nil>

2023-08-11 07:42:01     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkdir --parents /data/chunkserver2
    output: 
    error: <nil>

2023-08-11 07:42:08     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkfs.ext4 -F /dev/osa
    output: mke2fs 1.43.5 (04-Aug-2017)
Discarding device blocks: done                            
Creating filesystem with 937684566 4k blocks and 234422272 inodes
Filesystem UUID: fab8da31-7932-4e92-b3da-6b206870c394
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
        102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done       

    error: <nil>

2023-08-11 07:42:08     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mount  /dev/osa /data/chunkserver0
    output: 
    error: <nil>

2023-08-11 07:42:08     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo cp --no-clobber /etc/fstab /etc/fstab-2023-08-11.backup
    output: 
    error: <nil>

2023-08-11 07:42:08     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osa
    output: fab8da31-7932-4e92-b3da-6b206870c394
    error: <nil>

2023-08-11 07:42:08     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='/UUID=d4e2d0f1-6ae2-4907-885e-07f609674a41/d' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:08     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='$ a UUID=fab8da31-7932-4e92-b3da-6b206870c394  /data/chunkserver0  ext4  rw,errors=remount-ro  0  0  # GENERATED BY CURVEADM, DONT EDIT THIS' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:08     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo tune2fs -m 0 /dev/osa
    output: tune2fs 1.43.5 (04-Aug-2017)
Setting reserved blocks percentage to 0% (0 blocks)
    error: <nil>

2023-08-11 07:42:11     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkfs.ext4 -F /dev/osb
    output: mke2fs 1.43.5 (04-Aug-2017)
Discarding device blocks: done                            
Creating filesystem with 937684566 4k blocks and 234422272 inodes
Filesystem UUID: 14b8efe5-6a28-4af7-a470-2c00680c8eac
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
        102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done       

    error: <nil>

2023-08-11 07:42:12     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mount  /dev/osb /data/chunkserver1
    output: 
    error: <nil>

2023-08-11 07:42:12     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo cp --no-clobber /etc/fstab /etc/fstab-2023-08-11.backup
    output: 
    error: <nil>

2023-08-11 07:42:12     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osb
    output: 14b8efe5-6a28-4af7-a470-2c00680c8eac
    error: <nil>

2023-08-11 07:42:12     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='/UUID=e6ebdcc8-912c-46b7-8e14-23c7eab453dd/d' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:12     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='$ a UUID=14b8efe5-6a28-4af7-a470-2c00680c8eac  /data/chunkserver1  ext4  rw,errors=remount-ro  0  0  # GENERATED BY CURVEADM, DONT EDIT THIS' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:12     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo tune2fs -m 0 /dev/osb
    output: tune2fs 1.43.5 (04-Aug-2017)
Setting reserved blocks percentage to 0% (0 blocks)
Recovering journal.
    error: <nil>

2023-08-11 07:42:12     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkfs.ext4 -F /dev/osb
    output: mke2fs 1.43.5 (04-Aug-2017)
Discarding device blocks: done                            
Creating filesystem with 937684566 4k blocks and 234422272 inodes
Filesystem UUID: e24b3de1-294e-4bbc-97e8-e2f5ed9f6332
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
        102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done       

    error: <nil>

2023-08-11 07:42:12     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mount  /dev/osb /data/chunkserver1
    output: 
    error: <nil>

2023-08-11 07:42:12     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo cp --no-clobber /etc/fstab /etc/fstab-2023-08-11.backup
    output: 
    error: <nil>

2023-08-11 07:42:12     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osb
    output: e24b3de1-294e-4bbc-97e8-e2f5ed9f6332
    error: <nil>

2023-08-11 07:42:12     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='/UUID=d29ba228-4545-4e17-aa1b-70b37c4714e3/d' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:12     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='$ a UUID=e24b3de1-294e-4bbc-97e8-e2f5ed9f6332  /data/chunkserver1  ext4  rw,errors=remount-ro  0  0  # GENERATED BY CURVEADM, DONT EDIT THIS' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:12     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo tune2fs -m 0 /dev/osb
    output: tune2fs 1.43.5 (04-Aug-2017)
Setting reserved blocks percentage to 0% (0 blocks)
Recovering journal.
    error: <nil>

2023-08-11 07:42:13     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker pull  opencurvedocker/curvebs:v1.2
    output: v1.2: Pulling from opencurvedocker/curvebs
Digest: sha256:8e32478c321069c0b5c089f58adcef7c22f53399981499e7b3d5c10846f12977
Status: Image is up to date for opencurvedocker/curvebs:v1.2
    error: <nil>

2023-08-11 07:42:13     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker create --entrypoint /bin/bash --name curvebs-format-9e939ab24ceaaf69708ec2fb956ba69d --network host --rm --volume /data/chunkserver0:/curvebs/chunkserver/data opencurvedocker/curvebs:v1.2 /curvebs/tools/sbin/format.sh /curvebs/tools/sbin/curve_format 90 16777216 /curvebs/chunkserver/data/chunkfilepool /curvebs/chunkserver/data/chunkfilepool.meta 4096
    output: fb210aff283d773a7b1db78d12f7040dcfd6038aa04bfd673886b557f9f4326e
    error: <nil>

2023-08-11 07:42:13     [INFO]: (log.go:101):   UploadFile
    remoteAddress: root@*:22
    localPath: /tmp/CFEEo9M1
    remotePath: /tmp/JlalHFdZ
    error: <nil>

2023-08-11 07:42:14     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkfs.ext4 -F /dev/osa
    output: mke2fs 1.43.5 (04-Aug-2017)
Discarding device blocks: done                            
Creating filesystem with 937684566 4k blocks and 234422272 inodes
Filesystem UUID: bad6b401-5768-48a6-9502-90667a38fe2f
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
        102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done       

    error: <nil>

2023-08-11 07:42:14     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mount  /dev/osa /data/chunkserver0
    output: 
    error: <nil>

2023-08-11 07:42:14     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo cp --no-clobber /etc/fstab /etc/fstab-2023-08-11.backup
    output: 
    error: <nil>

2023-08-11 07:42:14     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osa
    output: bad6b401-5768-48a6-9502-90667a38fe2f
    error: <nil>

2023-08-11 07:42:14     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='/UUID=0557ac90-a627-48fb-91c7-cd6506ea93ad/d' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:14     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='$ a UUID=bad6b401-5768-48a6-9502-90667a38fe2f  /data/chunkserver0  ext4  rw,errors=remount-ro  0  0  # GENERATED BY CURVEADM, DONT EDIT THIS' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:14     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkfs.ext4 -F /dev/osa
    output: mke2fs 1.43.5 (04-Aug-2017)
Discarding device blocks: done                            
Creating filesystem with 937684566 4k blocks and 234422272 inodes
Filesystem UUID: f04bf2fd-54ae-4685-9347-a87b30c71e72
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
        102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done       

    error: <nil>

2023-08-11 07:42:14     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo tune2fs -m 0 /dev/osa
    output: tune2fs 1.43.5 (04-Aug-2017)
Setting reserved blocks percentage to 0% (0 blocks)
    error: <nil>

2023-08-11 07:42:14     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mount  /dev/osa /data/chunkserver0
    output: 
    error: <nil>

2023-08-11 07:42:14     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo cp --no-clobber /etc/fstab /etc/fstab-2023-08-11.backup
    output: 
    error: <nil>

2023-08-11 07:42:14     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osa
    output: f04bf2fd-54ae-4685-9347-a87b30c71e72
    error: <nil>

2023-08-11 07:42:14     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkfs.ext4 -F /dev/osb
    output: mke2fs 1.43.5 (04-Aug-2017)
Discarding device blocks: done                            
Creating filesystem with 937684566 4k blocks and 234422272 inodes
Filesystem UUID: 1c7f0f55-5bef-473e-8c27-ab76a7089831
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
        102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done       

    error: <nil>

2023-08-11 07:42:15     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='/UUID=7ae73c9c-a31a-43f6-8c3b-d4a35eaf9b9e/d' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:15     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='$ a UUID=f04bf2fd-54ae-4685-9347-a87b30c71e72  /data/chunkserver0  ext4  rw,errors=remount-ro  0  0  # GENERATED BY CURVEADM, DONT EDIT THIS' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:15     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mount  /dev/osb /data/chunkserver1
    output: 
    error: <nil>

2023-08-11 07:42:15     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo tune2fs -m 0 /dev/osa
    output: tune2fs 1.43.5 (04-Aug-2017)
Setting reserved blocks percentage to 0% (0 blocks)
    error: <nil>

2023-08-11 07:42:15     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo cp --no-clobber /etc/fstab /etc/fstab-2023-08-11.backup
    output: 
    error: <nil>

2023-08-11 07:42:15     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osb
    output: 1c7f0f55-5bef-473e-8c27-ab76a7089831
    error: <nil>

2023-08-11 07:42:15     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='/UUID=ad9a61ec-738b-49b6-a984-3a3995d0edbc/d' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:15     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker cp   /tmp/JlalHFdZ fb210aff283d773a7b1db78d12f7040dcfd6038aa04bfd673886b557f9f4326e:/curvebs/tools/sbin/format.sh
    output: 
    error: <nil>

2023-08-11 07:42:15     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='$ a UUID=1c7f0f55-5bef-473e-8c27-ab76a7089831  /data/chunkserver1  ext4  rw,errors=remount-ro  0  0  # GENERATED BY CURVEADM, DONT EDIT THIS' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:15     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo tune2fs -m 0 /dev/osb
    output: tune2fs 1.43.5 (04-Aug-2017)
Setting reserved blocks percentage to 0% (0 blocks)
    error: <nil>

2023-08-11 07:42:16     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker start  fb210aff283d773a7b1db78d12f7040dcfd6038aa04bfd673886b557f9f4326e
    output: fb210aff283d773a7b1db78d12f7040dcfd6038aa04bfd673886b557f9f4326e
    error: <nil>

2023-08-11 07:42:16     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkfs.ext4 -F /dev/osc
    output: mke2fs 1.43.5 (04-Aug-2017)
Discarding device blocks: done                            
Creating filesystem with 937684566 4k blocks and 234422272 inodes
Filesystem UUID: 53bcd332-7fe5-4c05-9868-d2b0bd4c2aa4
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
        102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done       

    error: <nil>

2023-08-11 07:42:16     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkfs.ext4 -F /dev/osc
    output: mke2fs 1.43.5 (04-Aug-2017)
Discarding device blocks: done                            
Creating filesystem with 937684566 4k blocks and 234422272 inodes
Filesystem UUID: f64c015f-0ec3-400d-97cb-a3d78307346e
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
        102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done       

    error: <nil>

2023-08-11 07:42:16     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mount  /dev/osc /data/chunkserver2
    output: 
    error: <nil>

2023-08-11 07:42:16     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mkfs.ext4 -F /dev/osc
    output: mke2fs 1.43.5 (04-Aug-2017)
Discarding device blocks: done                            
Creating filesystem with 937684566 4k blocks and 234422272 inodes
Filesystem UUID: 5a1e44e4-9c44-451d-873f-d5bc5c196625
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
        102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done       

    error: <nil>

2023-08-11 07:42:16     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo cp --no-clobber /etc/fstab /etc/fstab-2023-08-11.backup
    output: 
    error: <nil>

2023-08-11 07:42:16     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mount  /dev/osc /data/chunkserver2
    output: 
    error: <nil>

2023-08-11 07:42:16     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osc
    output: 53bcd332-7fe5-4c05-9868-d2b0bd4c2aa4
    error: <nil>

2023-08-11 07:42:16     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo mount  /dev/osc /data/chunkserver2
    output: 
    error: <nil>

2023-08-11 07:42:16     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='/UUID=e862665f-17e9-494a-b8f9-54f3b9dd9f27/d' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:16     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='$ a UUID=53bcd332-7fe5-4c05-9868-d2b0bd4c2aa4  /data/chunkserver2  ext4  rw,errors=remount-ro  0  0  # GENERATED BY CURVEADM, DONT EDIT THIS' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo cp --no-clobber /etc/fstab /etc/fstab-2023-08-11.backup
    output: 
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo tune2fs -m 0 /dev/osc
    output: tune2fs 1.43.5 (04-Aug-2017)
Setting reserved blocks percentage to 0% (0 blocks)
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker pull  opencurvedocker/curvebs:v1.2
    output: v1.2: Pulling from opencurvedocker/curvebs
Digest: sha256:8e32478c321069c0b5c089f58adcef7c22f53399981499e7b3d5c10846f12977
Status: Image is up to date for opencurvedocker/curvebs:v1.2
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osc
    output: f64c015f-0ec3-400d-97cb-a3d78307346e
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='/UUID=e2de31f3-70a3-48fc-93bd-9c9e003874de/d' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker create --entrypoint /bin/bash --name curvebs-format-f41da9000e5270a4930662b848d85aaa --network host --rm --volume /data/chunkserver1:/curvebs/chunkserver/data opencurvedocker/curvebs:v1.2 /curvebs/tools/sbin/format.sh /curvebs/tools/sbin/curve_format 90 16777216 /curvebs/chunkserver/data/chunkfilepool /curvebs/chunkserver/data/chunkfilepool.meta 4096
    output: 6da500abbb57787e0d96765e38d70b1af4152c66a31e68281b5a951439425d27
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='$ a UUID=f64c015f-0ec3-400d-97cb-a3d78307346e  /data/chunkserver2  ext4  rw,errors=remount-ro  0  0  # GENERATED BY CURVEADM, DONT EDIT THIS' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   UploadFile
    remoteAddress: root@*:22
    localPath: /tmp/MTrHDc3i
    remotePath: /tmp/Ll1LW8gy
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo cp --no-clobber /etc/fstab /etc/fstab-2023-08-11.backup
    output: 
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo tune2fs -m 0 /dev/osc
    output: tune2fs 1.43.5 (04-Aug-2017)
Setting reserved blocks percentage to 0% (0 blocks)
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo blkid -o value -s UUID /dev/osc
    output: 5a1e44e4-9c44-451d-873f-d5bc5c196625
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='/UUID=543403c4-1a8c-4543-9336-9c88ddbeebb8/d' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker cp   /tmp/Ll1LW8gy 6da500abbb57787e0d96765e38d70b1af4152c66a31e68281b5a951439425d27:/curvebs/tools/sbin/format.sh
    output: 
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo sed --in-place --expression='$ a UUID=5a1e44e4-9c44-451d-873f-d5bc5c196625  /data/chunkserver2  ext4  rw,errors=remount-ro  0  0  # GENERATED BY CURVEADM, DONT EDIT THIS' /etc/fstab
    output: 
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo tune2fs -m 0 /dev/osc
    output: tune2fs 1.43.5 (04-Aug-2017)
Setting reserved blocks percentage to 0% (0 blocks)
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker pull  opencurvedocker/curvebs:v1.2
    output: v1.2: Pulling from opencurvedocker/curvebs
Digest: sha256:8e32478c321069c0b5c089f58adcef7c22f53399981499e7b3d5c10846f12977
Status: Image is up to date for opencurvedocker/curvebs:v1.2
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker start  6da500abbb57787e0d96765e38d70b1af4152c66a31e68281b5a951439425d27
    output: 6da500abbb57787e0d96765e38d70b1af4152c66a31e68281b5a951439425d27
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker create --entrypoint /bin/bash --name curvebs-format-f41da9000e5270a4930662b848d85aaa --network host --rm --volume /data/chunkserver1:/curvebs/chunkserver/data opencurvedocker/curvebs:v1.2 /curvebs/tools/sbin/format.sh /curvebs/tools/sbin/curve_format 90 16777216 /curvebs/chunkserver/data/chunkfilepool /curvebs/chunkserver/data/chunkfilepool.meta 4096
    output: d142ba2304707d4684de056c6f347b5c52367b8478a7015fd350eaaa9f81dd2e
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   UploadFile
    remoteAddress: root@*:22
    localPath: /tmp/GM2pzHhy
    remotePath: /tmp/4vp0U0sd
    error: <nil>

2023-08-11 07:42:17     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker cp   /tmp/4vp0U0sd d142ba2304707d4684de056c6f347b5c52367b8478a7015fd350eaaa9f81dd2e:/curvebs/tools/sbin/format.sh
    output: 
    error: <nil>

2023-08-11 07:42:18     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker start  d142ba2304707d4684de056c6f347b5c52367b8478a7015fd350eaaa9f81dd2e
    output: d142ba2304707d4684de056c6f347b5c52367b8478a7015fd350eaaa9f81dd2e
    error: <nil>

2023-08-11 07:42:19     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker pull  opencurvedocker/curvebs:v1.2
    output: v1.2: Pulling from opencurvedocker/curvebs
Digest: sha256:8e32478c321069c0b5c089f58adcef7c22f53399981499e7b3d5c10846f12977
Status: Image is up to date for opencurvedocker/curvebs:v1.2
    error: <nil>

2023-08-11 07:42:19     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker create --entrypoint /bin/bash --name curvebs-format-9e939ab24ceaaf69708ec2fb956ba69d --network host --rm --volume /data/chunkserver0:/curvebs/chunkserver/data opencurvedocker/curvebs:v1.2 /curvebs/tools/sbin/format.sh /curvebs/tools/sbin/curve_format 90 16777216 /curvebs/chunkserver/data/chunkfilepool /curvebs/chunkserver/data/chunkfilepool.meta 4096
    output: 6e5e6ccf30448e3cbbf949ddcdea594ea145b416defb174c3e7e042e5c06ab6a
    error: <nil>

2023-08-11 07:42:19     [INFO]: (log.go:101):   UploadFile
    remoteAddress: root@*:22
    localPath: /tmp/c9OtyY8Y
    remotePath: /tmp/azOB5WZ5
    error: <nil>

2023-08-11 07:42:19     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker pull  opencurvedocker/curvebs:v1.2
    output: v1.2: Pulling from opencurvedocker/curvebs
Digest: sha256:8e32478c321069c0b5c089f58adcef7c22f53399981499e7b3d5c10846f12977
Status: Image is up to date for opencurvedocker/curvebs:v1.2
    error: <nil>

2023-08-11 07:42:19     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker create --entrypoint /bin/bash --name curvebs-format-9e939ab24ceaaf69708ec2fb956ba69d --network host --rm --volume /data/chunkserver0:/curvebs/chunkserver/data opencurvedocker/curvebs:v1.2 /curvebs/tools/sbin/format.sh /curvebs/tools/sbin/curve_format 90 16777216 /curvebs/chunkserver/data/chunkfilepool /curvebs/chunkserver/data/chunkfilepool.meta 4096
    output: 985d57d76b1a380b25fbac273c7d9b1e1c055643853ead1274ded1fe09010819
    error: <nil>

2023-08-11 07:42:19     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker cp   /tmp/azOB5WZ5 6e5e6ccf30448e3cbbf949ddcdea594ea145b416defb174c3e7e042e5c06ab6a:/curvebs/tools/sbin/format.sh
    output: 
    error: <nil>

2023-08-11 07:42:19     [INFO]: (log.go:101):   UploadFile
    remoteAddress: root@*:22
    localPath: /tmp/CnVc2Eib
    remotePath: /tmp/PCl5uSKQ
    error: <nil>

2023-08-11 07:42:19     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker pull  opencurvedocker/curvebs:v1.2
    output: v1.2: Pulling from opencurvedocker/curvebs
Digest: sha256:8e32478c321069c0b5c089f58adcef7c22f53399981499e7b3d5c10846f12977
Status: Image is up to date for opencurvedocker/curvebs:v1.2
    error: <nil>

2023-08-11 07:42:19     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker start  6e5e6ccf30448e3cbbf949ddcdea594ea145b416defb174c3e7e042e5c06ab6a
    output: 6e5e6ccf30448e3cbbf949ddcdea594ea145b416defb174c3e7e042e5c06ab6a
    error: <nil>

2023-08-11 07:42:19     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker create --entrypoint /bin/bash --name curvebs-format-f41da9000e5270a4930662b848d85aaa --network host --rm --volume /data/chunkserver1:/curvebs/chunkserver/data opencurvedocker/curvebs:v1.2 /curvebs/tools/sbin/format.sh /curvebs/tools/sbin/curve_format 90 16777216 /curvebs/chunkserver/data/chunkfilepool /curvebs/chunkserver/data/chunkfilepool.meta 4096
    output: 05f74a70f694cf1d80a63e41d6d4cd3834a095fc61ed8315defa14e252dc724e
    error: <nil>

2023-08-11 07:42:20     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker cp   /tmp/PCl5uSKQ 985d57d76b1a380b25fbac273c7d9b1e1c055643853ead1274ded1fe09010819:/curvebs/tools/sbin/format.sh
    output: 
    error: <nil>

2023-08-11 07:42:20     [INFO]: (log.go:101):   UploadFile
    remoteAddress: root@*:22
    localPath: /tmp/jqQL4VuK
    remotePath: /tmp/HQPvzMW9
    error: <nil>

2023-08-11 07:42:20     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker start  985d57d76b1a380b25fbac273c7d9b1e1c055643853ead1274ded1fe09010819
    output: 985d57d76b1a380b25fbac273c7d9b1e1c055643853ead1274ded1fe09010819
    error: <nil>

2023-08-11 07:42:20     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker cp   /tmp/HQPvzMW9 05f74a70f694cf1d80a63e41d6d4cd3834a095fc61ed8315defa14e252dc724e:/curvebs/tools/sbin/format.sh
    output: 
    error: <nil>

2023-08-11 07:42:20     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker start  05f74a70f694cf1d80a63e41d6d4cd3834a095fc61ed8315defa14e252dc724e
    output: 05f74a70f694cf1d80a63e41d6d4cd3834a095fc61ed8315defa14e252dc724e
    error: <nil>

2023-08-11 07:42:21     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker pull  opencurvedocker/curvebs:v1.2
    output: v1.2: Pulling from opencurvedocker/curvebs
Digest: sha256:8e32478c321069c0b5c089f58adcef7c22f53399981499e7b3d5c10846f12977
Status: Image is up to date for opencurvedocker/curvebs:v1.2
    error: <nil>

2023-08-11 07:42:21     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker pull  opencurvedocker/curvebs:v1.2
    output: v1.2: Pulling from opencurvedocker/curvebs
Digest: sha256:8e32478c321069c0b5c089f58adcef7c22f53399981499e7b3d5c10846f12977
Status: Image is up to date for opencurvedocker/curvebs:v1.2
    error: <nil>

2023-08-11 07:42:21     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker create --entrypoint /bin/bash --name curvebs-format-5382e6eb967461d786f8744b3104e9b7 --network host --rm --volume /data/chunkserver2:/curvebs/chunkserver/data opencurvedocker/curvebs:v1.2 /curvebs/tools/sbin/format.sh /curvebs/tools/sbin/curve_format 90 16777216 /curvebs/chunkserver/data/chunkfilepool /curvebs/chunkserver/data/chunkfilepool.meta 4096
    output: 17df9dd6f4a09f45984643b0e9248116a4232607c22fa0357a44388f5eb44960
    error: <nil>

2023-08-11 07:42:21     [INFO]: (log.go:101):   UploadFile
    remoteAddress: root@*:22
    localPath: /tmp/bWOINEYp
    remotePath: /tmp/u8EEFCHs
    error: <nil>

2023-08-11 07:42:21     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker create --entrypoint /bin/bash --name curvebs-format-5382e6eb967461d786f8744b3104e9b7 --network host --rm --volume /data/chunkserver2:/curvebs/chunkserver/data opencurvedocker/curvebs:v1.2 /curvebs/tools/sbin/format.sh /curvebs/tools/sbin/curve_format 90 16777216 /curvebs/chunkserver/data/chunkfilepool /curvebs/chunkserver/data/chunkfilepool.meta 4096
    output: 519b7584648d586bbad894faf8a1305b4ce05d44a2c5851ab5d28700f0a9cb6c
    error: <nil>

2023-08-11 07:42:21     [INFO]: (log.go:101):   UploadFile
    remoteAddress: root@*:22
    localPath: /tmp/Jtrj86Sa
    remotePath: /tmp/TwHC4xLz
    error: <nil>

2023-08-11 07:42:22     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker cp   /tmp/u8EEFCHs 17df9dd6f4a09f45984643b0e9248116a4232607c22fa0357a44388f5eb44960:/curvebs/tools/sbin/format.sh
    output: 
    error: <nil>

2023-08-11 07:42:22     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker pull  opencurvedocker/curvebs:v1.2
    output: v1.2: Pulling from opencurvedocker/curvebs
Digest: sha256:8e32478c321069c0b5c089f58adcef7c22f53399981499e7b3d5c10846f12977
Status: Image is up to date for opencurvedocker/curvebs:v1.2
    error: <nil>

2023-08-11 07:42:22     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker cp   /tmp/TwHC4xLz 519b7584648d586bbad894faf8a1305b4ce05d44a2c5851ab5d28700f0a9cb6c:/curvebs/tools/sbin/format.sh
    output: 
    error: <nil>

2023-08-11 07:42:22     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker create --entrypoint /bin/bash --name curvebs-format-5382e6eb967461d786f8744b3104e9b7 --network host --rm --volume /data/chunkserver2:/curvebs/chunkserver/data opencurvedocker/curvebs:v1.2 /curvebs/tools/sbin/format.sh /curvebs/tools/sbin/curve_format 90 16777216 /curvebs/chunkserver/data/chunkfilepool /curvebs/chunkserver/data/chunkfilepool.meta 4096
    output: 378121b52366fd3695d4cdd3602532197f3eeda6cc0a1d45f164ca4e6b556954
    error: <nil>

2023-08-11 07:42:22     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker start  17df9dd6f4a09f45984643b0e9248116a4232607c22fa0357a44388f5eb44960
    output: 17df9dd6f4a09f45984643b0e9248116a4232607c22fa0357a44388f5eb44960
    error: <nil>

2023-08-11 07:42:22     [INFO]: (log.go:101):   UploadFile
    remoteAddress: root@*:22
    localPath: /tmp/mGD93Z9v
    remotePath: /tmp/nCtUXnsF
    error: <nil>

2023-08-11 07:42:22     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker start  519b7584648d586bbad894faf8a1305b4ce05d44a2c5851ab5d28700f0a9cb6c
    output: 519b7584648d586bbad894faf8a1305b4ce05d44a2c5851ab5d28700f0a9cb6c
    error: <nil>

2023-08-11 07:42:22     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker cp   /tmp/nCtUXnsF 378121b52366fd3695d4cdd3602532197f3eeda6cc0a1d45f164ca4e6b556954:/curvebs/tools/sbin/format.sh
    output: 
    error: <nil>

2023-08-11 07:42:22     [INFO]: (log.go:101):   Execute command
    remoteAddr: root@*:22
    command: sudo docker start  378121b52366fd3695d4cdd3602532197f3eeda6cc0a1d45f164ca4e6b556954
    output: 378121b52366fd3695d4cdd3602532197f3eeda6cc0a1d45f164ca4e6b556954
    error: <nil>

Strange. There seems no Docker container running.

@caoxianfei1
Copy link
Contributor

docker ps and docker ps -a all no container? it seems that docker container has started.

@caoxianfei1 caoxianfei1 reopened this Aug 11, 2023
@mrdrivingduck
Copy link
Member Author

docker ps and docker ps -a all no container? it seems that docker container has started.

Yes. Strange. Maybe the container quickly error out?

#docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

@Songjf-ttk
Copy link
Contributor

I encountered the same problem.

@caoxianfei1
Copy link
Contributor

@mrdrivingduck It seems that format progress is not running because the format container exit because some exception，so the docker ps command not show the format container details and format device usage is 0 that show Mounting status.

@mrdrivingduck
Copy link
Member Author

mrdrivingduck commented Sep 6, 2023 •
edited
Loading

@caoxianfei1 Yes, I guess so. Any idea of why format container meets exception?

@caoxianfei1
Copy link
Contributor

CURVEADM_VERSION=0906 curveadm -u. update curveadm to a temporary version that format container will not be removed when error occur. and you can lookup the container log.

@mrdrivingduck
Copy link
Member Author

@caoxianfei1 Seems the error comes from disk free space not enough, but my disk is big enough.

$ docker ps -a
CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS                      PORTS               NAMES
d3f25d149c9c        opencurvedocker/curvebs:v1.2   "/bin/bash /curveb..."   10 seconds ago      Exited (0) 9 seconds ago                        curvebs-format-5382e6eb967461d786f8744b3104e9b7
4bf5f2e20c28        opencurvedocker/curvebs:v1.2   "/bin/bash /curveb..."   13 seconds ago      Exited (0) 11 seconds ago                       curvebs-format-9e939ab24ceaaf69708ec2fb956ba69d
d0dfdd9d006e        opencurvedocker/curvebs:v1.2   "/bin/bash /curveb..."   13 seconds ago      Exited (0) 11 seconds ago                       curvebs-format-f41da9000e5270a4930662b848d85aaa
$ docker logs d3f25d149c9c
E 2023-09-07T09:43:55.473361+0800     8 curve_format_main.cpp:215] disk free space not enough.
$ cat format.yaml 
container_image: opencurvedocker/curvebs:v1.2
host:
  - server-host1
  - server-host2
  - server-host3
disk:
  - /dev/osa:/data/chunkserver0:90  # device:mount_path:format_percent
  - /dev/osb:/data/chunkserver1:90
  - /dev/osc:/data/chunkserver2:90

$ lsblk 
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
ose    252:256  0   3.5T  0 disk 
osc    252:128  0   3.5T  0 disk 
osa    252:0    0   3.5T  0 disk 
osf    252:320  0   3.5T  0 disk 
osd    252:192  0   3.5T  0 disk 
osb    252:64   0   3.5T  0 disk

@caoxianfei1
Copy link
Contributor

@mrdrivingduck It may be the percent is too big, set to 80 and retry again. we are looking into the specific reason

@caoxianfei1
Copy link
Contributor

Close it because too long time repose. you can open it again if occur.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
wait-for-reply
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Cyber-SiKu @mrdrivingduck @caoxianfei1 @Songjf-ttk