Skip to content

is_warm_reboot_enabled() in xcvrd.py can not correctly detect warm reboot due to late start of pmon #531

@chiourung

Description

@chiourung

The delay of pmon is true, and pmon would start with a delay.
On a warm reboot, the value of WARM_RESTART_ENABLE_TABLE|system would be set to true, and after the WARMBOOT_FINALIZER it would be set to false.
But pmon will start after WARMBOOT_FINALIZER has finished, and is_warm_reboot_enabled() would return false.

[2024-08-07 14:33:34.623] admin@sonic:~$ redis-cli -n 6 hgetall "WARM_RESTART_ENABLE_TABLE|system"
[2024-08-07 14:33:36.695] 1) "enable"
[2024-08-07 14:33:36.695] 2) "true"
[2024-08-07 14:33:36.695] admin@sonic:~$ docker ps
[2024-08-07 14:33:38.784] CONTAINER ID   IMAGE                             COMMAND                  CREATED          STATUS         PORTS     NAMES
[2024-08-07 14:33:38.784] 42ba626dd413   docker-router-advertiser:latest   "/usr/bin/docker-ini…"   13 minutes ago   Up 2 minutes             radv
[2024-08-07 14:33:38.804] ff130d42c83e   docker-fpm-frr:latest             "/usr/bin/docker_ini…"   13 minutes ago   Up 2 minutes             bgp
[2024-08-07 14:33:38.804] 13bcfde779d8   docker-syncd-brcm:latest          "/usr/local/bin/supe…"   13 minutes ago   Up 2 minutes             syncd
[2024-08-07 14:33:38.842] c033a238d0ba   docker-teamd:latest               "/usr/local/bin/supe…"   13 minutes ago   Up 2 minutes             teamd
[2024-08-07 14:33:38.842] 52c8cfbcae85   docker-orchagent:latest           "/usr/bin/docker-ini…"   13 minutes ago   Up 2 minutes             swss
[2024-08-07 14:33:38.842] 0bf4250f1268   docker-eventd:latest              "/usr/local/bin/supe…"   13 minutes ago   Up 2 minutes             eventd
[2024-08-07 14:33:38.865] ab0e2e9b9120   docker-database:latest            "/usr/local/bin/dock…"   13 minutes ago   Up 2 minutes             database
[2024-08-07 14:33:38.865] admin@sonic:~$ redis-cli -n 6 hgetall "WARM_RESTART_ENABLE_TABLE|system"
[2024-08-07 14:33:41.034] 1) "enable"
[2024-08-07 14:33:41.034] 2) "false"
[2024-08-07 14:33:41.034] admin@sonic:~$ docker ps
[2024-08-07 14:33:43.004] CONTAINER ID   IMAGE                             COMMAND                  CREATED          STATUS         PORTS     NAMES
[2024-08-07 14:33:43.004] 5243a9eda309   docker-sonic-gnmi:latest          "/usr/local/bin/supe…"   11 minutes ago   Up 1 second              gnmi
[2024-08-07 14:33:43.025] 42ba626dd413   docker-router-advertiser:latest   "/usr/bin/docker-ini…"   13 minutes ago   Up 2 minutes             radv
[2024-08-07 14:33:43.025] ff130d42c83e   docker-fpm-frr:latest             "/usr/bin/docker_ini…"   13 minutes ago   Up 2 minutes             bgp
[2024-08-07 14:33:43.046] 13bcfde779d8   docker-syncd-brcm:latest          "/usr/local/bin/supe…"   13 minutes ago   Up 2 minutes             syncd
[2024-08-07 14:33:43.046] c033a238d0ba   docker-teamd:latest               "/usr/local/bin/supe…"   13 minutes ago   Up 2 minutes             teamd
[2024-08-07 14:33:43.046] 52c8cfbcae85   docker-orchagent:latest           "/usr/bin/docker-ini…"   13 minutes ago   Up 2 minutes             swss
[2024-08-07 14:33:43.088] 0bf4250f1268   docker-eventd:latest              "/usr/local/bin/supe…"   13 minutes ago   Up 2 minutes             eventd
[2024-08-07 14:33:43.088] ab0e2e9b9120   docker-database:latest            "/usr/local/bin/dock…"   13 minutes ago   Up 2 minutes             database
[2024-08-07 14:33:43.103] admin@sonic:~$ redis-cli -n 6 hgetall "WARM_RESTART_ENABLE_TABLE|system"docker ps
[2024-08-07 14:33:49.514] CONTAINER ID   IMAGE                                COMMAND                  CREATED          STATUS         PORTS     NAMES
[2024-08-07 14:33:49.514] c77c720ab127   docker-platform-monitor:latest       "/usr/bin/docker_ini…"   11 minutes ago   Up 2 seconds             pmon
[2024-08-07 14:33:49.537] 2965e04beadd   docker-sonic-mgmt-framework:latest   "/usr/local/bin/supe…"   11 minutes ago   Up 4 seconds             mgmt-framework
[2024-08-07 14:33:49.537] 4d653ad6155f   docker-lldp:latest                   "/usr/bin/docker-lld…"   11 minutes ago   Up 6 seconds             lldp
[2024-08-07 14:33:49.563] 5243a9eda309   docker-sonic-gnmi:latest             "/usr/local/bin/supe…"   11 minutes ago   Up 8 seconds             gnmi
[2024-08-07 14:33:49.563] 42ba626dd413   docker-router-advertiser:latest      "/usr/bin/docker-ini…"   13 minutes ago   Up 2 minutes             radv
[2024-08-07 14:33:49.604] ff130d42c83e   docker-fpm-frr:latest                "/usr/bin/docker_ini…"   13 minutes ago   Up 2 minutes             bgp
[2024-08-07 14:33:49.604] 13bcfde779d8   docker-syncd-brcm:latest             "/usr/local/bin/supe…"   13 minutes ago   Up 2 minutes             syncd
[2024-08-07 14:33:49.604] c033a238d0ba   docker-teamd:latest                  "/usr/local/bin/supe…"   13 minutes ago   Up 2 minutes             teamd
[2024-08-07 14:33:49.623] 52c8cfbcae85   docker-orchagent:latest              "/usr/bin/docker-ini…"   13 minutes ago   Up 2 minutes             swss
[2024-08-07 14:33:49.623] 0bf4250f1268   docker-eventd:latest                 "/usr/local/bin/supe…"   13 minutes ago   Up 2 minutes             eventd
[2024-08-07 14:33:49.655] ab0e2e9b9120   docker-database:latest               "/usr/local/bin/dock…"   13 minutes ago   Up 2 minutes             database

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions