[SmartSwitch] Add graceful shutdown and startup handling in platform daemons

[vvolam]

Description

HLD: https://github.com/sonic-net/SONiC/blob/master/doc/smart-switch/graceful-shutdown/graceful-shutdown.md
These changes build upon enhancements in sonic-platform-daemons#667

This PR introduces graceful shutdown and startup orchestration across SONiC platform daemons to ensure safe DPU and peripheral module transitions during reboot or administrative state changes.

Key updates include:

• Integration of ModuleBase lifecycle methods (module_pre_shutdown, module_post_startup, and set_admin_state_gracefully) into platform daemons.
• Move graceful handling of PCIe detach/reattach and sensor reload sequences into set_admin_state_gracefully.
• State tracking in CHASSIS_MODULE_TABLE via STATE_DB to synchronize transition state across processes.
• File-based operation locks to prevent concurrent access to shared hardware resources.

Motivation and Context

Platform daemons currently perform shutdown and startup independently, leading to:

• Race conditions during DPU detachment.
• Inconsistent Redis state across PMON daemons.
• Uncoordinated sensor and PCIe transitions during reboot.

This change introduces a unified graceful shutdown framework for SmartSwitch modules.
It ensures predictable module transitions, preserves hardware health, and supports orchestrated restarts without transient hardware errors.

How Has This Been Tested?

Testing performed on both DPU-enabled (SmartSwitch).

Functional validation

• Verified end-to-end reboot flow with DPU detach/reattach sequence.
• PCIe state (detaching/attaching) reflected in STATE_DB.
• pcied daemon logs confirm ordered detach before reboot and reattach after startup.
• Confirmed no stale Redis entries or orphaned locks post-reboot.

Unit tests executed

• tests/test_DaemonPcied.py
• tests/test_chassisd_graceful.py

Coverage includes:

• Transition flag handling
• Timeout behavior
• DB write/read operations
• Graceful admin state flow

Manual validation

Additional Information (Optional)

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Copilot]

Pull Request Overview

This PR refactors module admin state management by introducing a new set_admin_state_gracefully method that encapsulates the pre-shutdown and post-startup hooks alongside the admin state change. The refactor simplifies the code by removing the ModuleTransitionFlagHelper class and duplicate logic for managing module state transitions.

• Replaces explicit module_pre_shutdown, set_admin_state, and module_post_startup calls with a single set_admin_state_gracefully method
• Removes the ModuleTransitionFlagHelper class and all transition flag tracking logic
• Updates tests to reflect the new simplified API

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
sonic-chassisd/scripts/chassisd	Removes ModuleTransitionFlagHelper class, simplifies submit_callback and submit_dpu_callback methods to use set_admin_state_gracefully, removes duplicate initialization code
sonic-chassisd/tests/mock_platform.py	Adds mock implementation of set_admin_state_gracefully method
sonic-chassisd/tests/test_chassisd.py	Updates tests to mock set_admin_state_gracefully instead of individual pre/post hooks, adjusts assertions accordingly

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[vvolam]

@rameshraghupathy @gpunathilell could you please review this latest PR

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).