Can't use ZHA mode with Xiaomi Gateway 3 (1.5.6)

[ztdan4ik]

Hi all,
I've researched all issues here and didn't find any which fits my issue.

A can't connect to getaway with ZHA, openmiio agent fails on connection from ZHA:

# /data/openmiio_agent miio mqtt cache central z3 --zigbee.tcp=8888
22:27:10.321 INF openmiio_agent version 1.2.1 linux/mipsle
22:27:10.360 INF [miio] connected to miio_client
22:27:11.380 INF [zigb] listen TCP port=8888
22:27:11.380 INF [zigb] run app=Lumi_Z3GatewayHost_MQTT
22:27:11.428 INF [miio] accept conn app=central
22:27:11.429 INF [miio] accept conn app=zigbee
22:27:11.435 INF [miio] accept conn app=gateway
22:27:11.532 WRN [cent] can't open socket
22:27:11.537 INF [mqtt] online
22:27:11.797 INF [miio] accept conn app=automation
22:27:12.788 INF [miio] accept conn app=hardware
22:27:14.790 INF [miio] accept conn app=homekit
22:27:14.792 INF [miio] accept conn app=bluetooth
22:27:28.757 INF [zigb] run app=Lumi_Z3GatewayHost_MQTT
22:27:33.455 INF [zigb] accept conn remote=192.168.1.163:56520
22:27:33.463 INF [zigb] probe fail error="wrong response" baud_rate=115200 read=0d0a4765636b6f20426f6f746c6f616465722076312e382e300d0a312e2075706c6f61642067626c0d0a322e2072756e0d0a332e2065626c20696e666f0d0a42
22:27:33.467 INF [zigb] close app=Lumi_Z3GatewayHost_MQTT
22:27:33.482 INF [zigb] probe fail error="wrong response" baud_rate=38400 read=6c6f19c8474071b83b723a71b06b1ce838ae63c41e53ee677aa3c1474071b83b723a71b06b1ce838ae63c41e53eee77aa3c1474071b83b723a71b06b1ce838ae
22:27:33.483 FTL [zigb] can't open serial error="wrong response" port=/dev/ttyS2

I've tried also on stock firmware 1.5.0 - no luck.

Any ideas how to work it out? Or where I'm wrong.
Thanks in advance.

HA: 2024.10.3
Xiaomi Gateway 3 for Home Assistant: 4.0.6
Device: Xiaomi Gateway 3
FIRMWARE: 1.5.6_0001 (custom)

UPD
Found an error in Lumi_Z3GatewayHost_MQTT

# Lumi_Z3GatewayHost_MQTT
Reset info: 11 (SOFTWARE)
gatewayInfo.txt was not found
gateInfo.Vaild = FF gateInfo.state=00
Directory "" was created failed
Begin to Reset NCP
ERROR: ezspForceReset 0x21
Lumi_Z3GatewayHost_MQTT: ../../../protocol/zigbee/app/framework/util/af-main-host.c: 382: emAfResetAndInitNCP: Assertion `0' failed.

[AlexxIT]

Probably you have broken zigbee chip firmware. You can try to flash new firmware via this integration:
https://github.com/AlexxIT/XiaomiGateway3?tab=readme-ov-file#gateway-controls

[ztdan4ik]

Probably you have broken zigbee chip firmware. You can try to flash new firmware via this integration: https://github.com/AlexxIT/XiaomiGateway3?tab=readme-ov-file#gateway-controls

Its doesn't work for me, same as in this issue #1042
So, I didn't flash my chip and running original fw. Originally I had a problem on a stock fw for getaway as well, didn't flash anything and device was able to connect with getaway.

[AlexxIT]

So you have problem on stock or not?

Originally I had a problem on a stock fw for getaway as well

device was able to connect with getaway

Show integration diagnostics.

[ztdan4ik]

@AlexxIT first of all thanks for your time!
Let me try to give you all the information:

I have:
Xiaomi Multimode Gateway (EU) (YTC4044GL)
Acara Wireless Button (WXKG12LM)
Gateway at the beginning was on stock fw 1.5.0

• First of all I discovered and added button on Mi Home with gateway
• Then I installed module Xiaomi Gateway 3 for Home Assistant
• Configure and added Gateway integration, added statistic sensors. Button was also discovered in HA.
• After several tests I found that button works unstable. Sometimes it ignores events. Hold/release event worked time to time. So I was looking for improvements.
• I've tried to configure ZHA at that time and I got an error Filed to connect (see screenshot above)
• Than I tried to flash chip from integration - got an ERROR:
  Diagnostics log file
  "192.168.1.174 [FWUP] Can't update firmware"
sock.connect((host, 8889))\nConnectionRefusedError: [Errno 111] Connection refused",
• I realized that button doesn't work in Mi Home (automations didnt trigger) although it worked time to time in HA
• I did reset gateway (10 times press button), added it to Mi Home again. And after this reset I couldn't add button anymore in Mi Home. After research I found information that Xiaomi Multimode Gateway doesn't support Acara buttons. IDK how I added it at the first time... But I gave up with it and concentrated on ZHA mode.
• I've tried to flesh chip via telnet but failed Can't upgrade Zigbee Firmware #1042
• I successfully flashed gateway FW to custom 1.5.6_0001 and nothing changed
• I also tried use zigbee2mqtt mode but zigbee2mqtt add-on also fails to start:

[2024-10-28 22:17:28] error: 	z2m: Error while starting zigbee-herdsman
[2024-10-28 22:17:28] error: 	z2m: Failed to start zigbee
[2024-10-28 22:17:28] error: 	z2m: Check https://www.zigbee2mqtt.io/guide/installation/20_zigbee2mqtt-fails-to-start.html for possible solutions
[2024-10-28 22:17:28] error: 	z2m: Exiting...
[2024-10-28 22:17:28] error: 	z2m: Error: spawn udevadm ENOENT

Zigbee2MQTT add-on config:

data_path: /config/zigbee2mqtt
socat:
  enabled: false
  master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777
  slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5
  options: "-d -d"
  log: false
mqtt:
  base_topic: zigbee2mqtt
  server: mqtt://core-mosquitto
  user: mqtt
  password: 123$mqqt@
serial:
  serial:
    port: tcp://192.168.1.174:8888
    adapter: ezsp

Mosquitto broker add-on config:

logins:
  - username: mqtt
    password: 123$mqqt@
require_certificate: false
certfile: /ssl/fullchain.pem
keyfile: /ssl/privkey.pem
customize:
  active: false
  folder: mosquitto

• At the end I followed this manual Xiaomi Multimode Gateway - compiled MultiPAN RCP firmware (EZSP Firmware: 7.3.1.0 build 0) #1328 But I couldn't configure Silicon Labs Multiprotocol addon with Device '/dev/ttyS2' and still have issues with ZHA and z2m modes.

Here is latest diagnosis file:
xiaomi_gateway3-01JBAB5ZDSY0Q0F4RDZ70WY12M-Mi Smart Home Hub-6b27152d9593e2ecd151070cb5737591-3.json

So, as result any of 3 modes doesn't for me:

1. Mi Home - Gateway doesn't support Acara button / event in HA unstable
2. ZHA - openmiio agent fails, Lumi_Z3GatewayHost_MQTT crashes in loop all the time.
3. z2m - Can't start add-on

[AlexxIT]

You gateway in some crazy state. It has broken zigbee fw. And it has broken gateway fw.
I haven't checked zigbee chip flashing logic several months. Maybe it's broken.
But first, you should to do, is to fix zigbee chip firmware.

If you really has 1.5.0 firmware (as diagnostics said), you can try to flash zigbee chip with this script
https://github.com/zvldz/mgl03_fw/tree/main/zigbee

Turn off Hass and reboot gateway before any manipulations!!!

[ztdan4ik]

Diagnostic says 1.5.0 but when I login via telnet it says FIRMWARE: 1.5.6_0001
And when I fetch device from cloud with integration it says 1.5.6

[AlexxIT]

Try to flash older gateway firmware. Linked script not support new ones.

[ztdan4ik]

A flashed gateway and zigbee chip but still cannot access with ZHA :(
Looks like there is a problem with flashing zigbee chip.
$SX -vv -X -b "$FIRMWARE_FILE" < $DEV > $DEV - syntax error: unexpected redirection

Here is outputs:

Flashing gateway output: (mgl03_1.5.0_0026_stock)

* Trying to free up space in /data

* Downloading ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 12.3M  100 12.3M    0     0   280k      0  0:00:45  0:00:45 --:--:--  350k

* Content-Length: 12917839
* Firmware size: 12917839
* Firmware size is correct
* Unpacking ...
Archive:  /data/update.zip
  inflating: full_ble_1.5.0_0102.gbl
  inflating: linux_1.5.0_0102.bin
  inflating: root_1.5.0_0102.bin

* Flashing BLE firmware
uart-dfu /dev/ttyS1 /tmp/full_ble_1.5.0_0102.gbl 1 OTA_STATE:0
do app!!
start update ble firmware!!
killall: start_ot.sh: no process killed
ota_state.txt is exist
killall: ncp_daemon: no process killed
sh: write error: Device or resource busy
sh: write error: Device or resource busy
reset in loader now
Bytes to send:377280
Syncing..DFU OK
Bootloader version: 17432581 (0x10a0005)
DFU packet size:377280
99%
finish
ncp target version:130
--200--->app ota_state:0
----->ble app updated

* Flashing kernel
Info: Open /tmp/linux_1.5.0_0102.bin, size: 2157588
Info: Firmware type: RTX Linux
Info: data_sum: cad3
Info: Upgrade linux bank 0
Info: Write to /dev/mtdblock5, size:2157572 ...
Process: 100%
Done
Info: Verify ...
Process: 100%
Success

* Flashing root
Info: Open /tmp/root_1.5.0_0102.bin, size: 10502164
Info: Firmware type: RTX Root
Info: data_sum: f527
Info: Upgrade rootfs bank 0
Info: Write to /dev/mtdblock6, size:10502148 ...
Process: 100%
Done
Info: Verify ...
Process: 100%
Success

*** Congratulations ***
Gateway will restart in 10 seconds

Flashing zigbee chip output: (ncp-uart-sw_mgl03_6_7_10_z2m)

You selected: ncp-uart-sw_mgl03_6_7_10_z2m.gbl

Preparing
Killing default zigbee software
killall: socat: no process killed
Done

Press Ctrl+C to cancel in 5 seconds!

Detected EZSP v7
Rebooting to bootloader
Sending upload command
/data/sx: line 1: syntax error: unexpected redirection
Sending restart command
Detected EZSP v7

Flashing completed!

Reload XiaomiGateway3 integration or restart Home Assistant
Configure zigbee2mqtt port address to: tcp://192.168.1.174:8888

/data/openmiio_agent miio mqtt cache central z3 --zigbee.tcp=8888

21:04:59.849 INF openmiio_agent version 1.2.1 linux/mipsle
21:04:59.880 INF [miio] connected to miio_client
21:05:00.898 INF [zigb] listen TCP port=8888
21:05:00.898 INF [zigb] run app=Lumi_Z3GatewayHost_MQTT
21:05:00.925 INF [miio] accept conn app=zigbee
21:05:00.983 INF [mqtt] online
21:05:01.824 INF [miio] accept conn app=automation
21:05:02.819 INF [miio] accept conn app=hardware
21:05:04.820 INF [miio] accept conn app=homekit
21:05:04.824 INF [miio] accept conn app=bluetooth
# Lumi_Z3GatewayHost_MQTT restarting in loop
21:05:18.759 INF [zigb] run app=Lumi_Z3GatewayHost_MQTT
21:05:36.699 INF [zigb] run app=Lumi_Z3GatewayHost_MQTT
21:05:54.639 INF [zigb] run app=Lumi_Z3GatewayHost_MQTT
# Trying to connect from ZHA
21:05:56.230 INF [zigb] accept conn remote=192.168.1.163:39368
21:05:56.247 INF [zigb] close app=Lumi_Z3GatewayHost_MQTT
21:05:59.227 INF [zigb] probe fail error="wrong response" baud_rate=115200 read=
21:06:02.227 INF [zigb] probe fail error="wrong response" baud_rate=38400 read=
21:06:02.227 FTL [zigb] can't open serial error="wrong response" port=/dev/ttyS2

Lumi_Z3GatewayHost_MQTT -n 1 -b 115200 -p /dev/ttyS2 -d /data/silicon_zigbee_host/ -r c

Reset info: 11 (SOFTWARE)
gatewayInfo.txt was not found
gateInfo.Vaild = FF gateInfo.state=00
Directory "/data/silicon_zigbee_host/" was created successfully
Begin to Reset NCP
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=0
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=0
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=0
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=0
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=1
ZIGBEE_RESET VALUE=0
ZIGBEE_RESET VALUE=1
ERROR: ezspForceReset 0x21
Lumi_Z3GatewayHost_MQTT: ../../../protocol/zigbee/app/framework/util/af-main-host.c: 382: emAfResetAndInitNCP: Assertion `0' failed.
Aborted

UPD
/bin/sx was broken, link for download this tool is invalid in script. I've download it from http://master.dl.sourceforge.net/project/mgl03/bin/sx?viasf=1
Now fw script for chip stuck on
Sending ncp-uart-sw_mgl03_6_6_2_stock.gbl, 1443 blocks: Give your local XMODEM receive command now.
Simular to the comment here #1042 (comment) but his solution doesn't work for me as Lumi_Z3GatewayHost_MQTT failed to start with error 382: emAfResetAndInitNCP: Assertion 0 failed.

UPD2
I've tried to run manually fw process:

# restart
echo -en "2" > /dev/ttyS2
sleep 5
# Kill the processes
killall socat
killall ser2net
killall daemon_app.sh
killall Lumi_Z3GatewayHost_MQTT
sleep 15
# send v7
echo -en "\x7d\x31\x43\x21\x57\x54\x2a\x12\x05\x87\x7e" > /dev/ttyS2
sleep 5
# ask2
echo -en "\x82\x50\x3A\x7E" > /dev/ttyS2
sleep 5
# Rebooting to bootloader v7
echo -en "\x22\x40\x21\x57\x54\xa5\x14\x21\x08\x7e" > /dev/ttyS2
sleep 5
# sending upload command
echo -en "1" > /dev/ttyS2
sleep 5
# upload fw
/data/sx -vv -X -b "/tmp/ncp-uart-sw_mgl03_6_6_2_stock.gbl" < /dev/ttyS2 > /dev/ttyS2

and got the same result: stuck on ...Give your local XMODEM receive command now.

[ztdan4ik]

Hi @AlexxIT
No any ideas how to flash zigbee chip in my case?

[AlexxIT]

You can send me telnet access to your gateway. You can open port on router or use ngrok (same idea but 22 port):
https://github.com/AlexxIT/go2rtc/wiki/Tunnel-RTSP-camera-to-Intenet
You can find contacts in github profile.